Embedding a Checksum in an Image File

On 20/04/2023 18:45, Rick C wrote:
> On Thursday, April 20, 2023 at 11:33:28 AM UTC-4, George Neuner
> wrote:
>> On Wed, 19 Apr 2023 19:06:33 -0700 (PDT), Rick C 
>> <gnuarm.del...@gmail.com> wrote:
>> 
>>> This is a bit of the chicken and egg thing. If you want a embed
>>> a checksum in a code module to report the checksum, is there a
>>> way of doing this? It's a bit like being your own grandfather, I
>>> think.
>> Take a look at the old xmodem/ymodem CRC. It was designed such
>> that when the CRC was sent immediately following the data, a
>> receiver computing CRC over the whole incoming packet (data and CRC
>> both) would get a result of zero.
>> 
>> But AFAIK it doesn't work with CCITT equation(s) - you have to use 
>> xmodem/ymodem.
>>> I'm not thinking anything too fancy, like a CRC, but rather a
>>> simple modulo N addition, maybe N being 2^16.
>> Sorry, I don't know a way to do it with a modular checksum. YMMV,
>> but I think 16-bit CRC is pretty simple.
>> 
>> George
> 
> CRC is not complicated, but I would not know how to calculate an
> inserted value to force the resulting CRC to zero.  How do you do
> that?

You "insert" the value at the end.  Anything else is insane.

CRC's are quite good hashes, for suitable sized data.  There are perhaps 
some special cases, but basically you'd be doing trial-and-error 
searches to find an inserted value that gives you a zero CRC overall. 
2^16 is not an overwhelming search space, but the whole idea is pointless.

> 
> Even so, I'm not trying to validate the file.  I'm trying to come up
> with a substitute for a time stamp or version number.  I don't want
> to have to rely on my consistency in handling the version number
> correctly.  This would be a backup in case there was more than one
> version released, even only within the "lab", that were different.  A
> checksum that could be read by the controlling software would do the
> job.

A CRC is fine for that.

> 
> I have run into this before, where the version number was not a 100%
> indication of the uniqueness of an executable.  The checksum would be
> a second indicator.
> 
> I should mention that I'm not looking for a solution that relies on
> any specific details of the tools.
> 

A table-based CRC is easy, runs quickly, and can be quickly ported to 
pretty much any language (the C and Python code, for example, is almost 
the same).

On Thu, 20 Apr 2023 09:45:59 -0700 (PDT), Rick C
<gnuarm.deletethisbit@gmail.com> wrote:

>On Thursday, April 20, 2023 at 11:33:28?AM UTC-4, George Neuner wrote:
>> On Wed, 19 Apr 2023 19:06:33 -0700 (PDT), Rick C 
>> <gnuarm.del...@gmail.com> wrote: 
>> 
>> >This is a bit of the chicken and egg thing. If you want a embed a 
>> >checksum in a code module to report the checksum, is there a way of 
>> >doing this? It's a bit like being your own grandfather, I think.
>>
>> Take a look at the old xmodem/ymodem CRC. It was designed such that 
>> when the CRC was sent immediately following the data, a receiver 
>> computing CRC over the whole incoming packet (data and CRC both) would 
>> get a result of zero. 
>> 
>> But AFAIK it doesn't work with CCITT equation(s) - you have to use 
>> xmodem/ymodem.
>> >I'm not thinking anything too fancy, like a CRC, but rather a simple 
>> >modulo N addition, maybe N being 2^16.
>> Sorry, I don't know a way to do it with a modular checksum. 
>> YMMV, but I think 16-bit CRC is pretty simple. 
>> 
>> George
>
>CRC is not complicated, but I would not know how to calculate an
>inserted value to force the resulting CRC to zero.  How do you do
>that? 

It's implicit in the equation they chose.  I don't know how it works -
just that it does.


You have some block of data   |....data....|

You compute CRC on the data block and then append the resulting value
to the end of the block.  xmodem CRC is 16-bit, so it adds 2 bytes to
the data.

So now you have a new extended block   |....data....|crc|

Now if you compute a new CRC on the extended block, the resulting
value /should/ come out to zero. If it doesn't, either your data or
the original CRC value appended to it has been changed/corrupted.


>Even so, I'm not trying to validate the file.  I'm trying to come up
>with a substitute for a time stamp or version number.  I don't want
>to have to rely on my consistency in handling the version number
>correctly.  This would be a backup in case there was more than one
>version released, even only within the "lab", that were different.  A
>checksum that could be read by the controlling software would do the
>job.  

I've actually done this: in the early 90s I designed a system that
used a CRC based scheme to identify load modules and track
inter-module code dependencies.  

I computed both 16-bit Xmodem and CCITT CRCs on the modules and
concatenated the two values into a 32-bit identifier.  That identifier
then was used to sign the module and to demand load (or unload) it
when needed.

At the time it worked quite well: the system had quite limited memory,
so code modules were small enough that even a 16-bit CRC could
uniquely identify most/all of them. Combining the two different CRCs
into a 32-bit identifier provided more than enough uniqueness, it was
fast and easy to compute, and it saved a lot of space vs using
something with stronger guarantees like a UUID or crypto-strength
signing hash.
[A lot of the hashing functions available today either didn't exist or
just weren't widely known back then.  And still most of them that even
have 32-bit variants are weak in guarantees for those variants.]


>I have run into this before, where the version number was not a 100%
>indication of the uniqueness of an executable.  The checksum would be
>a second indicator. 

I made it the basis of dependency checking. Version numbers were
secondary and for the benefit of the programmer.


>I should mention that I'm not looking for a solution that relies on
>any specific details of the tools. 

YMMV.
George

On Thu, 20 Apr 2023 20:17:07 +0300, Tauno Voipio
<tauno.voipio@notused.fi.invalid> wrote:

>The method to check for a proper constant value after the whole
>block and CRC are received and put through the generator works
>with the CRC-CCITT (actually ITU-T). The proper final value
>depends on the initial CRC and whether the CRC is inverted before
>sending. The limitation is that the CRC has to be sent least
>significant octet first.
>
>For a reference, see RFC1662, Appendix C.

I remember seeing an explanation of it decades ago, but I never would
have been able to find it again.

Thanks,
George

On 4/19/23 10:06 PM, Rick C wrote:
> This is a bit of the chicken and egg thing.  If you want a embed a checksum in a code module to report the checksum, is there a way of doing this?  It's a bit like being your own grandfather, I think.
> 
> I'm not thinking anything too fancy, like a CRC, but rather a simple modulo N addition, maybe N being 2^16.
> 
> I keep thinking of using a placeholder, but that doesn't seem to work out in any useful way.  Even if you try to anticipate the impact of adding the checksum, that only gives you a different checksum, that you then need to anticipate further... ad infinitum.
> 
> I'm not thinking of any special checksum generator that excludes the checksum data.  That would be too messy.
> 
> I keep thinking there is a different way of looking at this to achieve the result I want...
> 
> Maybe I can prove it is impossible.  Assume the file checksums to X when the checksum data is zero.  The goal would then be to include the checksum data value Y in the file, that would change X to Y.  Given the properties of the module N checksum, this would appear to be impossible for the general case, unless...  Add another data value, called, checksum normalizer.  This data value checksums with the original checksum to give the result zero.  Then, when the checksum is also added, the resulting checksum is, in fact, the checksum.  Another way of looking at this is to add a value that combines with the added checksum, to be zero, leaving the original checksum intact.
> 
> This might be inordinately hard for a CRC, but a simple checksum would not be an issue, I think.  At least, this could work in software, where data can be included in an image file as itself.  In a device like an FPGA, it might not be included in the bit stream file so directly... but that might depend on where in the device it is inserted.  Memory might have data that is stored as itself.  I'll need to look into that.
> 


IF I understand you correctly, what you want is for the file to compute 
to some "checksum" that comes from the basic contents of the file, and 
then you want to add the "checksum" into the file so the program itself 
can print its checksum.

One fact to remember, is that "cryptographic hashes" were invented 
because it was too easy to create a faked file that matches a 
non-crptographic hash/checksum, so that couldn't be a key to make sure 
you really had the right file in the presence of a determined enemy, but 
the checksums were good enough to catch "random" errors.

This means that you can add the checksum into the file, and some 
additional bytes (likely at the end) and by knowing the propeties of the 
checksum algorithm, compute a value for those extra bytes such that the 
"undo" the changes caused by adding the checksum bytes to file.

I'm not sure exactly how to computes these, but the key is that you add 
something at the end of the file to get the checksum back to what the 
original file had before you added the checksum into the file.

On Thursday, April 20, 2023 at 10:09:35 PM UTC-4, Richard Damon wrote:
> On 4/19/23 10:06 PM, Rick C wrote: 
> > This is a bit of the chicken and egg thing. If you want a embed a checksum in a code module to report the checksum, is there a way of doing this? It's a bit like being your own grandfather, I think. 
> > 
> > I'm not thinking anything too fancy, like a CRC, but rather a simple modulo N addition, maybe N being 2^16. 
> > 
> > I keep thinking of using a placeholder, but that doesn't seem to work out in any useful way. Even if you try to anticipate the impact of adding the checksum, that only gives you a different checksum, that you then need to anticipate further... ad infinitum. 
> > 
> > I'm not thinking of any special checksum generator that excludes the checksum data. That would be too messy. 
> > 
> > I keep thinking there is a different way of looking at this to achieve the result I want... 
> > 
> > Maybe I can prove it is impossible. Assume the file checksums to X when the checksum data is zero. The goal would then be to include the checksum data value Y in the file, that would change X to Y. Given the properties of the module N checksum, this would appear to be impossible for the general case, unless... Add another data value, called, checksum normalizer. This data value checksums with the original checksum to give the result zero. Then, when the checksum is also added, the resulting checksum is, in fact, the checksum. Another way of looking at this is to add a value that combines with the added checksum, to be zero, leaving the original checksum intact. 
> > 
> > This might be inordinately hard for a CRC, but a simple checksum would not be an issue, I think. At least, this could work in software, where data can be included in an image file as itself. In a device like an FPGA, it might not be included in the bit stream file so directly... but that might depend on where in the device it is inserted. Memory might have data that is stored as itself. I'll need to look into that. 
> >
> IF I understand you correctly, what you want is for the file to compute 
> to some "checksum" that comes from the basic contents of the file, and 
> then you want to add the "checksum" into the file so the program itself 
> can print its checksum. 
> 
> One fact to remember, is that "cryptographic hashes" were invented 
> because it was too easy to create a faked file that matches a 
> non-crptographic hash/checksum, so that couldn't be a key to make sure 
> you really had the right file in the presence of a determined enemy, but 
> the checksums were good enough to catch "random" errors. 
> 
> This means that you can add the checksum into the file, and some 
> additional bytes (likely at the end) and by knowing the propeties of the 
> checksum algorithm, compute a value for those extra bytes such that the 
> "undo" the changes caused by adding the checksum bytes to file. 
> 
> I'm not sure exactly how to computes these, but the key is that you add 
> something at the end of the file to get the checksum back to what the 
> original file had before you added the checksum into the file.

Yeah, for a simple checksum, I think that would be easy, at least if "checksum" means a bitwise XOR operation.  If the checksum and extra bytes are both 16 bits, this would also work for an arithmetic checksum where each 16 bit word were added into the checksum.  All the carries would cascade out of the upper 16 bits from adding the inserted checksum and it's 2's complement.  

I don't even want to think about using a CRC to try to do this. 

-- 

  Rick C.

  +- Get 1,000 miles of free Supercharging
  +- Tesla referral code - https://ts.la/richard11209

On 4/20/2023 1:44 PM, George Neuner wrote:
> You have some block of data   |....data....|
> 
> You compute CRC on the data block and then append the resulting value
---------------------------------------------^^^^^^
> to the end of the block.  xmodem CRC is 16-bit, so it adds 2 bytes to
> the data.

Exactly.  You *don't* drag the "extra bits" into the initial
CRC calculation but *do* into the CRC *verification*.  Easy
peasy (since forever).

[Think about it:  your performing a division operation
and the residual is the "remainder".]

Note that you want to choose a polynomial that doesn't
give you a "win" result for "obviously" corrupt data.
E.g., if data is all zeros or all 0xFF (as these sorts of
conditions can happen with hardware failures) you probably
wouldn't want a "success" indication!

You can also "salt" the calculation so that the residual
is deliberately nonzero.  So, for example, "success" is
indicated by a residual of 0x474E.  :>

> So now you have a new extended block   |....data....|crc|
> 
> Now if you compute a new CRC on the extended block, the resulting
> value /should/ come out to zero. If it doesn't, either your data or
> the original CRC value appended to it has been changed/corrupted.

As there is usually a lack of originality in the algorithms
chosen, you have to consider if you are also hoping to use
this to safeguard the *integrity* of your image (i.e.,
against intentional modification).

I have an old Compaq Portable 386 (lunchbox) that obviously
wasn't designed to support the disk drives that I would
*later* install in it.  So, I patched the BIOS ROMs to
add another disk type to the Disk Parameter Table.  Then,
made compensating changes to other parts of the ROM
(that I knew would not be referenced) to ensure the original
checksum -- WHEREVER IT MAY HAVE BEEN "STORED" -- would remain
intact.

[I could similarly have altered the boot message to show
a copyright of "Don Y" in place of "Compaq" -- as it would
be pretty easy to locate the "Compaq" string (in plaintext)
in the image.]

On Thursday, April 20, 2023 at 12:06:36 PM UTC+10, Rick C wrote:
> This is a bit of the chicken and egg thing. If you want a embed a checksum in a code module to report the checksum, is there a way of doing this? It's a bit like being your own grandfather, I think. 
> 
> I'm not thinking anything too fancy, like a CRC, but rather a simple modulo N addition, maybe N being 2^16. 
> 
> I keep thinking of using a placeholder, but that doesn't seem to work out in any useful way. Even if you try to anticipate the impact of adding the checksum, that only gives you a different checksum, that you then need to anticipate further... ad infinitum. 
> 
> I'm not thinking of any special checksum generator that excludes the checksum data. That would be too messy. 
> 
> I keep thinking there is a different way of looking at this to achieve the result I want... 
> 
> Maybe I can prove it is impossible. Assume the file checksums to X when the checksum data is zero. The goal would then be to include the checksum data value Y in the file, that would change X to Y. Given the properties of the module N checksum, this would appear to be impossible for the general case, unless... Add another data value, called, checksum normalizer. This data value checksums with the original checksum to give the result zero. Then, when the checksum is also added, the resulting checksum is, in fact, the checksum. Another way of looking at this is to add a value that combines with the added checksum, to be zero, leaving the original checksum intact. 
> 
> This might be inordinately hard for a CRC, but a simple checksum would not be an issue, I think. At least, this could work in software, where data can be included in an image file as itself. In a device like an FPGA, it might not be included in the bit stream file so directly... but that might depend on where in the device it is inserted. Memory might have data that is stored as itself. I'll need to look into that. 
> 
> -- 
> 
> Rick C. 
> 
> - Get 1,000 miles of free Supercharging 
> - Tesla referral code - https://ts.la/richard11209
Rick, What is the purpose of this?  Is it (1) to be able to externally identify a binary, as one might a ROM image by computing a checksum?  Is it (2) for a run-able binary to be able to check itself?  This would of course only be able to detect corruption, not tampering.  Is it (3) for the loader (whatever that might be) to be able to say 'this binary has the correct checksum' and only jump to it if it does?  Again this would only be able to detect corruption, not tampering.  Are you hoping for more than corruption detection?

On 21/04/2023 07:37, Don Y wrote:
> On 4/20/2023 1:44 PM, George Neuner wrote:
>> You have some block of data   |....data....|
>>
>> You compute CRC on the data block and then append the resulting value
> ---------------------------------------------^^^^^^
>> to the end of the block.  xmodem CRC is 16-bit, so it adds 2 bytes to
>> the data.
> 
> Exactly.  You *don't* drag the "extra bits" into the initial
> CRC calculation but *do* into the CRC *verification*.  Easy
> peasy (since forever).
> 
> [Think about it:  your performing a division operation
> and the residual is the "remainder".]

George's earlier posts made it look like the algorithm was inserting 
("embedding") a value somewhere inside the image, so that the CRC over 
the modified image was zero.  This is easy to do for simple checksums 
such as XOR's or a sum-of-bytes checksum, but infeasible for CRC's.

It is a much easier matter when appending the checksum.  Depending 
somewhat on the details of the CRC (such as bit/byte reversals, 
inversions, starting values, etc.) it is typically the case that for a 
binary blob A, crc(A ++ crc(A)) = 0.  i.e., if you append the CRC of 
your data to the data, the CRC of the whole thing is 0.

Of course, this is pretty much irrelevant - whether you check the 
integrity of the final image by running CRC over it all and comparing to 
0, or running it over all but the last word and comparing to the last 
word is a minor matter.

> 
> Note that you want to choose a polynomial that doesn't
> give you a "win" result for "obviously" corrupt data.
> E.g., if data is all zeros or all 0xFF (as these sorts of
> conditions can happen with hardware failures) you probably
> wouldn't want a "success" indication!

No, that is pointless for something like a code image.  It just adds 
needless complexity to your CRC algorithm.

You should already have checks that would eliminate an all-zero image or 
other "obviously corrupt" data.  You'll be checking the image for a key 
or "magic number" that identifies the image as "program image for board 
X, project Y".  You'll be checking version numbers.  You'll be reading 
the length of the image so you know the range for your CRC function, and 
where to find the appended CRC check.  You might not have all of these 
in a given system, but you'll have some kind of check which would fail 
on an all-zero image.

> 
> You can also "salt" the calculation so that the residual
> is deliberately nonzero.  So, for example, "success" is
> indicated by a residual of 0x474E.  :>
> 

Again, pointless.

Salt is important for security-related hashes (like password hashes), 
not for integrity checks.

>> So now you have a new extended block   |....data....|crc|
>>
>> Now if you compute a new CRC on the extended block, the resulting
>> value /should/ come out to zero. If it doesn't, either your data or
>> the original CRC value appended to it has been changed/corrupted.
> 
> As there is usually a lack of originality in the algorithms
> chosen, you have to consider if you are also hoping to use
> this to safeguard the *integrity* of your image (i.e.,
> against intentional modification).
> 

"Integrity" has nothing to do with the motivation for change. 
/Security/ is concerned with intentional modifications that deliberately 
attempt to defeat /integrity/ checks.  Integrity is about detecting any 
changes.

If you are concerned about the possibility of intentional malicious 
changes, CRC's alone are useless.  All the attacker needs to do after 
modifying the image is calculate the CRC themselves, and replace the 
original checksum with their own.

Using non-standard algorithms for security is a simple way to get things 
completely wrong.  "Security by obscurity" is very rarely the right 
answer.  In reality, good security algorithms, and good implementations, 
are difficult and specialised tasks, best left to people who know what 
they are doing.

To make something secure, you have to ensure that the check algorithms 
depend on a key that you know, but that the attacker does not have. 
That's the basis of digital signatures (though you use a secure hash 
algorithm rather than a simple CRC).

On 4/21/2023 3:43 AM, David Brown wrote:
>> Note that you want to choose a polynomial that doesn't
>> give you a "win" result for "obviously" corrupt data.
>> E.g., if data is all zeros or all 0xFF (as these sorts of
>> conditions can happen with hardware failures) you probably
>> wouldn't want a "success" indication!
> 
> No, that is pointless for something like a code image.  It just adds needless 
> complexity to your CRC algorithm.

Perhaps you've forgotten that you don't just use CRCs (secure hashes, etc.)
on "code images"?

> You should already have checks that would eliminate an all-zero image or other 
> "obviously corrupt" data.  You'll be checking the image for a key or "magic 
> number" that identifies the image as "program image for board X, project Y".  
> You'll be checking version numbers.  You'll be reading the length of the image 
> so you know the range for your CRC function, and where to find the appended CRC 
> check.  You might not have all of these in a given system, but you'll have some 
> kind of check which would fail on an all-zero image.

See above.

>> You can also "salt" the calculation so that the residual
>> is deliberately nonzero.  So, for example, "success" is
>> indicated by a residual of 0x474E.  :>
> 
> Again, pointless.
> 
> Salt is important for security-related hashes (like password hashes), not for 
> integrity checks.

You've missed the point.  The correct "sum" can be anything.
Why is "0" more special than any other value?  As the value is
typically meaningless to anything other than the code that verifies
it, you couldn't look at an image (or the output of the verifier)
and gain anything from seeing that obscure value.

OTOH, if the CRC yields something familiar -- or useful -- then
it can tell you something about the image.  E.g., salt the algorithm
with the product code, version number, your initials, 0xDEADBEEF, etc.

>>> So now you have a new extended block   |....data....|crc|
>>>
>>> Now if you compute a new CRC on the extended block, the resulting
>>> value /should/ come out to zero. If it doesn't, either your data or
>>> the original CRC value appended to it has been changed/corrupted.
>>
>> As there is usually a lack of originality in the algorithms
>> chosen, you have to consider if you are also hoping to use
>> this to safeguard the *integrity* of your image (i.e.,
>> against intentional modification).
> 
> "Integrity" has nothing to do with the motivation for change. /Security/ is 
> concerned with intentional modifications that deliberately attempt to defeat 
> /integrity/ checks.  Integrity is about detecting any changes.
> 
> If you are concerned about the possibility of intentional malicious changes, 

Changes don't have to be malicious.  I altered the test procedure for a
piece of military gear we were building simply to skip some lengthy tests that 
I *knew* would pass (I don't want to inject an extra 20 minutes of wait time
just to get through a lengthy test I already know works before I can get
to the test of interest to me, now.

I failed to undo the change before the official signoff on the device.

The only evidence of this was the fact that I had also patched the
startup message to say "Go for coffee..." -- which remained on the
screen for the duration of the lengthy (even with the long test
elided) procedure...

..which alerted folks to the fact that this *probably* wasn't the
original image.  (The computer running the test suite on the DUT had
no problem accepting my patched binary)

> CRC's alone are useless.  All the attacker needs to do after modifying the 
> image is calculate the CRC themselves, and replace the original checksum with 
> their own.

That assumes the "alterer" knows how to replace the checksum, how it
is computed, where it is embedded in the image, etc.  I modified the Compaq
portable mentioned without ever knowing where the checksum was store
or *if* it was explicitly stored.  I had no desire to disassemble the
BIOS ROMs (though could obviously do so as there was no "proprietary
hardware" limiting access to their contents and the instruction set of
the processor is well known!).

Instead, I did this by *guessing* how they would implement such a check
in a bit of kit from that era (ERPOMs aren't easily modified by malware
so it wasn't likely that they would go to great lengths to "protect" the
image).  And, if my guess had been incorrect, I could always reinstall
the original EPROMs -- nothing lost, nothing gained.

Had much experience with folks counterfeiting your products and making
"simple" changes to the binaries?  Like changing the copyright notice
or splash screen?

Then, bringing the (accused) counterfeit of YOUR product into a courtroom
and revealing the *hidden* checksum that the counterfeiter wasn't aware of?

"Gee, why does YOUR (alleged) device have *my* name in it -- in addition
to behaving exactly like mine??"

[I guess obscurity has its place!]

Use a non-secret approach and you invite folks to alter it, as well.

> Using non-standard algorithms for security is a simple way to get things 
> completely wrong.  "Security by obscurity" is very rarely the right answer.  In 
> reality, good security algorithms, and good implementations, are difficult and 
> specialised tasks, best left to people who know what they are doing.
> 
> To make something secure, you have to ensure that the check algorithms depend 
> on a key that you know, but that the attacker does not have. That's the basis 
> of digital signatures (though you use a secure hash algorithm rather than a 
> simple CRC).

If you can remove the check, then what value the key's secrecy?  By your
criteria, the adversary KNOWS how you are implementing your security
so he knows exactly what to remove to bypass your checks and allow his
altered image to operate in its place.

Ever notice how manufacturers don't PUBLICLY disclose their security
hooks (without an NDA)?  If "security by obscurity" was not important,
they would publish these details INVITING challenges (instead of
trying to limit the knowledge to people with whom they've officially
contracted).

[If it was so good and they were trying to rely on trade secret, why
not just PATENT their approach, also disclosing it in the process?
Surely, the details will "leak" from one of the NDA signers long
before patent protection would expire...  And, presumably, these
are "people who know what they are doing"...]

Sign all the binaries and all I have to do is remove the *test* for
those signatures and the images can be as corrupted as I choose.

You need to "secure" the test if you want the image to be securable.
This is why it is so hard to use "open" security protocols on
hardware devices (cuz there are almost always ways to subvert the
verification process/hardware).  Having physical access to a device
usually means it can be compromised -- if worth your effort.

[The trick is to make the effort great enough to be on a par with
just copying the *functionality*, from scratch, and not bothering
trying to alter the executable in a way that is not detectable]

[[There are companies who's business models are exactly that -- cloning
other products (e.g., from folks like big blue) at the functional
level -- yet steering clear of any copyright issues.]]

On Friday, April 21, 2023 at 4:53:18 AM UTC-4, Brian Cockburn wrote:
> On Thursday, April 20, 2023 at 12:06:36 PM UTC+10, Rick C wrote: 
> > This is a bit of the chicken and egg thing. If you want a embed a checksum in a code module to report the checksum, is there a way of doing this? It's a bit like being your own grandfather, I think. 
> > 
> > I'm not thinking anything too fancy, like a CRC, but rather a simple modulo N addition, maybe N being 2^16. 
> > 
> > I keep thinking of using a placeholder, but that doesn't seem to work out in any useful way. Even if you try to anticipate the impact of adding the checksum, that only gives you a different checksum, that you then need to anticipate further... ad infinitum. 
> > 
> > I'm not thinking of any special checksum generator that excludes the checksum data. That would be too messy. 
> > 
> > I keep thinking there is a different way of looking at this to achieve the result I want... 
> > 
> > Maybe I can prove it is impossible. Assume the file checksums to X when the checksum data is zero. The goal would then be to include the checksum data value Y in the file, that would change X to Y. Given the properties of the module N checksum, this would appear to be impossible for the general case, unless... Add another data value, called, checksum normalizer. This data value checksums with the original checksum to give the result zero. Then, when the checksum is also added, the resulting checksum is, in fact, the checksum. Another way of looking at this is to add a value that combines with the added checksum, to be zero, leaving the original checksum intact. 
> > 
> > This might be inordinately hard for a CRC, but a simple checksum would not be an issue, I think. At least, this could work in software, where data can be included in an image file as itself. In a device like an FPGA, it might not be included in the bit stream file so directly... but that might depend on where in the device it is inserted. Memory might have data that is stored as itself. I'll need to look into that. 
> > 
> > -- 
> > 
> > Rick C. 
> > 
> > - Get 1,000 miles of free Supercharging 
> > - Tesla referral code - https://ts.la/richard11209
> Rick, What is the purpose of this? Is it (1) to be able to externally identify a binary, as one might a ROM image by computing a checksum? Is it (2) for a run-able binary to be able to check itself? This would of course only be able to detect corruption, not tampering. Is it (3) for the loader (whatever that might be) to be able to say 'this binary has the correct checksum' and only jump to it if it does? Again this would only be able to detect corruption, not tampering. Are you hoping for more than corruption detection?

This is simply to be able to say this version is unique, regardless of what the version number says.  Version numbers are set manually and not always done correctly.  I'm looking for something as a backup so that if the checksums are different, I can be sure the versions are not the same.  

The less work involved, the better. 

-- 

  Rick C.

  ++ Get 1,000 miles of free Supercharging
  ++ Tesla referral code - https://ts.la/richard11209