The disassembly for both versions of the file was a 100% match, though, which is a pretty good indicator that the binary difference must be something unimportant and not related to the actual code.

It's possible that the difference is in the actual code: x86 machine code sometimes has multiple different encodings for the same assembler instruction. These would show up as single-byte differences in the binary. See http://www.strchr.com/machine_code_redundancy for some examples. Compilers might use this to stenographically hide additional data in the binary. Printer manufacturers already do something similar (https://en.wikipedia.org/wiki/Printer_steganography). I wouldn't be surprised if MS compilers embedded a hidden data in binaries -- it could be useful to track down malware authors; or identify software created with pirated MS tooling.

Perhaps this could even be used by an organization to differentiate official binaries of closed-source software from leaked binaries.

If you read the whole article, you'd see that every single bit difference is accounted for; none of the bit differences occur in the actual code.

Yes that is a good step towards fully deterministic build. There are ways to manipulate the PE file though to achieve slightly different behaviour. If you want to be thorough and make sure that your binary was not infected by malware for example then comparing the binaries makes sense.

Indeed, that part itches me a little as well.

I share the author's belief it's probably benign, and on the assumption that the source code contains no crazy obfuscated magic, it'd be pretty hard to actually hide malicious code or behaviour there.

Still if they want to be thorough (and they should), I'd have liked a bit more explanation than waving it away with a "well these bytes seem to change all the time, so that's okay then". That's not really a justification, it's just an explanation. And it's not okay.

One possible explanation is that the compiler and linker don't produce 100.00% same output because they have somewhere uninitialized space which gets written to the disk. That's why disassembly can still match 100.00%: the bytes that differ don't contribute to any observable functionality. If you know C, you can imagine how it happens:

You have somewhere char buff[ 32 ] which you don't set all to zeros, then you do strcpy( buff, "something" ) then you write all 32 bytes of buff to the file. What's behind "something"? It can be different in every run.

Now look at the picture: https://madiba.encs.concordia.ca/~x_decarn/truecrypt-binarie...

See the "RSDS"? It's some initial value, just like c:\truec... that follows some bytes later. And what's behind? Some small sequence of random bytes. I've just checked and confirmed that it appears in the ".rdata" (read only data) section of the executable.

Now the fact that this doesn't happen on much more places also means that somebody in Microsoft obviously from time to time does fully clean the code, that is, somebody worries about such effects too, but it seems that from time to time some "late" fix then slips through, misbehaving.

Another explanation would be that there is something in the building code or script that produces some different values in the initialization area. That can be then observed by source inspection.

The third explanation would be some kind of "unique id" generated by the compiler or linker. Then this effect should be observed in all binaries even when the source is fully different (e.g. compile some program which generates Vogon poetry, observe the same effect). This hypothesis doesn't match the observations, according to the pictures presented.

So I believe the highest possibility is the first assumption being true.

RSDS is part of the debug information section. The structure looks like (using the construct library's syntax):

  CV_RSDS_HEADER = Struct("CV_RSDS",
      Const(Bytes("Signature", 4), "RSDS"),
      GUID("GUID"),
      ULInt32("Age"),
      CString("Filename"),
  )

So it's a GUID, and one property of GUIDs is that they're "Globally unique", hence regenerated each time. Mystery solved.

Edit: Here's a source [1] so you don't have to take my word for it.

[1] http://www.godevtool.com/Other/pdb.htm

It seems like if you can have the tools alter the embedded timestamp (even as a post-processing step), you can match the binaries pretty closely.

The last bit is the signature, which you can't duplicate, but you can also just take out, as it's not code. But, if you're paranoid about that, zero-out or remove the signature after verifying it.