When using a program that was compiled from open source code by a 3rd party and not by yourself directly, how is it possible to check that the binary program is well built against an exact source code version ? Or perhaps it's just impossible and you have to trust the 3rd party compilation.
The naive answer is, of course, to build it yourself and verify that your built binary is identical to the binary supplied by the other party.
So much for the theory. In practice, this can be tremendously difficult, because a body of source code can be validly compiled into an unbounded number of valid binaries, influenced by what compiler you're using, what compiler options you use, what versions of supporting system libraries you have, etc. One hands-on account of recreating binaries from source is Jos van den Oever's article "Is that really the source code for this software?", which compares self-built binaries with binaries from the repositories of popular GNU/Linux distributions. Some binaries were massively different, and even binaries that were almost identical still had differing time stamps, differing function offset values, and varying levels of debug code.
LWN.com also has an excellent summary by Jake Edge, "Verifying the source code for binaries". A key quote there:
The only reliable way to reproduce a binary is to build it in the exact same environment (including time stamps) that it was originally built in.
The article also notes that systems that care very deeply about verification, like Tor and Bitcoin, deliberately create reproducible builds. The article references a method called Gitian wherein a distributor packages that source code inside a complete virtual environment that includes the exact tools needed to reproduce a particular binary.
You should also be aware of "Trusting Trust" attacks (named after Ken Thompson's "Reflections on Trusting Trust" speech) on your compiler. The short version is: if you can't trust your compiler (because some malware is interfering with it) it is tremendously difficult to verify anything, including your compiler itself. (If you're really interested, you can also read about David A. Wheeler's double-compilation defense to this attack, and Bruce Schneiner's write-up on the subject.)