First FFmpeg can be built with different licensing depending on the build config resulting in binaries that are LGPL or GPL-licensed, but not always GPL-licensed.
Second, if you link with and call the libav* libraries from your proprietary code, the general community opinion is that you may eventually create some kind of derived of the library.
If the library is GPL-licensed then then GPL copyleft terms would flow back and apply to the calling code.
If the library is LGPL-licensed then then LGPL copyleft terms would flow back and apply to the calling code is you use some static linking and would not apply if you use dynamic linking.
In all cases, the details of which L/GPL version is in play and how you interact with FFmpeg matters a lot.
Say you use FFmpeg as-is, unmodified and call its command line tool in a subprocess, then in most cases neither a GPL nor an LGPL copyleft would back to the code that spawns the subprocess.
My understanding is that the GPL 2 licensed component does not affect my software if it is compiled/built - it would affect it if I was including some/all of the source code in my own project.
This does not sound a correct assumption to me, but there is not enough specifics in your question to be sure.
I want to bundle a built (.exe) copy FFMPEG, which is licensed under GPL 2, with my own software...But I don't want to have to open source my software project.
So based on all I said above, it depends on how you interact with FFmpeg.
And in all cases whether using the GPL or LGPL regardless of how you call FFmpeg you will need to redistribute the source of FFmpeg proper.
As a side note, there are also codec patents to consider wrt. FFmpeg especially when used in a commercial software package. You must take these into consideration too.
