Recipients would not be able to compile the project and obtain the binaries themselves because of the missing library A.
If you are distributing and including GPL code in your source code distribution (which you are, according to your comments), then to fulfill the GPL requirements you must distribute all source code. If a certain part of the source code (e.g. library A) is not available, then you cannot distribute that GPL component while also fulfilling its requirements. This is what is meant when we say a license is "incompatible" -- it means you tried to fulfill all the requirements (e.g. license A says you cannot distribute its source code, and license B says you must distribute all source code), but failed to do so.
A possible exception would be so-called "system libraries" or components, which are described in the following paragraph of the GPL v2:
The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.
The above quotation (emphasis added) is from the GPLv2, but similar language ("System Libraries") is in the GPLv3 as well.
Based on this, if you were developing, say, a Windows application, you would need not include, for example, the source code to the Windows modules that are normally distributed with the operating system, compiler, kernel, or other major components of the Windows operating system.
However, seeing as you intend to distribute the executable version of library A along with your program, and seeing as the library A does not qualify as "a major component of the compiler, kernel, and so on, of the operating system on which the executable runs," I would say that this this exception does not apply in your case. So you will need to be able to distribute its source code in order to comply with the GPL.
Based on your comments, you are using the FFTW library, which is specifically licensed under the GPL along with a dual-licensing offer. See the FFTW FAQ. It is common to license something under a GPL license as well as a non-GPL license for the use case you describe (for producing non-free software). In this case your only recourse is to ask the author for an alternative (non-GPL) license, or to remove dependence on the library A, which seems to be what is mainly preventing you from distributing all the source code and thus complying with the GPL.
