For the GPL version 3, if you provide a binary only distribution (a "Non-Source Form" according to the license), you need to provide the corresponding source "from the same place" -- if you merely link to the project download page or a site containing archives of several versions of the source code (one of which corresponds to the version you used), that is not sufficient.
One reason is that the GPL specifically requires you provide the "Corresponding Source" -- for example if you use library version 1.2.3 to build your product, you need to provide the exact same version in your source code distribution (not 1.2.4, not 1.2.2, not 1.2.3a, etc.). If all you do is link to an archive page where I can download (say) 10 different source code releases, how would I know which one corresponds to the one that your binary distribution uses?
However, if you provide a single page which provides multiple download links together -- one to your binary version and then one or more separate (direct) download links to the corresponding source(s), and it is made clear which source code corresponds to which binary release, I would consider that "the same place" and I believe it would satisfy the license, even if some links went to different servers.
The relevant information is from the GPL version 3, section 6 (Conveying Non-Source Forms). It gives the rules you must follow when distributing binaries:
You may convey a covered work in object code form under the terms of
sections 4 and 5, provided that you also convey the machine-readable
Corresponding Source under the terms of this License, in one of these
ways:
[...]
d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. [...] If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements.
https://www.gnu.org/licenses/gpl-3.0.html
I added bolding to the parts you need to pay attention to. In particular, if you go with this option, you need to ensure the link to download the corresponding source is valid as long as you are distributing the binary.
