In law, C has committed a copyright violation. Nothing they do now can undo that fact and any of the copyright holders of the GPL code has a very good chance of winning if they sue for damages.
Does the company C has an option to cease all distribution of the binary, get rid of the library dependency, and release a new version of the binary?
They certainly should do this ASAP; courts generally look a lot less favourably if a party continues to commit an offence once they have become aware of it. However and to re-emphasise the point: doing this does not make good the violation; C can still be sued for damage caused up to this point.
Or is it required to release the whole source code under GPL?
Almost certainly not. While the concept of specific performance does exist in law, this is a case where damages are a sufficient remedy so a court would be incredibly unlikely to order this. (Disclaimer: my knowledge is mostly based on UK law, with a bit of an extension to other legal systems in the same family. If you care about a specific jurisdiction, consult a lawyer in that jurisdiction)
What are ways to prevent such risks?
Properly audit the dependencies you are using. If you don't have the expertise to do this internally, there are plenty of companies who will do it for you.
Stepping back slightly here, unless this is an exceptional case (e.g. something like Apple discovering a GPL library which is crucial to all functionality of the iPhone), the actual damage caused here is likely to be small. Proactively contacting the authors of the GPL code and offering to make a donation to the project is probably the best solution for everyone, except the lawyers who now won't get lots of fees.
(Yes, I also know the above paragraph is probably naive and realistically what most companies will do is just to silently stop distributing the GPL code and hope nobody ever notices / actually sues. That would be unethical though)