I'm assuming the GPL version 3 for the purposes of this answer.
From section 8 of the GPL:
You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11).
However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.
The existence of the GPL code in your closed-source program is a violation of the GPL, and terminates your license to use the GPL code. However, if you find it and remove it, you're no longer violating the license and can use it once again.
As to your work, the GPL license doesn't cover it, and has no legal bearing on it if there is no interaction between the GPL code and your program. If the GPL code is included, then the rest of your work must also be distributed under the GPL. If the GPL code is then removed, you can distribute your work how you like.
However, you should note that if the copyright owner pushes a court case, you may end up having to publish your source code. Free Radical's answer here goes into more detail about the possible legal proceedings.