Matija provides the answer for when you remove code from the project, and it addresses the situation for non infringing parts of the project. In the open source world, fixing it immediately often won't cause problems. Even in the event that a violation is found, I wouldn't be surprised if licenses already have some "procedure" for handling such violation. CC BY 4.0 (although not strictly open source), includes the following clause:
Section 6 – Term and Termination.
a) This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically.
b) Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates:
automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or
upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License.
There's a few interesting things here. When you violate the license, it is terminated, but may be reinstated when you resolve the violation. However, I want to focus on the last part of this clause, which is what I had in mind when I commented on your question - liability for before you took restorative action to fix the violation.
So... We've got a few things to consider here:
- Was the license at any point infringed?
- Was the infringement wilful?
To make a more intense example, let's imagine the following situation:
Person A has full copyright on his code. He has never released it under any license, except proprietary licenses. Person B finds some code, releases it as part of another project, and licenses it under a very permissive license - the MIT license.
You find Person B's project. You think it's great, it's easy to use, moderately active (a couple of issues/pull requests a month) and all. Best of all, you can make money easily off of it! You do some due diligence, and you're happy that the project is fine, and there are no violations that you can see. You make a project, and it's pretty well off.
Let's ask the above two questions for yourself:
- Yes, the license is definitely infringed. You can't license someone else's work, and you can't use an improperly licensed project. The MIT license is nil and void. All rights reserved are what the terms and conditions are.
- Definitely not. You've done your due diligence, and you never deliberately infringed someone's work.
Let's continue with our example.
A year passes by. Your project becomes pretty successful. Now... Person A finds Person B. In the course of find Person B, he finds you too.
Person A sends you a notice, advising you to remove all his work. He informs you of his full copyright on the work, and that he never licensed it to you, or under the MIT license. As a result of this, you remove all his work.
Does this mean you are in the clear?
Unfortunately, no. Your use, and sale of his work, caused him damages, which you are liable to. However, the situation changes, if you were never wilfully infringing. Even though you fixed the situation upon receiving notice from the copyright holder, you still did infringe copyright at some point in time.
I don't want to get into the notion of damages here, since there are so many factors to consider, statutory damages, actual damages, license fees, and all that. If you're ever in this situation, get a lawyer, and get real legal advice, because I'm not a lawyer, and nothing in this post is legal advice.
TL;DR Yes, you infringed copyright, even if you did fix it after.