The GPL license doesn't have any specific requirements to make you enforce the license and the copyright holder is the only person who can take any action.
The copyright holder is the one who is legally authorized to take
action to enforce the license.
(Violations of the GNU Licenses - GNU.ORG)
So only something in base copyright law could cause you to lose your rights from non-enforcement. Short answer: No, you won't lose your copyright if you do not defend it. That is something that applies to a trademark.
In the United States, if a trademark owner stops using a mark without
intending to resume using it in the future, the mark will be deemed
abandoned. Under U.S. trademark law, non-use of a trademark for three
consecutive years creates a rebuttable presumption of abandonment of
the mark (whether or not it is registered). Temporary or seasonal
non-use of a trademark, particularly when such cyclical use is typical
in an industry or market, generally does not constitute grounds for
abandonment under U.S. law. Rather, periods of non-use must be
accompanied by the “intent not to resume use.”
In many other countries, grounds for abandonment depend solely on
whether the mark at issue has been in use, without regard to the
trademark owner’s intent. The period of non-use required to show
abandonment of a mark typically ranges from three years (e.g., in
Australia, Japan, South Korea, Canada, China, Russia and many Latin
American countries) to five years (e.g., in various European
countries, such as Ireland and Norway).
(Fact Sheet: Protecting a Trademark - International Trademark Association).
Another reason you may be unable to defend your IP rights after a long period of not enforcing it is what is called a laches defense. This is when there has been an unreasonable amount of time in pursing a claim. Laches is an affirmative defense and must be brought forth and proven by the defendant. The US Supreme court has stated that laches does not bar copyright infringement claims (Petrella vs MGM).
The majority rejected those arguments, however, saying copyright law
was clear in allowing lawsuits up to three years after an infringing
act, setting up a rolling period for owners like Petrella to sue
although they can only collect on profits earned during that
There is “nothing untoward” about waiting to see if an infringer’s
investment paid off, Ginsburg wrote. In fact, the law “allows a
copyright owner to defer suit until she can estimate whether
litigation is worth the candle.”
The Supreme Court’s decision “brings certainty to this issue, which
was unique in the Ninth Circuit,” said William Kane
(Unusual Split As Supreme Court Upholds `Raging Bull' Suit Vs. MGM - Forbes)
So there isn't anything different with someone telling you that someone has infringed on your copyright. You aren't required to enforce it. You can probably even wait and decide later that you want to enforce it but you can only do that within three years of the last infringing act and you can only expect damages based upon the last three years.
More useful info about copyright myths (including this): http://www.templetons.com/brad/copymyths.html
I Am Not A Lawyer... if you want real legal advice you should contact one.