This is akin to putting a basket with apples, no let's make this pears, out on the street and put a price tag on them.
Two days later, you decide that the price tag was silly and double it.
Does that mean that anybody can now take the same pears as were on offer for half the price at the original price? No. But someone who bought half the basket under the old conditions can put up his own basket with your original pears next to it and undercut your new price.
The license is not an inherent part of the software, just like the price tag is not an inherent part of the pears. But if somebody got the software under the GPLv3, they can redistribute that version and copies of it under the GPLv3, possibly subverting your own new desires for licensing. Because that is one right that you granted them by using the GPLv3 on the previous version of the software.
But nobody can demand that you put back the old license on copies you choose to distribute, whether they are a new version or actually the old one.
In theory, you can distribute software under a restrictive license even while every header of the software and the included license file state differently. In practice, you'll have a hard time proving that this kind of distribution was not misleading to the customer to the degree where your claims of damages are moot.
So while you do not need to remove evidence your software ever was licensed differently, you should make sure to counter the plausible impression that it is being distributed still under the old license by you.