If I release software under the GNU GPL v3, and a new version of the GPL is released, can I change the license to the new version if I choose to without having to say "GNU GPL v3.0 or later"?
If you are the copyright owner then you always have complete freedom at any time to change the license to anything you want. (This only affects people who download the project after the change, those who downloaded it before can continue to use it under the old license.) If other people have contributed code and there are multiple owners then you need to get everyone's approval to change the license.
"Or later" phrases allow other people to adopt a later license without you doing anything. In doing so you put your trust in the writers of the license that they won't fundamentally change it.
As curiousdannii said you can always release your own code under another license, even if it's incompatible with a license you used earlier. Furthermore, specifically with GPL versions, the FSF recommends that programs say “Version 3 of the GPL or any later version”, and in that case you can effectively even relicense other people's code under a new version: namely, if you use some code that's licensed under GPL3 or newer and combine it with any own code under GPL4, the resulting project will be under GPL4, legally speaking (even if most of the code still says GPL3).