It may help to read this answer to get a bit more insight into the mental model you're likely using, and how it sometimes goes wrong. In any case, it is likely that once someone has a copyright interest in a work, they retain that interest in adaptations or derivatives of the work until their interest expires due to the passage of time. No amount of incremental changes to the codebase will extinguish that interest.
Moreover, if you're using content under a free licence, you also very likely have a licence obligation to preserve these notices: your right to make a transformative work at all is conditional on (inter alia) their preservation.
Finally, the downsides of preserving all pre-existing copyright notices are very small, so I would preserve them as a matter of course.