TLDR; You must distribute any derivative work in source code form such that the originally licensed portion of the derivative work continues to be bound by (at least) one of those two licenses.
Since you have the option of which of two licenses to use, you can choose to conform to the restrictions of the less restrictive of the two licenses. However, the two licenses say basically the same thing about relicensing. Both state that you must include that license as part of any redistribution of the work.
The Apache license says:
You must give any other recipients of the Work or Derivative Works a copy of this License
So that one's pretty clear. The way I read this, the MIT license will continue to apply to the portion of a derived source code distribution that was initially bound by the license. The MIT license takes a little more thought to come to the same conclusion...
The MIT license states that the Copyright Notice and the Permission Notice from the license must be included in any distributed derivations of the work. But if you look closely, you'll discover that the MIT license consists of exactly those two notices. By that reasoning, it also requires that you distribute any derivative works such that the MIT license continues to apply to that portion which was originally licensed by the MIT license.
So neither license allows you to remove it from derivative source code distributions, and so it must apply to the derivative work.