This is a little tricky because the LGPL and AGPL are not strictly compatible. For LGPL code that is linked to AGPL code, compatibility is not required, because LGPL-licensed can be linked to anything, even proprietary software, without causing a licensing issue. However, for LGPL code that is fully transplanted into a larger software project, such that it becomes a part of that larger project (instead of merely used by it as a library), compatibility is required.
There is a path forward, though: you must distribute your changed LGPL files under the GPLv3, and then you may combine those GPLv3 files under the AGPLv3. This is described in brief in a footnote in the GPL compatibility matrix:
LGPLv2.1 gives you permission to relicense the code under any version of the GPL since GPLv2. If you can switch the LGPLed code in this case to using an appropriate version of the GPL instead (as noted in the table), you can make this combination.
The LGPLv2.1-to-GPL reliciensing permission is described in LGPLv2.1 section 3:
- You may opt to apply the terms of the ordinary GNU General Public License instead of this License to a given copy of the Library. To do this, you must alter all the notices that refer to this License, so that they refer to the ordinary GNU General Public License, version 2, instead of to this License. (If a newer version than version 2 of the ordinary GNU General Public License has appeared, then you can specify that version instead if you wish.) Do not make any other change in these notices.
Upgrading from LGPL2.1 to GPLv3 is important because GPLv3 and AGPLv3 are one-way compatible. You can include GPLv3 work within an AGPLv3 work, per the exception in AGPLv3 section 13:
Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the work with which it is combined will remain governed by version 3 of the GNU General Public License.
Therefore, for any parts of microhttpd that you incorporate directly into your AGPL project (directly, not used as a library), you must change their licensing headers to state their licensing as GPLv3. This change in licensing is allowed because the author has specifically allowed it under the above permission in LGPLv2.1. You do not need to mention that those files were formerly under the LGPL, since the upgrade process is explicitly detailed in the license and does not require any such notice.
With all that said, let's address your questions:
Can I just get rid of LGPL license?
For the files you directly incorporate into your project, you must replace the LGPL with the GPLv3. For other parts of the LGPL library that your work uses indirectly as a library, no change is necessary. Those can remain safely under the LGPL.
Can I change the file header to AGPL?
After you change microhttpd's LGPL license to GPLv3, you can make your changes under the AGPL. In that case, include a notice that some parts are licensed under the GPL, and some parts are licensed under the AGPL. Do your best to describe which parts are licensed which way, and include notices for both licenses in your header. For simplicity, you may wish to license your changes to that file under the GPLv3, but according to the GPL compatibility matrix, you can make your version of the file licensed under the AGPLv3 as a whole, with some GPLv3 parts (even if those GPL parts make up a majority of the work).
Where is the right place to mention that my code is a modified version of LGPL code, and link the microhttpd page?
There is no need to link to the microhttpd project, though doing so would always be a polite thing to do. You are only required to preserve license and copyright notices, along with a notice explaining how you changed the files. You can include the how-you-changed information in any reasonable place, either in the individual file headers, or in your project's README.