I am currently developing a library that uses libdwarf (licensed as LGPL 2.1). Suppose I modify the CMake files to get it to build with a different MSVCRT. What do I need to release and am I required to license it?
Build scripts are part of the software. More precisely, build scripts are part of the Corresponding Machine-Readable Source Code as defined by the LGPL 2.1 (emphasis mine):
"Source code" for a work means the preferred form of the work for making modifications to it. For a library, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the library.
If you distribute your software that includes LGPL components, you can distribute your software as a whole under any terms you want. But you must provide the source code for any LGPL-covered components, and must enable recipients to replace these components. Typically, this means that LGPL components will be included as dynamically loaded libraries. Since the source code as defined by the LGPL includes build scripts, you will also have to disclose your changes to the build scripts, and you can only license your changes under the LGPL as well (or a compatible license such as GPL). Note that you have to provide the source code for LGPL components regardless of whether you modified those components.
But only if you distribute the software. The GPL/LGPL licenses do not require you to disclose changes to software that you don't share. Indeed, the goal of the GPL/LGPL licenses is not that people are forced to disclose their source code, but that end users have the freedom to run, inspect, share, and modify the software they use.