Is it legal to install and run FOSS (free open source software) in a closed operating system like Windows? If so then why did Richard Stallman felt it necessary that there "must" be a GNU operating system to run free open source software in it (and was reluctant to go with Windows OS for free open source software)?
The Windows/MS-DOS licensing terms do not apply any conditions on the software that run under Windows/MS-DOS. That is useful if you want developers to write as much software as possible for your platform – and Microsoft certainly wanted that, especially in its early time where it was far from set to become the dominant OS for personal computers. This is in contrast to iOS where the platform owner is more interested in exercising total control over the software on the platform.
Open source licenses do not prevent installation on proprietary systems. For starters, this runs into the practical problem that nearly no useful pure-FLOSS computers exist – firmware is an especially tricky case. I would argue that such requirements would also be a usage restriction that runs against the ideas of Free Software.
While the GPL licenses do not allow GPL-covered software to depend on non-free software, the licenses include an exception for system libraries. So it is perfectly fine to create a GPL covered software that can only run under Windows, or only under Solaris.
The goal of Free Software (in the sense of Stallman) is that everyone, in particular also end users have the freedom to inspect and modify the software they use. This is primarily an ethical question. But it can also be viewed as a consumer-protection effort that leverages copyright law to ensure these freedoms: software that is covered by a copyleft license cannot be included in a software that denies users these rights.
Using copyright law as the legal vehicle for these freedoms also has the practical advantage that it is internationally harmonized through the Berne Convention and WIPO treaty. In contrast, contract law differs drastically across jurisdictions, especially between common law countries like the U.S. and civil law countries like continental Europe.
When Richard Stallman came up with the idea of Free Software, no free software system was available. (BSD did reimplement proprietary Unix components, but did not have a copyleft license to ensure end user freedom, and spent a good part of its early life in limbo while its copyright status was litigated). So the GNU project set out to build such a system, including the GNU Hurd kernel and the GCC compiler. In the meanwhile proprietary platforms have to be used by definition, for example running GNU Emacs on a proprietary Unix kernel. Of course, the GNU Hurd kernel didn't get much traction and was overshot some years later by the Linux kernel, which unlike BSD was unencumbered by legal troubles.
If Free Software is good and ethical, you want as many people as possible to be able to use Free Software. Using individual Free Software programs in a non-Free environment brings users at least part of the benefits of free software. The GPL system library exception is also necessary to enable and speed up the spread of Free Software. Similarly, the linking exception in the LGPL can enable the use of Free Software in contexts where Free Software would otherwise not be able to gain any traction.
Is it legal to install and run FOSS (free open source software) in a closed operating system like Windows?
Absolutely yes. Even a strong copyleft license like the GNU GPL, which requires derivative works to be under the same GPL terms whenever they are distributed, does not extend to the operating system on which GPL-licensed userland software is run, because userland software does not create a derivative work with the operating system. In the case that a piece of software must be linked with a proprietary (or otherwise GPL-incompatible) system library -- in which case it may indeed form a derivative work -- the GPL explicitly exempts this action as allowed. Permissively-licensed software carries even fewer requirements, and may absolutely be run on proprietary operating systems as well.
It is possible for an proprietary operating system to impose requirements against the execution of freely-licensed programs: e.g., the terms of service for Apple's App Store effectively forbid the distribution of GPL-licensed software on its platform, and iOS restricts the execution of binaries from outside the App Store. However, this is a limitation that would be imposed by the legal terms of the operating system itself.
why did Richard Stallman felt it necessary that there "must" be a GNU operating system to run free open source software in it (and was reluctant to go with Windows OS for free open source software)?
For one, Windows did not exist in 1983 at the time GNU was announced, but it is worth considering his objection to proprietary operating systems in general, which certainly extended to Windows when it was released in 1985. We may consider what he has to say about proprietary UNIX systems that were contemporaneous with GNU's initial development; his objections are ideological and will apply equally well to all proprietary operating systems.
Your question is the central focus of Stallman's GNU Manifesto in which he details the reasons for writing GNU and includes defenses for various arguments against GNU and free software in general. For the purposes of your specific question, though, I think the main point may be answered like this:
Why did Stallman feel it necessary to write a free-software operating system? This had nothing to do with the fact that free software cannot be run on a proprietary operating system (as mentioned above, it can). Instead, Stallman believed it was necessary to build GNU for the same reason he believed it necessary to write every other piece of free software, namely, to avoid the use of proprietary software that could not be freely shared, inspected, and modified. This requirement of Stallman's applies no differently to a text editor, to a music player, or to an operating system.
It is perhaps worth noting that, insofar as GNU is a re-implementation of UNIX, early GNU developers did indeed use a proprietary UNIX system to perform their initial software development of GNU. They gradually replaced the proprietary UNIX system, component by component, until it consisted entirely of free software. [I recall reading about this on the FSF's or GNU's website; I'd be grateful if someone could find the original passage I'm paraphrasing here.]