You remember mostly correctly.

The GPL says nothing about whether a C runtime has been GPLed. The relevant section is part of section 0: Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. However gcc is designed so that the binary it outputs does not incorporate copyrighted GPLed material. Therefore you can use gcc to compile non-GPLed material.

Whether you can legally use Microsoft's compiler to compile GPLed software depends on whether the output includes material that Microsoft has copyright on. The answer to that is not easily discerned.