in reply to instances of CGI in web server

I don't think there is some magical way around this problem. If you need to serve thousands of requests in parallel, you'll need to have the respective server resources to do so... (This might mean having a cluster of machines with load sharing, in case an individual machine can't take it.)

Switching from CGI to some persistent environment (like mod_perl, FastCGI) essentially only reduces the startup overhead per request. This might, however, help to some degree, if the actual request handling only takes a short time. In other words, if the startup makes up a substantial part, reducing it can help increase the overall server throughput...