in reply to Re^2: Need to Improve Scraping Speed
in thread Need to Improve Scraping Speed

I already gave you that answer after I gave you a program to scrape the data once and only once

The solution is to scrape the data once and only once, repackage it, compress it, and host it as a few compressed files on an ftp server.

CGI is the wrong way to distribute this data, and you shouldn't distribute this scraper program, its like distributing like a denial of service tool.