in reply to alternatives to emailing the data back to the user
MIME::Lite allows you to specify alternate mail transports to sendmail, and will even work on systems like stock Win32 systems that don't have any built-in mail transport.
Another alternative is to FTP the data somewhere. If your site's user has an open FTP space, you could use Net::FTP to send the data at a later time. Or, you could make it available for him to pick up within your FTP area, although that solution seems less ideal than simply putting it in your webspace and providing the user a link to use to retrieve it once it's done.
I wonder if it would be possible for you to preprocess at least some of what the user will be after, so that processing time on demand doesn't reach ten hours. If some portions of what the user will need are predictable, you could run a cron-job to keep those portions preprocessed, and then pull the pieces together on demand in quicker time. I don't know if this is feasable, but maybe...
Another possibility is that if your processing time is constrained by the fact that you're pulling data from other websites (a time-consuming process because connections are not instantaneous), maybe you could fork multiple connections simultaneously to speed up the data-retrieval process. Take care not to bog yourself down though; if you have dozens or more CGI processes each forking dozens or more times, you'll get into trouble quickly.
It's wrong for us to assume that anything can be done to speed up processing time, but it doesn't hurt for us to offer suggestions just in case something can actually be done. ;)
Dave
|
|---|