| [reply] |
In the absence of any other information, I can offer the following tips:
- Get a faster server
- Process smaller files
- Write better/faster code
- Design such that you do not have to process as much
- Cache where possible
..."I don't know what the facts are but somebody's certainly going to sit down with him and find out what he knows that they may not know, and make sure he knows what they know that he may not know, and that's a good thing. I think it's a very constructive exchange," --Donald Rumsfeld
| [reply] |
It would be best to do some benchmarking and some profiling of the code before trying to optimize it. Benchmarking involves checking the execution time as a function of file size, to see if these show a linear vs. step function. Profiling involves timing specific blocks of code separately, and ranking them in terms of how much of the execution time is tied up in each block; the blocks that take the most time are the ones you want to focus on when trying to optimize. (Check out Devel::DProf)
Apart from that, some general issues to look at might include:
- If you're holding the full content of a given file in memory, maybe you shouldn't do that, because it makes the process too heavy when the file happens to be really big.
- If you're reading each file multiple times or your're reading, modifying, writing a data set in multiple passes, maybe you need to figure out an approach that only reads a file once or only does one pass over the data (and doesn't try to hold the entire file in memory).
(update: fixed wording and spelling in next-to-list item) | [reply] |
Those are pretty vague terms - are you running mod_perl or normal CGI? How big are the files? Do you have some sample code to look at? How is read in? Slurped or line by line? What OS?
Other than that...put some timings in (as mentioned above or using Time::HiRes for your own(?)) and see where the bottleneck is... then work on it. Big files (> XMB) usually will take a while, especially over CGI + Perl.. it just depends what you're doing with them. | [reply] |
- Use a faster algorithm.
- Be nice for your computer - buy it more memory for instance.
- Rewrite your program in C.
- Threaten your computer. Say that you'll rip out the DVD player if it doesn't speed up.
- Hire a more competent programmer.
- Put your computer in a really fast spaceship.
The more things you do of the above list, the better.
| [reply] |
Apart from the advice above, you may also want to take a look at FastCGI. | [reply] |