Processing is fast. I/O is slow.

Whenever someone asks me to look at their code for speedups, we often get into this discussion. For the more obstinate, I often suggest making a "do-nothing" version of their program that just reads in (files or whatever) and writes it back out again. Run this on a sample/benchmark input. We can then argue about Big-Oh numbers, and how this do-nothing version's input/output volume compares with a real example.

Results are often along these lines: "The Do Nothing version took 5 minutes to read a bazillion records and write them out again. The real program is taking an extra 5 seconds on top of that (on average). If you really need a speedup, use faster storage media."

I've always found Perl's IO to be very fast, roughly comparable to something written in C. To check, compare a "passthrough" script like:

perl -pe '' blah.in >/dev/null
[download]