Your problem lies in your loop. The Angle operator (i.e. <IN>) works in two contexts: list and scalar. In list context, it returns all the lines of the file as a list; In scalar context, it reads and returns one line at a time.

So:

for (<IN>) {
   # Do some stuff;
}
[download]

Reads all the files from IN into an array, then processes them one at a time. You want:

while (<IN>) {
   # Do some stuff;
}
[download]

In the above, the angle operator is called in scalar context, so only one file line will be read during each loop iteration. That should (I hope) solve your memory and timing problems.

Update: I see that I type too slowly. ;-)