in reply to Processing large files many times over

Given this little snippet, it appears that the goal of the first loop is to remove all the lines that don't end with 0 and put those into an array. The next loop then reads that array. So, you're iterating over your data twice, but you don't need to. You've got 1 completely copy of the file in memory, and another mostly completely copy. It takes time to create and destroy those copies. You don't need to do that either.
open(IN,$my_file) || die "Can't open: $!"; while (my $line = <IN>) { next unless $line =~ /0\s*$/; $line =~ /^(\d+.?\d*)//; if ($1 > .5) { # etc, etc } }
here's the break down line by line HTH

OmG! They killed tilly! You *bleep*!!