In addition to the excellent answers above, your practice of reading the whole file into an array could be causing a performance issue. Your process might be paging like mad.
Just processing one record at a time, if that is possible, could speed it up, or maybe using
tie?