Making some assumptions and writing some 20 lines of custom import code has made the whole program roughly 3x faster (about 5 minutes per set of data files on a 900mhz athlon).