The run time for any solution will be dictated by disk I/O time. At least some options are:

• get faster disks - a raid array may help
• store less data - use binary files rather than text files
• reorganise the data to make retrieval more efficient - one file with 1000 numbers per line maybe? (I doubt this will help much in this case though.)

Of course those options aren't exclusive so you may be able to apply some or all of them. The way the files are generated may exclude some options too.