The only "value" your script adds to the data is the reformulation of the employee number. You might consider a multi-pass approach. First, write a simple script that reformulates the employee number and builds the sort key, then write the result into a temporary table, with line looking likes this:   sortable-key original-data-with-modified-empnumber Since you're processing a line at a time, the memory footprint will be small.

Next, use an external sort program to sort the temporary file. A good sort program can handle data that won't otherwise fit into virtual memory.

Finally, write a small script that strips the sortable-key from the file. Clean up your temporary files, and Voila!, you're done.

This is essentially a Schwartzian Transform using intermediate files.