You can avoid sorting altogether. And you should.
As your records are fixed length, (barring the first and last?), there is a direct arithmetic relationship between the address of a record and its position in the file. So, use a big scalar--the most efficient form of Perl memory--and write records directly in-place. Or even directly to disk and avoid large memory usage.
As for tracking whether all the records have been written, the same arithmetic properties can be used to reduce the absence/presence of a given record, to a single bit in a bit-string. With 46-byte records, 200 MB reduces to a 1/2 MB bit-string for tracking. And this is easily and quickly checked for completion by a simple:
if( $tracking =~ m[[^\0]] ) { ## ... }
I know you are using threading. If the records of an individual file are being produced by different threads, then some care will need to be taken to ensure these large scalars are not replicate per thread. Perhaps the simplest way would be to use a Queue to a single writing and tracking thread.
In reply to Re: Storing unordered data from file in memory
by BrowserUk
in thread Storing unordered data from file in memory
by Dirk80
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |