P is for Practical | |
PerlMonks |
comment on |
( [id://3333]=superdoc: print w/replies, xml ) | Need Help?? |
> Depending upon the data of course, your HoH (hash of hash) structure could consume quite a bit more memory than the actual file size in MB.
This shouldn't be a problem if you a apply a sliding window technique° plus splitting the hashes into easily swappable chunks². The trick is to balance time, space and disk access, by minimizing the the number of swaps. This will scale well, until the limit given by disk-space.
Cheers Rolf °) see ²) seeIn reply to Re^3: Get unique fields from file
by LanX
|
|