«Storing all data in a hash, so this might break down once your files become huge.»

Mmh, why not (storing all data in a hash)?

I take a look at my box @home. It has 8 GBytes RAM. Sometime ago something like this was a high-end server, short time later a high-end work station and now it is a standard desktop solution:

Karls-Mac-mini:monks karl$ top
PhysMem: 833M wired, 1541M active, 260M inactive, 2634M used, 5558M fr
+ee.
[download]

So i don't have any pain in the ass to store my data in a big hash on my box.

And if i have such huge files, i have another problem.

Best regards, Karl