in reply to Re^10: Sorting a (very) large file (bigger and faster?)
in thread Sorting a (very) large file

I'm not convinced. I think you're neglecting to consider just how much slower disks are than memory. If you can sort the array directly in memory (which I originally thought was possible) but you're close to the memory limit (which I also thought was likely) then you'd be pretty foolish to use more memory (and thus start swapping) just to try to save time.

If that bothers you sufficiently, here's how you can convince me - write some benchmarking code. I want to see a benchmark that shows that, even if you have to go significantly into swap to do it, that doing a parallel sort is worth it absolutely.

Who knows, you might be right - I've learned to distrust my gut when it comes to performance. I find hard numbers much more convincing than argument.

If you don't feel like spending the time I'll understand. I certainly don't.

-sam

  • Comment on Re^11: Sorting a (very) large file (bigger and faster?)