Can you check if your computer started swapping memory to disk when doing the sort? I assume that the sort will need additional memory to hold intermediary results and you could easily cross that magic boundary where you swap memory to disk.

"Cleaning up" would then involve reading the data back from disk to fill the memory again with the data for other application which were temporarily swapped out of your computer's memory when the sort ran.