Re: Slow at sorting?

Perl's sorting algorithim is basically a quicksort. I don't know exactly what your data looks like but quicksort usually behaves very badly with nearly sorted data. This is for a logfile correct? It's possible that it's nearly sorted.

Another possibility. When faced with sorting an 800 Meg Search index, I gave up using pure Perl and just used system to punt it off to GNU sort. It went from over 8 hours to around 30 minutes. I had no desire to recreate the functionality of handling files this size. I just needed it to work.

You also might want to check out Mastering Algorithims in Perl by O'Reilly. It has a really good chapter on sorting.

-Lee

"To be civilized is to deny one's nature."

Comment on Re: Slow at sorting?

Replies are listed 'Best First'.
Re (tilly) 2: Slow at sorting? by tilly (Archbishop) on Nov 22, 2001 at 01:56 UTC
Perl's sorting algorithm is a modified quicksort which chooses elements in a pattern which makes it fast on sorted data sets. There are slow patterns, which is why Perl 5.8 will default to a mergesort. But the slow patterns are a lot harder to hit than just sorting the data.	[reply]
Re(3): Slow at sorting? by FoxtrotUniform (Prior) on Nov 22, 2001 at 02:39 UTC
Out of curiosity, why mergesort over (for example) heapsort? `-- :wq`	[reply]
Re (tilly) 4: Slow at sorting? by tilly (Archbishop) on Nov 22, 2001 at 03:21 UTC
I think that Abigail gave the the best explanation of why heapsort would not be a good choice. For more, including an analysis of how to get bad numbers out of Perl 5.6, try the whole thread.	[reply]
Re: Re (tilly) 2: Slow at sorting? by shotgunefx (Parson) on Nov 22, 2001 at 02:15 UTC
Thanks tilly. Didn't know that. Learn something new everyday. -Lee "To be civilized is to deny one's nature."	[reply]