in reply to Re: Linux sort or FILE::SORT
in thread Linux sort or FILE::SORT

It is actually quite easy to beat gnusort for performance.

Firstly it uses miniscule buffers relative to modern memory sizes necessitating far more write read cycles than could ever be optimum.

Secondly, with most machines having multiple cores these days, it is silly not to utilise them.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^3: Linux sort or FILE::SORT
by Utilitarian (Vicar) on Feb 08, 2011 at 13:41 UTC
    It does take the -S $BUFFER_SIZE flag.

    Are sort operations usually CPU bound?

    print "Good ",qw(night morning afternoon evening)[(localtime)[2]/6]," fellow monks."
[reply]
[d/l]
      It does take the -S $BUFFER_SIZE flag.

      I guess you must have a later version of gnu sort than I? 

      C:\test>sort --version
sort (GNU textutils) 2.0
Written by Mike Haertel.

      [download]
      Are sort operations usually CPU bound?

      The in-memory sort stages certainly are. If you overlap the sorting of one batch with the reading of the second, you save some time.

      Have one thread reading/or writing and 1/3/7/15/... threads sorting, you save more time.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
        The in-memory sort stages certainly are

        When sorting big data memory bandwidth may also be a limiting factor.

        Actually it would be interesting to see how parallel sorting algorithms behave on multi-core processors with shared caches.

[reply]
        sort (coreutils) 5.2.1
Written by Mike Haertel and Paul Eggert.

        [download]
        print "Good ",qw(night morning afternoon evening)[(localtime)[2]/6]," fellow monks."
[reply]
[d/l]
[select]
        sort (GNU textutils) 2.0 is 10 years old 
        $ C:/PROGRA~1/gnuwin32/bin/sort.EXE --version
sort (GNU coreutils) 5.3.0

$ C:\msys\1.0\bin\sort.exe --version
sort (GNU coreutils) 5.97

        [download]
[reply]
[d/l]

In Section Seekers of Perl Wisdom