in reply to parallelism v.python

See python docs. I believe this is a problem with their hashlib implementation. (In python2.6 ?)

Replies are listed 'Best First'.
Re^2: parallelism v.python
by BrowserUk (Patriarch) on Feb 28, 2014 at 22:09 UTC

    Perhaps this is (part of) the cause?

    CPython implementation detail: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once
    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
      Holy batpoo! Yeah, that might explain a bit...
[reply]
      BTW, been thinking about this a bit...

      If it was just a matter of parallelism V non-parallelism, that would be 1 issue.

      But the 2nd big issue: Look at the total cpu time used: 

      lang    #thrds  #cores_used  %efficency   clocktime     cputime(Usr+Sy
+s)
python    9          1.82       20.2%        67.35s         122.53s
perl      9          8.70       96.7%         3.36s         29.23

      [download]
      From the above, stated as percentages or multipliers, perl is 478% more efficient in making use of multicore resources.

      In real time, python takes 64 seconds longer over the base time that was needed, of 3.36s. Python is 1900% SLOWER.

      Someone mentioned python might not be optimal in parallelism due to threading problems.

      So look at the actual amount of CPUtime used for each to do the work. 122.53/29.23. For heavy number crunching, perl (with max precision possible in x86-64 HW, takes less than 1/4th the time, i.e. perl is 4.2x the speed of python.
      ----

      (Notes collected for a response to the "said" note writer...)

[reply]
[d/l]
        Yup. https://wiki.python.org/moin/Concurrency/ applies. In short: Don't use threading on CPython unless you have to.

        Also for multiline prints, you can either wrap in brackets, use triple quotes, or use \ line continuance.

        ...and some say that Python has only one way of doing things... ;-)

[reply]

        What processor was this run on? (Can I safely assume that you were running in a non-VM environment?)

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Cool Uses for Perl