in reply to Thread sharing a bit vector??

There is nothing in this memory-intensive operation that will benefit in any way from multiprogramming. Therefore, scrap that idea. Now your program is simple again.

Replies are listed 'Best First'.
Re^2: Thread sharing a bit vector??
by BrowserUk (Patriarch) on Apr 11, 2012 at 23:45 UTC

    Balderdash, poppycock & bunkum.

    If s/he needs to do a full cross-intersection between 3500 bitstrings, 6.125 million operations, then on one thread it will take 6.125e6*t seconds.

    Do it on 4 threads and it can take 1/4 that time. The source vectors are read-only for teh intersection operation, so don't require locking. It only requires subdividing the results vectors logically to avoid thread/data overlaps, and they require no locking either.

    Perl isn't the right language for this type of work at this time, but advising against multiprogramming because you're personally scared of it is just malicious.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

      The difference is "shared," not "threading." As many threads as you have CPUs, but any sort of locking requirement would knock that advantage almost completely out. Not a good Perl candidate.
        The difference is "shared," not "threading."

        With the (slight; and possibly implied) modification of s/"shared"/threads::shared/, we agree.

        Though I disagree with "any sort of locking requirement would knock that advantage almost completely out".

        Mutual exclusion to prevent concurrent access can be achieved far more cheaply than it is currently implemented in Perl. Sufficiently cheaply in the no-contention case, to allow locking to be almost costless for low-but-not-zero contention algorithms.

        Indeed, in my experiments with lock-free and wait-free algorithms, I've found that "locking" algorithms can be more efficient than either, for many applications, where: a) the no-contention branch avoids dips into the kernel; b) the contend branch spins a little before deciding it needs to wait (and therefore dips into hte kernel).

        Of course, that doesn't work so well for long-term locks, but that is an architectural problem rather than a fundamental algorithmic one.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?