in reply to Module development: concurrent versions (Updated)

If you were in Java I would say, load each class in a seperate sibling class loader and put any classes shared in common into a parent class loader. Java identifies classes using class loader + name, so even if two classes have the same name if you've loaded them into sibling class loaders Java won't confuse their inheritance chains and inter-class relationships.

Although Java is not Perl, and Perl is not Java, it appears you can use Perl threads in much the same way as the Java class loaders. Each thread gets its own copy of @INC and %INC. Furthermore, each thread inherits the contents of @INC and %INC from its parent thread, as illustrated in this little demo:

use strict;
use warnings;
use threads;
use threads::shared;

our @STARRING_CLASSES=qw(Carp.pm List/Util.pm Test/More.pm);

my @aBig;
my $aBig=bless(\@aBig,"MaybeSharable");
my $aBigShared=&share($aBig);

my $t;

require Carp;

$t = threads->new(sub {
                    require List::Util;
                    print "-------------------------------------\n"
                      , "tid=".threads->tid
                      . " \\\%INC=".\%INC
                      . "  \\\@INC=".\@INC."\n";
                    while(my($k,$v)=each(%INC)) {
                      $k="**$k" if grep{$k eq $_} @STARRING_CLASSES;
                      print "$k=$v\n"
                    };
                    print "\n";
                  });
$t->join();

$t = threads->new(sub {
                    require Test::More;
                    print "-------------------------------------\n"
                      , "tid=".threads->tid
                      . " \\\%INC=".\%INC
                      . "  \\\@INC=".\@INC."\n";
                    while(my($k,$v)=each(%INC)) {
                      $k="**$k" if grep{$k eq $_} @STARRING_CLASSES;
                      print "$k=$v\n"
                    };
                    print "\n";
                  });
$t->join();

[download]

Given that, why not just use two sibling threads in your program and load one version in thread 1 and the other in thread 2?

Update: changed choice of classes to use in demo so that there is at least one package each thread loads that the other does not. (Turns out Scalar::Util pulls in List::Util, the two modules used in the original version - who knew!).

Replies are listed 'Best First'.
Re^2: Module development: concurrent versions (Updated)
by BrowserUk (Patriarch) on Dec 24, 2010 at 00:16 UTC
    why not just use two sibling threads in your program and load one version in thread 1 and the other in thread 2?

    Cool thought++ But ... :)

    I need to instrument (with trace) the same (address for address identical) array using both versions of total_size().

    This is impossible if the two routines are loaded into different threads.

    It's still good to see someone else around here considering threads for a possible solution, rather than a problem :)

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

      Sadly it appears that there is no way to trick threads::shared into allowing a shared array reference. Perhaps an internals person might have an idea of how to get around that without causing ithreads to die, refuse to work with your array, or silently convert the array reference into a different address per thread.

      Clearly threads::shared has a way of knowing that array ref X on thread 1 is equivalent to array ref X on thread 2 because it has to know both of them in order to keep the two arrays in sync. Also the main reason for ithreads to refuse to share array references is concern that data not meant to be shared will be inadvertantly shared and lead to nasty race conditions and total nonsense as two threads try to insert elements into the same data structure using non-atomic operations. However, if an array and all of its elements are marked readonly in the C guts, then that should not be an issue. The question is there a way to get ithreads to agree it isn't an issue without rewriting part of ithreads? And further, would your tests be reliable if they ran on such a read-only marked array?

      But before wading deep into ithread internals, perhaps it is worth asking: what is the reason behind needing to instrument the exact same address? Is this because you don't trust the results unless they are physically working off the same spot in memory? Or is it because you can't properly scan the instrumentation output if addresses are different.

      If it is for ease of scanning (either by human or computer), I wonder if you could just come up with a way of transforming the trace output so that each stream replaces its own address with a like placeholder?

[reply]
        what is the reason behind needing to instrument the exact same address?

        The problem I am trying to trace is entirely dependant upon the internal handling of addresses.

        My essential contribution to the module revolves around replacing the use of a hash to track whether the components of complex data-structure hierarchies had already be traversed--and therefore counted--, with a bit vector where each possible address in the virtual memory space is mapped to a single bit. This involves some notionally straight forward masking & shifting operations on the actual addresses of Perl's many internal data structures.

        The complexities arise because of the need to perform those operations in terms of the size of those pointers--32 or 64-bit--as determined at runtime.

        This is further complicated by the fact that MS compilers use the LLP64 programming model, but gcc uses the LP64 programming model. (See [OT] LLP64 .v. LP64 portability for a little more background on this.) And this is further complicated because there are no cross-compiler compatible types for performing the math safely.

        As the addresses used in the allocations for large arrays is influenced by what has been previously allocated, there is no way to map the addresses of a large array allocated in one script to the addresses allocated for an apparently identical array in another script.

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom