Re^2: PDL and srand puzzle - support needed in MCE

Thank you, syphilis for this thread. I did not realize this is broken. PDL overriding "srand" and not "rand" also breaks parallel code. The results are not repeatable.

use v5.030;
use PDL;
use MCE;

srand(3);

MCE->new(
  max_workers => 4, user_func => sub {
    MCE->say(MCE->wid, " ", rand());
  }
)->run;
[download]

$ perl ex.pl | sort
1 0.455586975281225
2 0.224720416431413
3 0.993853857581602
4 0.76298729873179

$ perl ex.pl | sort
1 0.778804464106546
2 0.547937905256735
3 0.317071346406923
4 0.0862047875571115

$ perl ex.pl | sort
1 0.49638699840671
2 0.265520439556898
3 0.0346538807070864
4 0.803787321857275
[download]

This requires a workaround in MCE. Ditto for MCE::Child and MCE::Hobo.

# The PDL module 2.062 ~ 2.089 exports its own srand() function, that
# silently clobbers Perl's srand function, and does not seed Perl's
# pseudo-random generator. https://perlmonks.org/?node_id=11159773
      
if ( $INC{'PDL/Primitive.pm'} && PDL::Primitive->can('srand') ) {
   # Call PDL's random() function if exported i.e. use PDL.
         
   my $caller = caller(); local $@;
   $caller = caller(1) if ( $caller =~ /^MCE/ );
   $caller = caller(2) if ( $caller =~ /^MCE/ ); 
   $caller = caller(3) if ( $caller =~ /^MCE/ );
      
   $self->{_seed} = eval "$caller->can('random')"
      ? int(PDL::Primitive::random() * 1e9)
      : int(CORE::rand() * 1e9);
}  
else {
   $self->{_seed} = int(CORE::rand() * 1e9);
}
[download]

Edit 1: MCE v1.894, MCE::Shared v1.889

if ( $INC{'PDL/Primitive.pm'} ) {
   ...
}
[download]

Edit 2: MCE configures an internal seed. It turns out that MCE may not know the srand or setter used by the application. Releasing MCE 1.895 and MCE::Shared 1.890. I updated the demonstration to process a sequence of numbers (lesser memory consumption). See also, Predictability Summary.

Reverting back to the following.

$self->{_seed} = int(CORE::rand() * 1e9);
[download]

Comment on Re^2: PDL and srand puzzle - support needed in MCE Select or Download Code

Replies are listed 'Best First'.
Re^3: PDL and srand puzzle - support needed in MCE by marioroy (Prior) on Jun 04, 2024 at 23:57 UTC
I will add PDL to the list in MCE, MCE::Child, and MCE::Hobo. # Set the seed of the base generator uniquely between workers. # The new seed is computed using the current seed and $_wid value. # One may set the seed at the application level for predictable # results (non-thread workers only). Ditto for PDL, Math::Prime::Util, # Math::Random, and Math::Random::MT::Auto. `if ( !$self->{use_threads} ) { my $_wid = $_args[1]; my $_seed = abs($self->{_seed} - ($_wid * 100000)) % 2147483560; CORE::srand($_seed); # PDL 2.062 ~ 2.089 PDL::srand($_seed) if $INC{'PDL.pm'} && PDL->can('srand'); # PDL 2.089_01+ PDL::srandom($_seed) if $INC{'PDL.pm'} && PDL->can('srandom'); ... }` [download] This resolves calling PDL->random at the application level and expecting repeatable results.	[reply] [d/l]

Replies are listed 'Best First'.

Re^3: PDL and srand puzzle - support needed in MCE
by marioroy (Prior) on Jun 04, 2024 at 23:57 UTC

I will add PDL to the list in MCE, MCE::Child, and MCE::Hobo.

# Set the seed of the base generator uniquely between workers.
# The new seed is computed using the current seed and $_wid value.
# One may set the seed at the application level for predictable
# results (non-thread workers only). Ditto for PDL, Math::Prime::Util,
# Math::Random, and Math::Random::MT::Auto.

if ( !$self->{use_threads} ) {
   my $_wid  = $_args[1];
   my $_seed = abs($self->{_seed} - ($_wid * 100000)) % 2147483560;

   CORE::srand($_seed);

   # PDL 2.062 ~ 2.089
   PDL::srand($_seed) if $INC{'PDL.pm'} && PDL->can('srand');

   # PDL 2.089_01+
   PDL::srandom($_seed) if $INC{'PDL.pm'} && PDL->can('srandom');

   ...
}
[download]

This resolves calling PDL->random at the application level and expecting repeatable results.

[reply]
[d/l]