I'm facing an issue with my parallel-processing code that has been very frustrating (currently perl 5.8.8, but I will be pushing for 5.14.0, or newer, as soon as I can). It's frustrating in a few ways, not least of which is that it's only intermittently repeatable (it happens rarely - often rerunning without changing anything will get past the issue), it has never happened on my Linux box, only on AIX, and I can't figure out a small piece of code that repeats it.

The code that actually gets the error message is in a piece of code that pseudo-JSONises a data structure for logging purposes. Encoding to JSON-ish output is actually fairly straight-forward, and mine is recursive (a table lookup of what to do next, head to that function to handle the current item, which may call back to the table lookup for embedded objects). In this case, it's handling a simple scalar:

sub _scalar_to_string { my $s = shift; return 'null' unless defined $s; if (ref $s) # object other than plain HASH, ARRAY, or SCALAR? { #... (this is something JSON doesn't handle) } elsif (looks_like_number($s)) { return $s } else { eval { $s = "" . $s; $s =~ s[\\][\\\\]g; ### error points here $s =~ s["][\\"]g; $s =~ s[\n][\\n]g; $s =~ s[[\b]][\\b]g; $s =~ s[\f][\\f]g; $s =~ s[\r][\\r]g; $s =~ s[\t][\\t]g; $s =~ s[([^[:print:]])][sprintf "\\u%04x", ord $1]eg; 1; } and return qq["$s"]; return qq[":$@:$s"]; } }
As you can see, I've already found the error occuring, and wrapped it in an eval, so that I could log the error. The output looks like this:
End process handler: ":Modification of a read-only value attempted at +$file line 228. :172.23.1.10"
The first confusion is how the heck $s could be readonly when I've assigned to it - twice. It's not like it should be an alias of the input, even if that were readonly (which I'm not sure how that's happening either). If there's any way to eliminate this error, that would be very helpful.

The second issue is in the caller. The whole thing would likely work better in Coro than it does right now, but, since we're actually shipping this code to customers, the lawyers want to spend way too much time on "due diligence" - so I don't have approvals yet. Even AnyEvent would be an improvement, I suspect. Regardless, the basic idea is that I'm running ssh to each of a bunch of machines (via IPC::Open3), using an IO::Select loop to read stdout/stdin from each ssh, and a SIGCHLD handler to suck in the return codes. I'm thinking that the SIGCHLD must be coming back at just the wrong time, and a hash being used to track all of this is getting clobbered: a value from the hash is pushed into an array to track children that need to be handled while that value is being deleted:

# keep track of the reaped children so we can deal with th +em # later. push @reaped, delete $pids{$child};
I'm not sure if a SIGCHLD could come in to interrupt this - what it then looks like is that %pids has the key still, but the value is undef ("null" in the trace file). And when I see this in the trace file, I get the read-only error as well, even though the trace is showing that it does have the right value somehow.

Note that the stuff below doesn't quite work because JSON can't handle CODE refs as far as I can tell. Other than that, it's a close approximation of the code - but I still can't force it to fail on demand.

#!/usr/bin/perl use JSON; sub UNIVERSAL::TO_JSON { "@_" } use strict; use warnings; use Time::HiRes qw(sleep); use Scalar::Util qw(refaddr weaken readonly); use POSIX qw(:sys_wait_h); my $sub_kids = 30; $|=1; sub run { my %rcs; my %pids; my $reaper; my @reaped; $reaper = sub { my $child = shift; my $rc = shift; if ($pids{$child}) { $rcs{$pids{$child}}{rc} = $rc; print "Child: $child RC: $rc ITER: $pids{$child}\n"; # keep track of the reaped children so we can deal with th +em # later. push @reaped, delete $pids{$child}; print 'reaped list: ', encode_json(\@reaped), "\n"; # JSON + is very compact cf Data::Dump et al return 1; } return 0; }; register_sigchild($reaper); my $dealt_with = 0; my $deal_with_children = sub { ++$dealt_with; my $child = shift @reaped; if(readonly($child)) { print "********child readonly?**************\n" ; die; } print "Dealing with: ", encode_json([$child]), "\n"; print " - got: ", encode_json($rcs{$child}), "\n"# if $child; # code shouldn't make a difference here. }; for my $kid (1..$sub_kids) { my $pid = run_async($kid % 2 ? '/bin/true' : '/bin/false'); $pids{$pid} = $kid; } while (@reaped or keys %pids) { print "Waiting for children to end: ", encode_json(\%pids), "\ +n"; sleep 10 unless @reaped; while (@reaped) { $deal_with_children->(); } } unless ($dealt_with == $sub_kids) { warn "Only dealt with $dealt_with of $sub_kids?"; } unregister_sigchild($reaper); } sub run_async { # normally, I use IPC::Open3, but here we're just being quick/dirt +y. # with that, we're assuming fork will work. my $pid = fork(); return $pid if $pid; die "failed to fork" unless defined $pid; sleep 0.01 * rand(50); exec @_; die "Huh? exec failed?" } my %sigs; sub register_sigchild { my $sig = shift; $sigs{refaddr $sig} = { sig => $sig, from => (caller(1))[3], }; weaken($sigs{refaddr $sig}{sig}); $SIG{CHLD} = \&handle_sigchild; } my @buffer; sub handle_sigchild { my $child; unregister_sigchild(); while (($child = 0+ waitpid(-1, WNOHANG)) > 0) { my $rc = 0+ $?; print "Child: $child RC: $?\n"; push @buffer, [$child, $rc]; } my @exited = @buffer; @buffer = (); for my $exit (@exited) { for my $sig (values %sigs) { my $last; my $ref = $sig->{sig}; # ensure strong reference now next unless $ref; if ( eval { $last = 1 if $ref->(@$exit); 1; } ) { last if $last; } else { delete $sigs{refaddr $ref}; } # no one handled yet? push @buffer, $exit; } } unregister_sigchild(); } sub unregister_sigchild { if (@_) { delete $sigs{refaddr $_[0]} if $_[0]; $_[0] = undef; # action at a distance! } for my $addr (keys %sigs) { delete $sigs{$addr} unless defined $sigs{$addr}{sig}; } print 'About to set $SIG{CHLD}', "\n"; $SIG{CHLD} = keys %sigs ? \&handle_sigchild : 'DEFAULT'; #print 'SIGS: ', encode_json(\%sigs), "\n"; } run();

In reply to Modification of read-only value by Tanktalus

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.