Hi, I'm writing a multi-threaded web spider. I've written a threading pool with a dispatcher for processes which works very well.
My problem is getting the children to modify a global array. I've tried using the IPC::Shareable module, with no avail. Then i tried this IPC::ShareLite. I use it exactly as documented, and I'm getting an error. Maybe someone can help me out with this.
Here's some code: 
use IPC::ShareLite;
use Storable qw(freeze thaw);
use POSIX qw(:sys_wait_h);

my $share = new IPC::ShareLite( -key     => 'link',
                                -create  => 'yes',
                                -destroy => 'yes',
                                -size    => 262144,
                                -mode    => 0660,
                              ) or die "Cannot create segment: $!";

my $arrayref;   # the array i want to share with my children
push @{$arrayref}, 'some value';
$share->store(freeze($arrayref) );

    if ($pid = fork()) { 
#************* parent
        ++$num_procs;
        ++$flag;
        print "$num_procs processes running\n";

      } elsif (defined $pid) {
#************* child 
        my $alllinks;
        my @temp_array;

        $alllinks = thaw( $share->fetch() ); # fetch the global array 
+from memory    THIS IS THE LINE THAT'S GIVING ME AN ERROR 

        foreach $blah (@hrefs) {
          $blah = url($blah, $base)->abs;
          if ($blah =~ m/^http:(.*)[|\.]$localdomain[\/|:|]/i)  {
            $found = 0;

            for (@{$alllinks}) {
              (split /^/, $_)[0] eq $blah and $found++, last
            }

            if ($found == 0) {
              push @temp_array, $blah;
            }
          }
        }

        $share->lock("LOCK_EX");  # obtain an exclusive lock on the sh
+ared memory segment 
        push @{$alllinks}, @temp_array;  # modify the global array
        $share->store( freeze($alllinks)); # store it back into shared
+ memory 

        $share->lock("LOCK_UN"); # unlock it 
        exit; # always exit the child
      } else {
# can't fork
      }

[download]
Basically that's all it does. When i execute it, It forks one child and dies on this line
$alllinks = thaw( $share->fetch() );
Saying : IPC::ShareLite fetch() error: Invalid argument at myscript.pl line 254

Has anyone used this module before ? Successfully ?
Please give me some insight, I'm pulling my hair here.


Show me some sample scripts, anything.
Thanks in advance for any input.

- DAFT PUNK ARE ROBOTS

In reply to Problems with IPC::ShareLite by raxor

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.