msergeant has asked for the wisdom of the Perl Monks concerning the following question:

I've got a perl script here that uses IPC::Shareable to store an array that is generated using parallel::Forkmanager. I am able to write data into the array fine 500 - 11000 lines and all works well but when I try reading from the array I always receive an out of memory error even though nothing is writing to shared memory only reading from it. Has anyone come across this before and if so how was it solved ? My code is as follows...

tie @$output, 'IPC::Shareable', $glue, { %options } || die "Tie failed
+\n";
my $pm=new Parallel::ForkManager(50);
foreach $keyword ( @keywords ) {
    $pm->start and next;
    $key= join('\+', split(/\ /, $keyword));
    $remote = IO::Socket::INET->new( Proto     => "tcp",
                                     PeerAddr  => $host,
                                     PeerPort  => $port,
                                   );
    unless ($remote) { die "cannot connect to http daemon on $host on 
+$port" }
    $remote->autoflush(1);
    print $remote "GET $prefix$key HTTP/1.0" . $BLANK;
    $count="0";
    while ( <$remote> ) { 
        if (/flist/i) {
            $count++;
        }
    }
    push (@$output, "$keyword,$count");
      close $remote;
    $pm->finish();
}
$pm->wait_all_children;

# Write message to tmp file.
$tempfile = basename($file);
$tmpfile = "/tmp/$tempfile";
open(TMPFILE, ">$tmpfile") || die "Can't open $tmpfile for writing";
foreach $line ( @$output ) {
    $count++;
    print "$count\n";
    print "$line\n";
    print TMPFILE "$line\n";
}
}
close(TMPFILE);

[download]

Replies are listed 'Best First'.
Re: Out of memory error when using IPC::Shareable
by diotalevi (Canon) on Jan 09, 2003 at 05:52 UTC

    My first inclination is to key off your use of the phrase "shared memory". I've dealt with this before for the PostgreSQL database and I pulled up it's documentation on the issue at http://www.postgresql.org/idocs/index.php?kernel-resources.html. The net result is that if you're on a UNIX-alike then you'll need to configure your kernel to provide more resources for System V IPC (shared memory / semaphores). On a Win32 computer I use an IPC daemon from cygwin and haven't ever bothered to figure out how to tweak it. Anyhow - just follow the link and it's fully laid out there.

    Fun Fun Fun in the Fluffy Chair

[reply]
      The OS I am using is FreeBSD, I've gone a bit overboard and given it 128MB of shared memory to play with yet it still bombs out memory wise. The following are my current settings pertaining to shared memory.

      kern.ipc.shmmax: 134217728
      kern.ipc.shmmin: 1
      kern.ipc.shmmni: 192
      kern.ipc.shmseg: 128
      kern.ipc.shmall: 32768
      kern.ipc.shm_use_phys: 1

      I was going to have a look at the site you posted but unfortunately it seems postgres can't connect to it's own DB ;)
      Cheers,

      Mark
[reply]

        It appears someone is doing maintenance. It was up an hour ago. Either that or the site is being hacked. Try tomorrow. Or wait for other monks to get back with other suggestions. That was my best thought.

        Fun Fun Fun in the Fluffy Chair

[reply]
Re: Out of memory error when using IPC::Shareable
by shotgunefx (Parson) on Jan 09, 2003 at 08:21 UTC
    My one attempt at using Shareable, I had the same result. Never did find out what is was but didn't investigate it much either.

    -Lee

    "To be civilized is to deny one's nature."
[reply]

Back to Seekers of Perl Wisdom