in reply to Threads memory consumption

From my experience with threads and memory consumption, you have 2 choices.

1. Reuse existing threads over and over, passing in new data to them.

2. Better yet, especially if you are on Linux, use forks instead of threads. The forks will release memory when they exit. The only useful reason for incurring the overhead associated with threads is if you need to share data between threads in real-time. Even then, forks with SysV IPC shared memory works better and faster. Threads is just a lazy easy way of getting IPC, but it comes at a cost, as you have seen.

I'm not really a human, but I play one on earth.
Old Perl Programmer Haiku ................... flash japh

Replies are listed 'Best First'.
Re^2: Threads memory consumption
by mojo2405 (Acolyte) on Dec 05, 2013 at 15:07 UTC
    my need is to return data from each thread / fork child. it can be hash / object / string or whatever . This is the reason I'm using threads instead of forks. As far as I know - there is no way returning data from fork child.. only with threads... am I correct ?
[reply]
      It is possible to pass result to parent process, and is quite easy. You can use Parallel::ForkManager for example: 
      use 5.010;
use strict;
use warnings;

use Parallel::ForkManager;
use Data::Printer;

my $pm = Parallel::ForkManager->new(2);

$pm->run_on_finish(
    sub {
        # result from the child will be passed as 6th arg to callback
        my $res = $_[5];
        p $res;
    }
);

for (1..3) {
    $pm->start and next;
    # from here and till $pm->finish child process running
    # do something useful and store result in $res
    my $res = { aaa => $_ };
    # this will terminate child and pass $res to parent process
    $pm->finish(0, $res);
}

$pm->wait_all_children;

      [download]
[reply]
[d/l]
        Thanks alot ! used this method - and everything works for me great now. I have a question about this method - Can I have some shared data between the child forks ?
[reply]

In Section Seekers of Perl Wisdom