in reply to Re^19: Strange memory leak using just threads (forks.pm)
in thread Strange memory leak using just threads

Yes. And if every worker fork needs to modify 1 bit on every 4x page of 1 GB of shared data, it's gonna take forever.

You show me your use-case, and I'll show you mine.

  • Comment on Re^20: Strange memory leak using just threads (forks.pm)

Replies are listed 'Best First'.
Re^21: Strange memory leak using just threads (forks.pm)
by zwon (Abbot) on Sep 23, 2010 at 16:54 UTC
    And if every worker fork needs to modify 1 bit on every 4x page of 1 GB of shared data, it's gonna take forever.

    Rubbish

    use strict; use warnings; use 5.010; use Time::HiRes qw(time); my $data = 'x' x 500_000_000; my $t = time; $data =~ tr/x/y/; say time - $t; __END__ 0.469923973083496
    use strict; use warnings; use 5.010; use Time::HiRes qw(time); my $data = 'x' x 500_000_000; my $t = time; unless (fork) { $data =~ tr/x/y/; exit; } wait; say time - $t; __END__ 0.765344142913818
    use strict; use warnings; use 5.010; use threads; use Time::HiRes qw(time); my $data = 'x' x 500_000_000; my $t = time; threads->create( sub { $data =~ tr/x/y/; } )->join; say time - $t; __END__ 1.29563307762146

      That's not quite what I said is it :)

      See how you fare with this:

      use strict; use warnings; use 5.010; use Time::HiRes qw(time); my $data = 'x' x 500 * 1024**2; my $t = time; for my $n ( 1 .. 100 ) { unless (fork) { substr( $data, 4096 * $_ + $n, 1 ) |= 1 for 0 .. 124; exit; } } waitall; say time - $t;
        This takes 5.25 seconds on my machine (4GB, 4 cores, Linux).

        The respective threads versions take:

        use 5.010; use threads; use Time::HiRes qw(time); my $data = 'x' x (500 * 1024**2); my $t = time; for my $n ( 1 .. 100 ) { threads->create( sub { substr( $data, 4096 * $_ + $n, 1 ) |= 1 for 0 .. 124; } )->join; } say time - $t; __END__ 164.842848062515
        use 5.010; use threads; use Time::HiRes qw(time); my $t = time; for my $n ( 1 .. 100 ) { threads->create( sub { my $data = 'x' x (500 * 1024**2); substr( $data, 4096 * $_ + $n, 1 ) |= 1 for 0 .. 124; } )->join; } say time - $t; __END__ 172.88907790184