tinita has asked for the wisdom of the Perl Monks concerning the following question:

hello, this might be more a linux question, but maybe you have experienced this too. I spotted that problem when working with mod_perl and Apache::SizeLimit.

For reproducing I made a simple perl one-liner (well, more like 13 lines) that has a big string (100,000,000) which uses about 200MB RAM.
if I fork this script, on a linux kernel 2.4 all memory is shared, so still 200MB in usage for both. only if I change the string in the child, memory is copied (400MB). I think this is called copy-on-write (correct me if I'm wrong).
now, with a 2.6 kernel on amd64 (debian both systems), almost no memory is shared (like 1MB or so). here's the test script:

perl -we'my $cmd = qq{top -b -n 1 |grep }; print "$$ start\n";print qx{$cmd $$}; my $s="";my $x = "_" x $ARGV[0]; print "string\n";print qx{$cmd $$}; if (my $pid = fork) { print "parent $$ forked $pid\n";print qx{$cmd $$}; $s = "<".$s.">"; } else { print "child $$\n";print qx{$cmd $$}; $s = "(".$s.")";sleep 3; print "child slept 3\n"; } print "$$ end\n";print qx{$cmd $$};' 100000000
perl 5.6.1 and 5.8.4 on kernel 2.6 both show that effect, while 2.4er kernels all share the whole memory (somebody else tested the above script on several machines).

so, question: if this was a bug I would have probably found it with google, but I didn't find anything helpful. what can I do about it, has anyone else experienced this? I guess a pre-forking Apache/mod_perl is pretty common, so there must be others who have the same problem...

update: s/200/400/g and s/100/200/g for correct sizes

Replies are listed 'Best First'.
Re: fork on linux 2.6 does not share memory
by perrin (Chancellor) on Jul 26, 2005 at 14:51 UTC
    It does share memory, using copy-on-write. However, this sharing was not reported accurately in the 2.4 kernel and is basically not reported at all in 2.6. You'll have to look at something else, like how much total physical RAM is being used after each process forks, to determine how much sharing is happening. There was a thread about this on the mod_perl mailing list.
      ok, thanks for the link.
      i actually looked also at the free memory my system reported (with free or xosview), and there I saw that the double size was used. i'll look at that in detail to assure that i looked correctly.

      Update: Ok, now that I checked with a qx{free} directly in the script I saw that the memory size didn't double. *phooey* thanks again. don't know at what I was looking before.

      Unfortunately that means I can dump Apache::SizeLimit. Maybe I can at least use MAX_PROCESS_SIZE instead of MAX_UNSHARED_SIZE. bummer.

Re: fork on linux 2.6 does not share memory
by PreferredUserName (Pilgrim) on Jul 26, 2005 at 16:34 UTC
    Note also that just touching that 200MB chunk doesn't copy the whole thing. It just copies the page that you touched (4 or 8 KB usually).
      well, it actually depends on how you touch it and how perl internals perform the requested action.

      For instance, running a s/foo/bar/ operation will reallocate the full string even if it just substitutes a few bytes.

Re: fork on linux 2.6 does not share memory
by Monk Eugene (Initiate) on Jul 26, 2005 at 14:52 UTC
    Well, i believe, fork'ed processes should not share any memory between each other. Linux 2.6 behaviour looks correct..
      Well, i believe, fork'ed processes should not share any memory between each other. Linux 2.6 behaviour looks correct.
      Most modern OSes, including Linux, use a copy-on-write scheme: the two processes initially share each page of memory, but the first attempt to write to such a page is detected by the MMU (memory management unit) hardware, and the page is invisibly copied. This is why fork is so fast.

      Dave.