Re^2: Monitoring the death of oversized Perl processes

In order PERL_EMERGENCY_SBRK to have effect you should use Perl's malloc, which doesn't look like a good idea to me.

And you make this claim based on what exactly? From the install file:

=head2 Malloc Issues

Perl relies heavily on malloc(3) to grow data structures as needed,   
so perl's performance can be noticeably affected by the performance of
the malloc function on your system.  The perl source is shipped with a
version of malloc that has been optimized for the typical requests from
perl, so there's a chance that it may be both faster and use less memory
than your system malloc.
        
However, if your system already has an excellent malloc, or if you are
experiencing difficulties with extensions that use third-party libraries
that call malloc, then you should probably use your system's malloc.
(Or, you might wish to explore the malloc flags discussed below.)

=over 4

=item Using the system malloc

To build without perl's malloc, you can use the Configure command

        sh Configure -Uusemymalloc

or you can answer 'n' at the appropriate interactive Configure prompt.

Note that Perl's malloc isn't always used by default; that actually
depends on your system. For example, on Linux and FreeBSD (and many more
systems), Configure chooses to use the system's malloc by default.
See the appropriate file in the F<hints/> directory to see how the
default is set.

Comment on Re^2: Monitoring the death of oversized Perl processes

Replies are listed 'Best First'.
Re^3: Monitoring the death of oversized Perl processes by zwon (Abbot) on Mar 06, 2010 at 09:37 UTC
And you make this claim based on what exactly? Which claim? In order PERL_EMERGENCY_SBRK to have effect you should use Perl's malloc doesn't look like a good idea to me	[reply]
Re^4: Monitoring the death of oversized Perl processes by JavaFan (Canon) on Mar 06, 2010 at 15:45 UTC
Why doesn't it look like a good idea to you? It's good enough that it's the default on many platforms.	[reply]
Re^5: Monitoring the death of oversized Perl processes by zwon (Abbot) on Mar 06, 2010 at 16:24 UTC
Yes, some platforms use it as default: `$ grep "mymalloc='y'" hints/* hints/hpux.sh: $undef\|false\|[nN]*) usemymalloc='y' ;; hints/ncr_tower.sh:usemymalloc='y' hints/next_3.sh:#usemymalloc='y' hints/next_4.sh:usemymalloc='y' hints/sco_2_3_4.sh:usemymalloc='y' hints/super-ux.sh: usemymalloc='y' hints/unicosmk.sh:'') usemymalloc='y' hints/unicos.sh: # usemymalloc='y' hints/utekv.sh:usemymalloc='y'` [download] But not so many. It seems using only sbrk for memory allocation, so it will never reclaim allocated memory back to the OS. Many modern malloc implementations are using mmap for allocating large blocks. Here's the example script: `use strict; use warnings; system "ps vp $$"; local $/; open my $fd, "<", $ARGV[0] or die $!; my $content = <$fd>; system "ps vp $$"; undef $content; system "ps vp $$";` [download] And here are results with different perl's on Ubuntu: $ dd if=/dev/zero of=zero.dat bs=8192 count=131072 131072+0 records in 131072+0 records out 1073741824 bytes (1.1 GB) copied, 8.40646 s, 128 MB/s $ ls -lh zero.dat -rw-r--r-- 1 zwon zwon 1.0G 2010-03-06 13:23 zero.dat $ perl -V \| grep mymalloc usemymalloc=n, bincompat5005=undef $ perl slurp_file.pl zero.dat PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 3775 pts/1 S+ 0:00 0 3 18140 2244 0.0 perl slurp_fi +le.pl zero.dat PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 3775 pts/1 S+ 0:01 0 3 1066720 1050840 25.9 perl slur +p_file.pl zero.dat PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 3775 pts/1 R+ 0:01 0 3 18140 2264 0.0 perl slurp_fi +le.pl zero.dat $ /opt/perl/5.10.1/bin/perl -V \| grep mymalloc config_args='-des -Dusemymalloc -Dusethreads -Dprefix=/opt/perl/5. +10.1' usemymalloc=y, bincompat5005=undef $ /opt/perl/5.10.1/bin/perl slurp_file.pl zero.dat PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 3792 pts/1 S+ 0:00 0 1354 18925 2212 0.0 /opt/perl/5.1 +0.1/bin/perl slurp_file.pl zero.dat PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 3792 pts/1 S+ 0:01 0 1354 1067505 1050812 25.9 /opt/perl +/5.10.1/bin/perl slurp_file.pl zero.dat PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 3792 pts/1 S+ 0:01 0 1354 1067505 1050820 25.9 /opt/perl +/5.10.1/bin/perl slurp_file.pl zero.dat [download] As you can see perl which uses system malloc was able to return memory. BTW, perl compiled with `-Dusemymalloc` failed its tests on `t/op/threads.t`, I checked this on Ubuntu amd64 and OpenBSD i386. Perhaps this is not a big problem, but it shows the level of support for this option. PS note, that my opinion regarding perl's malloc is highly subjective and not based on a lot of facts. If you have any evidence that prove me wrong, I would be grateful to know about it.	[reply] [d/l] [select]