in reply to Re^8: Howto Avoid Memory Problem in List::MoreUtils
in thread Howto Avoid Memory Problem in List::MoreUtils

how have you created the scalars inside @numbers?

If you have read them from a file and never used them as numbers before, comparing then numerically also forces their internal structure to grow to accomodate the NV and/or IV slot.

Also, in not too old versions of perl, there was a bug that caused numbers to be converted to strings inside sort even when numeric comparison was used.

Besides that, in general, perl mergesort uses the memory equivalent to two pointers per value, one to pass the value on the stack and the other for the merge. Sort::Key functions use the same plus the memory required for the key (for instance, 4 bytes for an integer key on a 32bits computer or 8 bytes for a floating point number).

Replies are listed 'Best First'.
Re^10: Howto Avoid Memory Problem in List::MoreUtils
by BrowserUk (Patriarch) on May 04, 2006 at 14:40 UTC
    how have you created the scalars inside @numbers?

    I am creating the array with rand, so they should already be IVs: 

    #! perl -slw
use strict;
use sort '_quicksort';

sub mem {
    my( $usage ) =  `tasklist /NH /FI \"pid eq $$\" ` =~ m[ (\S+) \s+ 
+K \s* $ ]x;
    return $usage;
}

$| = 1;

my @a; $#a = 20e6;
$a[ $_ ] = int rand 32767 for 0 .. 20e6;

printf "Mem after building array: %s kb\n", mem;

@a = sort{ $a <=> $b } @a;

printf "Mem after sorting array: %s kb\n", mem;

    [download]
    Also, in not too old versions of perl, there was a bug that caused numbers to be converted to strings inside sort even when numeric comparison was used.

    That was it. I'm still using 5.8.6 as my general install. Running the above script under 5.8.8 fixes that problem. Many thanks for your patience.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]

In Section Seekers of Perl Wisdom