Re: Using Statistics::Descriptive for percentiles

I used this script, an example from the docs, to test memory use. 5m is way too much. Cut it back to 2m, and you're ready to roll:

#!/usr/bin/perl

use strict;
use warnings;
use Statistics::Descriptive::Weighted;

my @data = (1..2000000);
my $stat = Statistics::Descriptive::Full->new(); 
$stat->add_data(@data); 
print $stat->quantile(1), "\n";
[download]

Update: Maybe this might help:

#!/usr/local/bin/perl

use strict;
use warnings;

my @numbers = (1..2000000);
printf "Percentile %d%% at %f\n", $_, percentile($_,\@numbers)
for qw/25 75/;

sub percentile {
    my ($p,$aref) = @_;
    my $percentile = int($p * $#{$aref}/100);
    return (sort @$aref)[$percentile];
}
[download]

Comment on Re: Using Statistics::Descriptive for percentiles Select or Download Code

Replies are listed 'Best First'.
Re^2: Using Statistics::Descriptive for percentiles by Hena (Friar) on Jun 02, 2010 at 10:23 UTC
Doesn't help me unfortunately if I drop it down to 2m nor to 200k.	[reply]
Re^3: Using Statistics::Descriptive for percentiles by ww (Archbishop) on Jun 02, 2010 at 13:21 UTC
That raises questions: how much RAM do you have, what OS are you running, and are you absolutely, positively sure you copied the code correctly? If you're not extraordinarily short of RAM (absolute value will vary by OS) and your code is correct, it would be a kindness to the module's author and others unknown to see if you can identify the cause of your problem... and to file a bug report if relevant.	[reply]
Re^4: Using Statistics::Descriptive for percentiles by Hena (Friar) on Jun 02, 2010 at 18:52 UTC
Hum ... perhaps I didn't say things properly. I wanted to avoid using too much memory not that I couldn't use the memory. I was just wondering if there would be more efficient as with 10m numbers, that is using nearly a gig of memory. The problem of not getting 25th percentile as far as I'm aware isn't due to any memory issues. Sorry if I cause confusion there.	[reply]
Re^2: Using Statistics::Descriptive for percentiles by Anonymous Monk on Dec 18, 2019 at 21:02 UTC
didn't work for my Perl v5.18.4 need to change the final line to: `return (sort {$a <=> $b} @$aref)[$percentile];` because otherwise it sorts in alphabetical not numeric order.	[reply] [d/l]