Re: Grouping one piddle based on ranges of another

Hello astroman,

I took some ideas from your reply to djerius to create this solution. I think this will work for you, and it keeps all the operations in piddles/PDL. Basically, I create masks for your greater than and less than or equal to conditions and apply them to "expanded" versions of your $a and $b piddles.

#!/usr/bin/env perl

use strict;
use warnings;

use PDL;

# The number of intervals you require (your example shows 3)
my $n = 3;

my $a = pdl (0,6,18,7,19,3,10,2,12,4,8,9,1,15,11,11,19,17,0,9);
my $b = pdl (0,1,2,3,4,5,6,7,8,9,10,10.5,11,11.5,12,12.5,13,13.5,14,14
+.5);

my $gt = sequence($n)*5; # equivalent to pdl(0, 5, 10) for $n = 3 
my $ltoe = (sequence($n)+1)*5; # equivalent to pdl(5, 10, 15) for $n =
+ 3

my $expanded_a = $a->dummy(0,$n);
my $expanded_b = $b->dummy(0,$n);

my $gt_mask = $expanded_b > $gt;
my $ltoe_mask = $expanded_b <= $ltoe;

my $mask = $gt_mask & $ltoe_mask;
my $mask_w_bad = $mask->setbadif($mask == 0);

my $masked = $expanded_a * $mask_w_bad;

my $medians = $masked->transpose->medover;

print $medians, "\n";

exit;
[download]

Comment on Re: Grouping one piddle based on ranges of another Download Code

Replies are listed 'Best First'.
Re^2: Grouping one piddle based on ranges of another by astroman (Novice) on Aug 28, 2015 at 09:11 UTC
Cool! Thanks for that solution, it's exactly what I was going for. :) Sadly, testing reveals my original hypothesis (that I could obtain stupendous speed improvements by keeping everything as piddles instead of iterating through a "for" loop) to be incorrect. I think the extra computational expense of the second dimension kills my runtime: with $a and $b of size ~3 million, and n=100, it finishes in ~20 seconds vs ~10 seconds for the "for" loop I was trying to improve: `sub test_medians { use strict; use warnings; use PDL; $PDL::BIGPDL = 1; my $n = 100; my $step = 5; my $a = random(3000000)100; my $b = random(3000000)1000; $b = $b->qsort; my $d = zeroes($n); for (my $i=0;$i<$n;$i++) { $d(($i)) .= $a((($b>($i$step))($b<=($i+1)*$step));?)->medove +r; } return $d; }` [download] Nevertheless, thanks for your help! :)	[reply] [d/l]

Replies are listed 'Best First'.

Re^2: Grouping one piddle based on ranges of another
by astroman (Novice) on Aug 28, 2015 at 09:11 UTC

Sadly, testing reveals my original hypothesis (that I could obtain stupendous speed improvements by keeping everything as piddles instead of iterating through a "for" loop) to be incorrect. I think the extra computational expense of the second dimension kills my runtime: with $a and $b of size ~3 million, and n=100, it finishes in ~20 seconds vs ~10 seconds for the "for" loop I was trying to improve:

sub test_medians
{
    use strict;
    use warnings;
    use PDL;
    $PDL::BIGPDL = 1;

    my $n = 100;
    my $step = 5;

    my $a = random(3000000)*100;  
    my $b = random(3000000)*1000; 
    $b = $b->qsort;

    my $d = zeroes($n);

    for (my $i=0;$i<$n;$i++)
    {
        $d(($i)) .= $a((($b>($i*$step))*($b<=($i+1)*$step));?)->medove
+r;
    }
    return $d;
}
[download]

Nevertheless, thanks for your help! :)

[reply]
[d/l]