I used Devel::Profile and found that 84.74% of the running time is in the main::transpose function.

I don't know how the data in @records actually arrives there, but I would consider capturing it in an array of columns rather than an array of rows -- eliminating the transpose operation completely.

I tried replacing the downsample() function and the functions themselves, to operate directly on the columns. (The code for that is included below.) In order to get some timings I expanded the sample @records by the simple expedient of @records = (@records) x 100_000. On my machine, operating directly on the columns downsampled the records in 1.1s, where your code took 12.8s.

Mind you, I doubt whether you downsample this many samples at once, so these timings will overstate the impact of improving the inside of the downsample function.

Finally, depending on how the data is input, I would consider combining the downsampling with the input -- avoiding construction of the @records array altogether...

# GMCH.... my @funcs_gmch = ( undef, # first_timestamp \&avg_gmch, \&max_gmch, \&min_gmch, \&first_gmch, \&last_gmch, ) ; sub downsample_gmch { my( $resolution, $data ) = @_; my $first_timestamp = first_timestamp_for_resolution( $resolution, $data->[0][0] ); my @output; push( @output, $first_timestamp ); push( @output, $funcs_gmch[$_]->( $data, $_ ) ) for ( 1..$#funcs_g +mch); return( @output ); } sub avg_gmch { my ($rd, $i) = @_ ; my $s = 0 ; $s += $_->[$i] foreach (@$rd) ; return $s / scalar(@$rd) ; } ; sub max_gmch { my ($rd, $i) = @_ ; my $m = $rd->[0]->[$i] ; $m < $_->[$i] and $m = $_->[$i] foreach (@$rd) ; return $m ; } ; sub min_gmch { my ($rd, $i) = @_ ; my $m = $rd->[0]->[$i] ; $m > $_->[$i] and $m = $_->[$i] foreach (@$rd) ; return $m ; } ; sub first_gmch { my ($rd, $i) = @_ ; return $rd->[0]->[$i] ; } ; sub last_gmch { my ($rd, $i) = @_ ; return $rd->[-1]->[$i] ; } ;

In reply to Re^3: An efficient, scalable matrix transformation algorithm by gone2015
in thread An efficient, scalable matrix transformation algorithm by Luftkissenboot

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.