Equally important, how can I do this and use less memory?
As always (it seems to me), Perl provides the tools to do the job.
Heres a crude, but working implementation of a float array class that allows me to allocate your 16,000,000 float array using 125,278k of ram. That's an overhead of just 0.01764 bytes per float (276k total).
It's not to shabby on the performance stakes either coming at 0.000238 seconds per write/read access, and allocation of the 16 million taking under 4.5 seconds. That includes initialising them (to the same value. 0.0 by default).
Of course, what you loose is flexibility. It will only store floats (doubles are an easy mod) and you can't shrink or grow the array.
#! perl -slw use strict; package SEIFA; #! Space Efficient, Inflexible Float Array my %instances; use constant ARYREF => 0; use constant X_SIZE => 1; use constant Y_SIZE => 2; sub new { my ($class, $x, $y, $init ) = @_; my $size = $x * $y; my $mem = pack('f', $init||'0.0') x $size; my $data = [ \$mem, $x, $y ]; my $self = bless $data, $class; $instances{$self} = $data; return $self; } use constant X => 1; use constant Y => 2; sub seifa{ my ($data, $x, $y) = ($instances{+shift}, shift, shift); my $pos = ($data->[X_SIZE] * $y * 4) + ($x * 4); return unpack 'f*', substr( ${$data->[ARYREF]}, $pos, 4) unless @_ +; return unpack 'f*', substr( ${$data->[ARYREF]}, $pos, @_ * 4) = pa +ck 'f*', @_; } package main; use vars qw[$X $Y]; use Benchmark::Timer; my $timer = Benchmark::Timer->new(); $X ||= 20; $Y ||= 20; print 'Before'; <>; $timer->start("allocate ${\($X * $Y)}"); my $matrix = SEIFA->new( $X, $Y, 1.0); $timer->stop("allocate ${\($X * $Y)}"); print 'After'; <>; $matrix->seifa( $X-1, 0, 3.1415926 ); $matrix->seifa( $X-1, $Y-1, 3.1415926 ); local $\; for my $y (0 .. 5, '...', $Y-6 .. $Y-1) { print( " ... " x 8, $/), next if $y eq '...'; for my $x (0 .. 3, '...', $X-3 .. $X-1) { print( ' ... '), next if $x eq '...'; $timer->start('write-read'); my $value = $matrix->seifa($x, $y, "$y.$x"); $timer->stop('write-read'); printf '%9.4f, ', $value; } print $/, } print $timer->report(); __END__ c:\test>230052 -X=4000 -Y=4000 Before After 0.0000, 0.1000, 0.2000, 0.3000, ... 0.3997, 0.3 +998, 0.3999, 1.0000, 1.1000, 1.2000, 1.3000, ... 1.3997, 1.3 +998, 1.3999, 2.0000, 2.1000, 2.2000, 2.3000, ... 2.3997, 2.3 +998, 2.3999, 3.0000, 3.1000, 3.2000, 3.3000, ... 3.3997, 3.3 +998, 3.3999, 4.0000, 4.1000, 4.2000, 4.3000, ... 4.3997, 4.3 +998, 4.3999, 5.0000, 5.1000, 5.2000, 5.3000, ... 5.3997, 5.3 +998, 5.3999, ... ... ... ... ... ... +... ... 3994.0000, 3994.1001, 3994.2000, 3994.3000, ... 3994.3997, 3994.3 +999, 3994.3999, 3995.0000, 3995.1001, 3995.2000, 3995.3000, ... 3995.3997, 3995.3 +999, 3995.3999, 3996.0000, 3996.1001, 3996.2000, 3996.3000, ... 3996.3997, 3996.3 +999, 3996.3999, 3997.0000, 3997.1001, 3997.2000, 3997.3000, ... 3997.3997, 3997.3 +999, 3997.3999, 3998.0000, 3998.1001, 3998.2000, 3998.3000, ... 3998.3997, 3998.3 +999, 3998.3999, 3999.0000, 3999.1001, 3999.2000, 3999.3000, ... 3999.3997, 3999.3 +999, 3999.3999, 1 trial of allocate 16000000 (4.467s total) 84 trials of write-read (20ms total), 238us/trial c:\test>
Of course, you could get fancy and use tie and/or overload--shame you can't overload [..] (can you?)--but the performance hit may not be worth hit.
I also played with a version that used a standard array for the first dimension and pack'd strings of doubles for the second. gives back some of the flexibility in that you can shrink and grow the main array and the sub-arrays (independantly) including autovivfying on assignment, but the penalty is x16 memory consumption and 1/4 of the speed.
Examine what is said, not who speaks.
The 7th Rule of perl club is -- pearl clubs are easily damaged. Use a diamond club instead.
In reply to Re: why does this array take up so much memory?
by BrowserUk
in thread why does this array take up so much memory?
by Anonymous Monk
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |