in reply to pushing multidimensional arrays

This is the kind of problem for which PDL was created. I have only just started learning PDL, so my contrib below is going to be naive, but here is how I would do it 
# a [3 x 100K] piddle
$a = sequence 3, 100_000

[
 [ 0  1  2]
 [ 3  4  5]
 [ 6  7  8]
..
]

# create an extra [1 x 100K] piddle to hold the norms
$b = zeros 1, 100_1000

[
 [0]
 [0]
 [0]
..
]

# append $b to $a
$c = $a->append($b)

[
 [ 0  1  2  0]
 [ 3  4  5  0]
 [ 6  7  8  0]
..
]

# get the slices to the different cols
$col1 = $c->slice('0,:')
$col2 = $c->slice('1,:')
$col4 = $c->slice('3,:')

# calc the norms
$col4 .= (($col1 ** 2) + ($col2 ** 2)) ** 0.5

[
 [         0          1          2          1]
 [         3          4          5          5]
 [         6          7          8  9.2195445]
..
]

# or, do it all in one line

$b = $a->append(zeros 1, 100_000)
$norm = $b->slice('3,:')
$norm .= (($b->slice('0,:') ** 2) + ($b->slice('1,:') ** 2)) ** 0.5

[download]

I am sure PDL vets would improve the above many different ways, however, PDL is ideally suited for the kind of problem you are posing.

--

when small people start casting long shadows, it is time to go to bed

Replies are listed 'Best First'.
Re^2: pushing multidimensional arrays
by etj (Priest) on May 29, 2022 at 22:11 UTC
    The only thing I'd change in the above is to first initialise a block of zeroes (which these days is very quick) to the final size, then assign the starting information into it (using a slice, in standard PDL style), then do the calculation of norms: 
    $out = zeroes 4,100_000;
$out->slice('0:2')->inplace->sequence;
$out->slice('3') .= (($out->slice('0') ** 2) + ($out->slice('1') ** 2)
+) ** 0.5;

    [download]
    The benefit of starting with the final size is there's only one big allocation. When loop-fusion arrives, the data will only go in and out of RAM once. Note the above slices don't need to specify the trailing ,:.
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom