Re^2: upper or lower triangular matrix to full

Thanks for your response, and for pointing me to the tips on posting- quite new around here and much appreciate it. Here are some more details, to add to my original question. The matrix looks like this:

NA 0.0132 0.0116 0.0082 0.0157 0.0246 0.0000 0.0338 0.0000 0.0173
NA NA 0.0084 0.0042 0.0025 0.0126 0.0061 0.0107 0.0214 0.0177
NA NA NA 0.0159 0.0078 0.0216 0.0133 0.0080 0.0029 0.0067    
NA NA NA NA 0.0185 0.0362 0.0167 0.0266 0.0000 0.0093    
NA NA NA NA NA 0.0268 0.0046 0.0134 0.0000 0.0000
NA NA NA NA NA NA 0.0063 0.0112 0.0138 0.0000    
NA NA NA NA NA NA NA 0.0128 0.0239 0.0000    
NA NA NA NA NA NA NA NA 0.0102 0.0188
NA NA NA NA NA NA NA NA NA 0.0088
[download]

The elements are tab separated. All I want in the end, is for the NAs to be replaced by the numbers, no other operations need to be done. I need this because a program I want to run requires the input formatted in this way. I will take a look at PDL as sugegsted. Thanks again, Savita

Comment on Re^2: upper or lower triangular matrix to full Download Code

Replies are listed 'Best First'.
Re^3: upper or lower triangular matrix to full by etj (Priest) on May 09, 2022 at 13:31 UTC
The first thing to check is that the data consumer (which the OP notes is not them) may be able to actually process the upper-triangular matrix as-is; look at LAPACK, which has routines that operate on triangular matrices directly without needing to copy - in other words, this task may not actually be needed. If it is, the approach I would take using PDL is to pre-allocate a correctly-sized disk-based ndarray (so that RAM is not a limitation) using PDL::IO::FastRaw (untested), from `perldl`: `use PDL::IO::FastRaw; $pdl = mapfraw('fname', {Creat => 1, Dims => [460_000, 460_000], Datat +ype => double()});` [download] It looks like currently PDL::IO::Misc (with `rcols`) and PDL::IO::CSV don't have a facility to read into an existing (possibly disk-based) ndarray. It would still be possible, albeit slow, to read the file line-by-line, probably using Text::CSV_XS, using `slice` to insert each row into the ndarray. Once the ndarray exists, to copy one triangle into the other would be super-easy using PDL::LinearAlgebra::Real#tricpy.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^3: upper or lower triangular matrix to full
by etj (Priest) on May 09, 2022 at 13:31 UTC

If it is, the approach I would take using PDL is to pre-allocate a correctly-sized disk-based ndarray (so that RAM is not a limitation) using PDL::IO::FastRaw (untested), from perldl:

use PDL::IO::FastRaw;
$pdl = mapfraw('fname', {Creat => 1, Dims => [460_000, 460_000], Datat
+ype => double()});
[download]

PDL::IO::Misc

rcols

PDL::IO::CSV

Text::CSV_XS

slice

Once the ndarray exists, to copy one triangle into the other would be super-easy using PDL::LinearAlgebra::Real#tricpy.

[reply]
[d/l]
[select]