Interpolating data using pdl

karthik248 has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks,

I'm new to perl, as well as PDL. So here's the question..

I have a matrix of size 1800x1800, and only few points have valid data(say 10000). Now, i want to interpolate(bilinear) whole matrix using the available data.

I've already implemented this using pure-perl, but due to complexity issues, i'm trying to do this using PDL

I have a few ideas in mind, but i'm too sure about them, as i don't know if i can implement the same using PDL.

Any suggestions or solutions are appreciated. Thanks in advance.

Edit:

Monk vr provided the solution in his replies. Very thankful for his responses.

Updated question

How to rescale a 2-dimensional data by interpolation, using piddles?

Solution

    use PDL;
    use PDL::Image2D;
    use PDL::IO::GD;

    my $data_pdl; #a pdl(289,145)
    my $interpolated_pdl = zeroes(21600,10800);
    bilin2n($interpolated_pdl,$data_pdl);

    my $range1 = ( $x >= 5 ) * ( $x < 10 );
    my $range2 = ( $x >= 10 ) * ( $x < 15 );
    my $range3 = $x >= 15;

    my $plotted_pdl = (     $range1 
                + 2 * $range2
                + 3 * $range3 )-> byte;

    my $palette = byte [ 
        [   0,   0,   0 ],
        [ 255,   0,   0 ],
        [   0, 255,   0 ],
        [   0,   0, 255 ]
    ];

    my $plotted_image = PDL::IO::Image->new_from_pdl($plotted_pdl,$pal
+lete);

    my $gd_image=PDL::IO::GD->new(pdl=>$plotted_image->pixels_to_pdl);

    #now all kinds of GD operations can be applied.
[download]

I'm sure there are much better ways of doing this, like directly generating a PDL::IO::GD object using PDL::IO::GD->new({ pdl => $pdl_var, lut => $lut_piddle }) but the above solution works fine.

Comment on Interpolating data using pdl Select or Download Code

Replies are listed 'Best First'.
Re: Interpolating data using pdl by vr (Curate) on Mar 15, 2017 at 11:56 UTC
`pdl> $x = sequence( 3, 3)-> float pdl> p $x [ [0 1 2] [3 4 5] [6 7 8] ] pdl> $i = PDL::IO::Image-> new_from_pdl( $x ) pdl> $i-> rescale(5, 5, 2) pdl> $x_new = $i-> pixels_to_pdl pdl> p $x_new [ [ 0 0.4 1 1.6 2] [ 1.2 1.6 2.2 2.8 3.2] [ 3 3.4 4 4.6 5] [ 4.8 5.2 5.8 6.4 6.8] [ 6 6.4 7 7.6 8] ]` [download] Note, that "float" is important instead of default "double". Also, check possible interpolation methods.	[reply] [d/l]
Re^2: Interpolating data using pdl by karthik248 (Acolyte) on Mar 15, 2017 at 16:35 UTC
Many thanks for the solution. I might be able to use this. Can you also help me with the further processing? The data-values can be categorized into three ranges and now i have a pdl with data for each point(pixel) of the map(as described here). And now i have to plot these points onto a image, with different colors based on the ranges. Can you provide a script for generating such image.	[reply]
Re^3: Interpolating data using pdl by vr (Curate) on Mar 15, 2017 at 17:11 UTC
Actually, looking back at results in my answer, this bilinear is "somewhat" strange (and other interpolation methods are probably useless for the given task). `pdl> $x = sequence 3,3 pdl> $y = zeroes 5,5 pdl> bilin2d( $x, $y ) pdl> p $y [ [ 0 0.5 1 1.5 2] [1.5 2 2.5 3 3.5] [ 3 3.5 4 4.5 5] [4.5 5 5.5 6 6.5] [ 6 6.5 7 7.5 8] ]` [download] That's better now. We can even visualize it with `mesh3d [$y]`. karthik248, it's not clear what kind of plot do you mean? 3-D? Over-impose some visualization onto a world map? Or just 2-D image with 3 different colors?	[reply] [d/l] [select]
Re^4: Interpolating data using pdl by karthik248 (Acolyte) on Mar 15, 2017 at 17:53 UTC
Re^5: Interpolating data using pdl by vr (Curate) on Mar 15, 2017 at 19:43 UTC
Some notes below your chosen depth have not been shown here
Re: Interpolating data using pdl by thanos1983 (Parson) on Mar 15, 2017 at 10:41 UTC
Hello karthik248, I do not have any experience on Perl and Matrix. I did some research online and I found (PDL::Matrix, PDL::MatrixOps, PDL::NiceSlice). These modules should be a good start for what you want to achieve, create a sample of code and a sample of input and output for all of us "not experienced" with matrices to replicate your problem and possibly try to find a solution to your problem. Update: What about (PDL::Interpolate)? Update2: What about (PDL::Primitive)? Hope this helps, or at least gives you new ideas on how to proceed with your problem. Seeking for Perl wisdom...on the process of learning...not there...yet!	[reply] [d/l] [select]
Re^2: Interpolating data using pdl by karthik248 (Acolyte) on Mar 15, 2017 at 11:27 UTC
Thanks for your response. I'm aware of PDL::Interpolate but as i am new to PDL, i was unable to understand the terminology, so i was going through the docs. Soon I was lost in the docs getting nowhere. So I was hoping for a monk to guide me through it. Now I'm trying use this code snippet from the docs `$source = 10*xvals(10,10) + yvals(10,10); $index = pdl([[2.2,3.5],[4.1,5.0]],[[6.0,7.4],[8,9]]); print $source->interpND( $index );` [download]	[reply] [d/l]
Re: Interpolating data using pdl by vrk (Chaplain) on Mar 15, 2017 at 09:46 UTC
It's been some years since I've used PDL, so please clarify: what kind of invalid data does the matrix have? Is it NaN, Inf, 0 or something else? Are the valid data regularly spaced or do they appear randomly anywhere within the matrix?	[reply]
Re^2: Interpolating data using pdl by karthik248 (Acolyte) on Mar 15, 2017 at 10:10 UTC
Actually we have data only at few points. There is no invalid data. What i'm trying to do is something like resizing a image. Suppose i have a 100x100 image and i want to resize it to 1000x1000. To be precise, I have a world map, and a dataset for every 1.25 interval along both latitude and longitudes. Now I want to be able to interpolate the existing data, so that i can calculate the data at any point on the map.	[reply]
Re^3: Interpolating data using pdl by thanos1983 (Parson) on Mar 15, 2017 at 11:46 UTC
Hello Again, Based on What i'm trying to do is something like resizing a image. Suppose i have a 100x100 image and i want to resize it to 1000x1000. you could use Image::Resize. Sample of code from documentation: `use Image::Resize; $image = Image::Resize->new('large.jpg'); $gd = $image->resize(250, 250);` [download] But also Image::Imlib2. Sample of code from Stretch, resize, or thumbnail an image using Perl: `use Image::Imlib2; # load image from file my $image = Image::Imlib2->load("in.png"); # get some info if you want my $width = $image->width; my $height = $image->height; # scale the image down to $x and $y # you can set $x or $y to zero and it will maintain aspect ratio my $image2 = $image->create_scaled_image($x,$y); # save thumbnail to file $image2->save("out.png");` [download] Seeking for Perl wisdom...on the process of learning...not there...yet!	[reply] [d/l] [select]