Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello,

I'm reading a netCDF file to extract a timeseries using a command line named ncks.

I dump its output in a temporary file and then I read the temporary file to extract the values and push them into an array.

Is there a way to store in memory the extracted records (@tmplines in the code below) without writing them on a temporary file?

Since I have to call the subroutine thousands of times I imagine that it would speed up things.

Thanks for any suggestion and best regards.

use strict; use IPC::Run qw( run ); use Tie::File; use Fcntl 'O_RDONLY'; # ... cut ... my @ts = get_time_series(File=>$ncfile,Variable=>$ncvar,Id=>$nc_id); # ... cut ... sub get_time_series { my %args = @_; my $file = $args{File}; my $variable = $args{Variable}; my $id = $args{Id}; my $tmpfile = "$id.tmp"; # Example: # ncks -v vmro3 -d station,1,1 myfile.nc | grep "station\[" # my $command = qq(ncks -v $variable -d station,$id,$id $file | grep + \"station\\[\" > $tmpfile); run $command; # This will write in file the following records: #station[3385]=3386 #time[0]=3287.02083333 station[3385]=3386 vmro3[3385]=1.53524e-08 mole + mole-1 #time[1]=3287.0625 station[3385]=3386 vmro3[7513]=1.55109e-08 mole mol +e-1 #time[2]=3287.10416667 station[3385]=3386 vmro3[11641]=1.56481e-08 mol +e mole-1 my @ts; tie my @tmplines, 'Tie::File', $tmpfile, mode => O_RDONLY; foreach my $ii (1..$#tmplines) { my $line = $tmplines[$ii]; my @pa = anytrim(split(/\s/,$line)); my ($dummy,$value) = split(/=|\s/,$pa[2]); push @ts, $value; } unlink $tmpfile or die "$tmpfile: $!"; return @ts; }

Replies are listed 'Best First'.
Re: Execute a command line and save in memory its output
by Discipulus (Canon) on Apr 29, 2016 at 08:39 UTC
    You can open a program for reading it's output: see open for details.

    Something like this will just suffice:

    open my $cmd, "ncks -v $variable -d station,$id,$id $file|" or die "un +able to open a CMD for read!"; while (<$cmd>) { next unless $_ =~ /station\[/; # put your grep logic in perl side too +! chomp; ...

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
      Thanks!
Re: Execute a command line and save in memory its output
by Laurent_R (Canon) on Apr 29, 2016 at 18:01 UTC
    Using back ticks, something like this should work in most cases:
    my $command = qq(ncks -v $variable -d station,$id,$id $file | grep \"s +tation\\[\"); my @output = `$command`;