marcokuma has asked for the wisdom of the Perl Monks concerning the following question:

Dear Perl Monks,

Is it possible, somehow, to pass the contents of a variable to an external program without actually explicitely printing the contents of the variable to a file?

In other words, to do something like: 
$lots_of_data = function_creating_a_lot_of_data();
`anotherprogram $lots_of_data > $processed_data`;

[download]

where $lots_of_data and $processed_data are variables for perl but the outside program magically treats their contents as the contents of files?

Of course, I realize that what I just described is science fiction, but perhaps there is some intermediate solution between printing/reading contents to/from temporary files and the above science fiction scenario?

Thanks in advance for your advice!

Regards,

Marco

Replies are listed 'Best First'.
Re: pretending that the contents of a variable are files
by merlyn (Sage) on Jan 28, 2005 at 12:32 UTC
    If your other program can read STDIN and write to STDOUT, you can construct a data pump by forking twice (this particular code is from memory and therefore untested, but it's something like this): 
    defined(my $kid = open RESULT, "-|") or die "fork: $!";
unless ($kid) { # kid does:
  defined(my $grandkid = open PUMP, "-|") or die "child fork: $!";
  unless ($grandkid) { # grandkid does:
    print @LotsOfDataHere;
    exit 0;
  }
  open STDIN, "<&PUMP" or die "Cannot dup PUMP to STDIN: $!";
  exec "anotherprogram";
  die "exec anotherprogram: $!";
}
while (<RESULT>) {
 # process the large result
 ...
}

    [download]
    This is effectively setting up the pipeline of "some_perl_widget | anotherprogram | your_program", but from within the program itself.

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

[reply]
[d/l]
Re: pretending that the contents of a variable are files
by deibyz (Hermit) on Jan 28, 2005 at 12:27 UTC
    Try to open a "double" pipe (assuming read from STDIN is possible): 
    open FILE, "|du -sk|" or die "$!";
print FILE `ls`;
print while(<FILE>);

    [download]
    perldoc -f open for more information.

    Edit: This is false... :(
    As noted in the replies, IPC::Open2 was the correct solution. From perldoc IPC::Open2: 

    The open2() function runs the given $cmd and connects $rdrfh
for reading and $wtrfh for writing.  It's what you think
should work when you try

   $pid = open(HANDLE, "|cmd args|");

The write filehandle will have autoflush turned on.

    [download]

    That was what confused my. I'm sorry for the error.
    This is the syntax for Open2: 

    use IPC::Open2;

         $pid = open2(\*RDRFH, \*WTRFH, 'some cmd and args');
           # or without using the shell
         $pid = open2(\*RDRFH, \*WTRFH, 'some', 'cmd', 'and', 'args');

         # or with handle autovivification
         my($rdrfh, $wtrfh);
         $pid = open2($rdrfh, $wtrfh, 'some cmd and args');
           # or without using the shell
         $pid = open2($rdrfh, $wtrfh, 'some', 'cmd', 'and', 'args');

    [download]

    Open3 gives you also STDERR.

[reply]
[d/l]
[select]
      perldoc -f open says (among other things): 
      You are not allowed to "open" to a command that pipes both in *and* ou
+t, but see IPC::Open2, IPC::Open3, 
"Bidirectional Communication with Another Process" in perlipc for alte
+rnatives.

      [download]
      so I suggest hopping to perldoc IPC::Open2 and perldoc IPC::Open3 directly... update: changed pre- to code-tags
[reply]
[d/l]
[select]
[reply]
Re: pretending that the contents of a variable are files
by marcokuma (Novice) on Jan 28, 2005 at 13:42 UTC
    Thanks everybody for your helpful replies!

    Marco
[reply]
Re: pretending that the contents of a variable are files
by Qiang (Friar) on Jan 29, 2005 at 05:03 UTC
    Perl's I/O layers include support for input and output from a scalar. When you read a record with <$fh>, you are reading the next line from $string. 

    open($fh, "<", \$string);  # read only

    [download]
    the handle behaves in all respects like regular filehandles, so all I/O functions work, such as seek, truncate, sysread.

    hm,now i am not so sure how this will help you but I hope it does

    I found this trick from the cookbook receipt on perl.com.

[reply]
[d/l]
Re: pretending that the contents of a variable are files
by itub (Priest) on Jan 28, 2005 at 16:07 UTC
    If the program in question can read from stdin, a quick and dirty, although non-portable and non-scalable solution is the following: 
    $output = `anotherprogram <<END_OF_DATA\n$lots_of_data\nEND_OF_DATA`;

    [download]
    In my tests on Linux, it works for up to about 100 KB of data.
[reply]
[d/l]

Back to Seekers of Perl Wisdom