Re: How do I capture large output in a perl variable

my $var = `property <dataid>`;
print "<td align="middle">$var</td>";
[download]
This command (property) sometimes produces an output of size almost 0.5Mb to 1Mb for some values of <dataid>(I checked the size of the output by executing the "property" command on the web server CLI).

So, your plan is to up upwards of 1MB of data into a single "<td>" element in the html that you output? Okay, I guess.

Regarding the presence of XML-tagged stuff in the middle of a plain-text stream: are you trying to show that verbatim, making all the XML tags visible?

And regarding your testing of the "property" program on the command line: are you sure that all the data ouput from this program is text? (Could there be non-printing control characters or other non-text, binary content? If so, what sort of operating system are you using, and do you need to specify "binary mode" for reading the data?)

It might be better (or at least, easier to debug) with a pipeline open statement:

my $var;
open( PROG, "property $dataid |" );
# binmode PROG;  # might need to do this?
{
    local $/;  # set input rec. separator to undef
    $var = <PROG>;  # slurp output from property
}
close PROG;

$var =~ s/</&lt;/g; # make sure browsers don't see XML tags as tags

# if there is binary data, there's more you'll need to do
# to make it presentable to a browser...
[download]

If you still have problems with that, you can switch to reading the "property" output line by line, and/or add some diagnostics, and/or move the data to the output HTML stream in smaller chunks.

Comment on Re: How do I capture large output in a perl variable Select or Download Code

Replies are listed 'Best First'.
Re^2: How do I capture large output in a perl variable by vsmurthy (Initiate) on Mar 23, 2005 at 16:09 UTC
Hi, Thanks for the suggestion. I got the html formatting working. Regarding the incomplete output, I tried using pipe open like you suggested but I am still not able to get the entire output(just the same incomplete output). Also I tried to redirect the output from the "property" command to a file on my account on the webserver instead of directly sending it to the browser and still the file didn't contain the entire output. It just contained the incomplete output that I used to get earlier on the browser. But if I log onto my account on the webserver and run the "property" command and redirect it to a file, I get the COMPLETE output in the file. I also observed that the file was about 400Kb in size which is not too big. Any suggestions on what else I could try? Thanks, vinay	[reply]
Re^3: How do I capture large output in a perl variable by Anonymous Monk on Mar 30, 2005 at 14:35 UTC
Hi guys, Thanks for all your suggestions. I did learn some good debugging techniques in perl. I finally got the script working to retrieve the large output. The problem was that the web admin had put a cap on the apache memlimit directive and hence I wasn't able to retrieve the complete output. Once we increased that limit observing the memory requirements of the "property" command, everything started working!! :) Thanks a lot :) Regards, Vinay	[reply]