Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine

null output on program

by Nobby (Novice)
on Nov 19, 2022 at 00:37 UTC ( #11148249=perlquestion: print w/replies, xml ) Need Help??

Nobby has asked for the wisdom of the Perl Monks concerning the following question:


A mini summary, I have a script that has a hash ref of programs, this works normally if there is content output, but if there is an error by the program called, I get no output - including the error line.

For speed and brevity, in debugging I used traceroute as an example, if it looks up and performs the trace, it works but if it cant lookup, there is nothing., example...

sub main { print "Content-type: text/html\n\n"; print "<HTML> <HEAD>\n"; ....(omitted since this part works if real output print "</HEAD>\n"; open("runmain","$commands{$PROGRAM} $TARGET_HOST |"); while (<runmain>) { chomp $_; print "$_\n"; } print "</pre> </div></html>\n"; close ("runmain");
if I use traceroute to a hostname using -4 that resolves, traceroute output returns, however if I use -6 where it is known there is no ipv6 record, it just outputs nothing, there is no testing for null out put it should just print what it is given.

using traceroute on command line shows output

" No address associated with hostname Cannot handle "host" cmdline arg `' on position 1 (argc + 2)
But the script returns
<pre> Content-type: text/html <HTML><HEAD> .... </HEAD> <BODY bgcolor=white text=#2554C7> </pre></div></html>
As far as the script is concerned, it should have no clue the above output is any less relevant to an actual traceroute return with a reply.

I guess i'm missing the obvious, but i've spent way too many hours trying to sort this out, commenting out line by line and multi lines trying to find where it is going wrong, thus I am am here asking for some perl spiritual learning :)

I have omitted a lot of the header and html code, that would be irrelevant, I might have missed one or two so if the html looks wrong, dont worry it is OK, given the above, Thanks

Replies are listed 'Best First'.
Re: null output on program
by tybalt89 (Monsignor) on Nov 19, 2022 at 01:02 UTC

    The error message is being printed on stderr, not stdout, which is why you see nothing.


    traceroute -6 2>&1
      Ahh yes that's right, thank you guys, I new it was a simple fix
Re: null output on program
by atcroft (Abbot) on Nov 19, 2022 at 01:39 UTC

    Short form: Were I to make a semi-educated guess, it would be this: You need to capture the output of both STDOUT and STDERR.

    Long form:
    In *nix environments, many programs write their normal output to Standard Output (known as STDOUT), but write warning and error messages to Standard Error (known as STDERR). When your 'traceroute' command fails, the output from the error is written on STDERR, as illustrated by the following (STDOUT is file handle 1, STDERR file handle 2):

    # Test to an entry that does not exist in DNS $ traceroute Name or service not known Cannot handle "host" cmdline arg `' on position 1 ( +argc 1) $ # Same test, with STDOUT directed to /dev/null (discard output to STDO +UT) $ traceroute 1>/dev/null Name or service not known Cannot handle "host" cmdline arg `' on position 1 ( +argc 1) $ # Same test, with STDERR directed to /dev/null (discard output to STDE +RR) $ traceroute 2>/dev/null $

    If you are using a *nix-y OS, you may be able to get what you expect by simply appending the string '2>&1' to the command (which in many shells redirects anything sent to STDERR to STDOUT). Alternately, if you use one of the IPC::Run* modules you may be able to capture the error output explicitly.

    All that to say that in your example the 'open' line would be changed to read: open("runmain","$commands{$PROGRAM} $TARGET_HOST 2>&1 |"); (Although you might want to consider checking if the open failed or not.)

    Hope that helps.

Re: null output on program
by kcott (Archbishop) on Nov 19, 2022 at 09:17 UTC

    G'day Nobby,

    I see that the STDOUT/STDERR problem has been explained. Here's some additional comments.

    • Always start your code with the strict and warnings pragmata.
    • Let Perl do the I/O exception handling for you with the autodie pragma.
    • You'll need to escape characters that are special to HTML (e.g. change '&' to '&amp;'). In the code below, I used HTML::Escape.
    • Use lexical variables for your filehandles. Package variables have pretty much the same problems as global variables and should generally be avoided.
    • Use the 3-argument form of open(). For this particular exercise, see the "Opening a filehandle into a command" section.
    • Aim to not leave filehandles open any longer than is absolutely necessary. In the code below, see how I achieved this using an anonymous block.
    • Tip: With 'chomp $_; print "$_\n";' you've removed the newline in the first statement then added it back in the next. The chomp and subsequent \n are both unnecessary. As many Perl functions use $_ in the absence of any other expression, $_ is unnecessary in both statements. You could have written that four-line (six-line if you include the two lines of pointless whitespace) while loop as the one-line: 'print while <runmain>;'; compare with my "print escape_html($_) while <$cmd_pipe>;" (in the code below).
    • If you lay out your code with consistent indentation, it will be easier to read; in addition, this will often highlight logic errors.

    Here's some output from my command line:

    $ ls -al a_real_dir total 4 drwxr-xr-x 1 ken None 0 Nov 19 18:20 . drwxr-xr-x 1 ken None 0 Nov 19 19:10 .. -rw-r--r-- 1 ken None 0 Nov 19 18:20 demo_file_A -rw-r--r-- 1 ken None 0 Nov 19 18:20 demo_file_B $ ls -al not_a_real_dir ls: cannot access 'not_a_real_dir': No such file or directory

    Here's a demo script that uses all of the points I raised:

    #!/usr/bin/env perl use strict; use warnings; use autodie; use HTML::Escape 'escape_html'; my %commands = (ls => '/usr/bin/ls -al'); my @targets = qw{a_real_dir not_a_real_dir}; my $program = 'ls'; print "<html>\n<head>...</head>\n<body>\n"; for my $target (@targets) { my $cmd = "$commands{$program} $target 2>&1"; print '<h1>', escape_html($cmd), "</h1>\n<pre>\n"; { open my $cmd_pipe, '-|', $cmd; print escape_html($_) while <$cmd_pipe>; } print "</pre>\n"; } print "</body>\n</html>\n";


    <html> <head>...</head> <body> <h1>/usr/bin/ls -al a_real_dir 2&gt;&amp;1</h1> <pre> total 4 drwxr-xr-x 1 ken None 0 Nov 19 18:20 . drwxr-xr-x 1 ken None 0 Nov 19 19:10 .. -rw-r--r-- 1 ken None 0 Nov 19 18:20 demo_file_A -rw-r--r-- 1 ken None 0 Nov 19 18:20 demo_file_B </pre> <h1>/usr/bin/ls -al not_a_real_dir 2&gt;&amp;1</h1> <pre> /usr/bin/ls: cannot access &#39;not_a_real_dir&#39;: No such file or d +irectory </pre> </body> </html>
    "I have omitted a lot of the header and html code, ..."

    That's fine. My code is only intended as a demo example: adapt to your needs.

    — Ken

      Thanks, I've removed chomp and newline, as mentioned for brevity some code was not included (the sanitization parts that only allow set chars), this script has worked safely for over 20 years, but it was ipv4 limited, I am moving it to work with ipv6 as well. can't really do the print while runmain, as the runmain output is checked for a (simplified eg) if (runcode eq x} { print foo if /Y/; print bar if /Z/} kind of thing that it looks for in line by line output (SPF records and different options, produces this found or not found, I will be moving that into another external script one day so I can get it out of this one, it has about a dozen options as $program, and the spf is only one thats checked inline) Thanks for other pointers, I've made notes so when time comes I can clean it up more
Re: null output on program
by haukex (Archbishop) on Nov 19, 2022 at 13:51 UTC
    print "Content-type: text/html\n\n"; print "<HTML> <HEAD>\n"; open("runmain","$commands{$PROGRAM} $TARGET_HOST |");

    I pray that $commands{$PROGRAM} and $TARGET_HOST aren't based on user input, because otherwise you've built yourself a massive security hole, which I discuss here, including solutions.

    The 2>&1 workaround requires the shell to work and therefore I would definitely not recommend it. Perl does give it a bit of special treatment to avoid the shell sometimes, but that only works e.g. as long as there aren't other shell metacharacters in the command string, and I would consider it an optimization rather than some kind of security feature.

    In this case, if you don't mind waiting for the external command to finish before getting its results, I might suggest IPC::Run3. If you need the "live" output of the command, IPC::Run can provide this. Both of these modules support capturing STDERR as well. As I write in the first link above, IPC::Run doesn't explicitly say that it avoids the shell in the documentation, but looking at the code shows that it does. If you want to play it really safe, use one of the modules that explicitly state they avoid the shell, like IPC::Run3.

    (As for perhaps wanting "live" output of the command showing up in the browser, which I am guessing you might want to do based on the traceroute example, writing that out to the browser with autoflush turned on is a pretty old-school way to do it, and it may not work at all e.g. in the presence of proxies. For a modern approach, see e.g. the code I posted here.)

    Update: Added a few small clarifications.

      Thanks, as above I have sanitizing, no doubt it is not 110% foolproof and someone very clever may find a way, but its been running 20 years without incident so I think my sanitizing is, at least at an acceptable level, but I'm not so nieve to think this is as safe as bank vault, ever vigilant we must be.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11148249]
Approved by atcroft
Front-paged by kcott
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (7)
As of 2023-02-06 00:16 GMT
Find Nodes?
    Voting Booth?
    I prefer not to run the latest version of Perl because:

    Results (33 votes). Check out past polls.