Socrates has asked for the wisdom of the Perl Monks concerning the following question:

I've been attempting to run jslint from the commandline via spidermonkey. I came up with a silly little hack by which I could simply cat a javascript file at it, by putting it into a readline(); loop, but I wanted to try my hand at writing a small script that could potentially pass multiple files into a single instance of spidermonkey (and also to allow for argument parsing and such).

For whatever reason, I wasn't able to build the CPAN Javascript module (and, by extension, Javascript::JSLint), so I thought I'd try my hand at a simple little script and, in the meantime, increase my understanding of Open2/3.

So, I wrote the following script to test it out: 

#!/usr/bin/perl
use IPC::Open2;
use strict;

my ($JSWRITE, $JSREAD);

my $pid = open2($JSREAD, $JSWRITE, "js");

print $JSWRITE "load('jslint.js');\n";

#INPUT INTO JSLINT
open FILE, '<', "test.js";
while (<FILE>)
{
    print $JSWRITE "$_\n";
}
close(FILE);

#END INPUT

print $JSWRITE "END\n";
close($JSWRITE);
waitpid($pid, 0);

my @output=<$JSREAD>;
close($JSREAD);

print join ("\n", @output);

[download]

And, in jslint.js, I put it into a readline(); loop like this: 

    var input="";
    var line="";
    while (true){
        line=readline();
        if (line=="END") break;
        if (line!="") {
            input += line;
            input += "\n";
        }
    }

[download]

After which, it calls the function to scan the text in 'input' for errors.

And it doesn't work. And I'm pretty sure I know why, but I'd like to know for sure and also to increase my understanding of what's going on behind the scenes here and the particulars of the blocking.

So, I know that the problem lies in the readline(); loop. I know this because I can do something like the following in the perl script: 

print $JSWRITE "var foo='bar';\n";
print $JSWRITE "print(foo);\n";
close($JSWRITE);
waitpid($pid, 0);

my @output=<$JSREAD>;
close($JSREAD);

print join ("\n", @output);

[download]

And I get 'bar'. So, my understanding is that the interpreter is properly signaling that it's ready for input, and then $JSWRITE is unlocked, and everything can continue.

So my guess is that, when my hacked version of jslint.js tells the interpreter to enter into that readline() loop, that signal stops getting sent, but I don't know the particulars of it, or what I can do to diagnose or prove that.

I'm sure there's a better way to do all of this. I could use an interpeter (like Rhino), that has support for reading entire files in by one command (or I could compile it into spidermonkey). I could also read the file in a line at a time by doing something like: 

print $JSWRITE "input += '$escaped_line_of_file';\n";

[download]

But, at this point, I'm just trying to increase my understanding of IPC, and how to get more information about similar problems,

Replies are listed 'Best First'.
Re: Diagnosing blocking io (or: finding the WHY of my Open2 woes).
by almut (Canon) on Nov 18, 2008 at 22:34 UTC

    My suspicion is (though I haven't verified (because I don't have 'js' here))  that the subprocess (js) is using block buffering for its output when being run non-interactively...  in which case your readline might hang waiting for an end-of-line.

    Unless you have control over the source of the subprocess to change such buffering behaviour, you're essentially out of luck.

[reply]
[d/l]
[select]

      If this is the case you may be able to trick it by using Expect or IPC::Run to run it on a pty so that it thinks it's being run interactively.

      The cake is a lie.
      The cake is a lie.
      The cake is a lie.

[reply]
        I'd thought of both of these, but I was curious as to whether there was any way to do it without requiring non-core modules.
[reply]
      Sure, and if I'm out of luck, that's fine, but I'm also trying to increase my understanding here, so would there be any way to go about verifying that it's using block buffering? Also, aren't I sending it an end-of-line by using the newline character at the end of each print? As I said, it works if I don't enter the readline() loop via Javascript. I can issue other javascript commands perfectly fine.
[reply]

        Actually, on second thought, your problem might rather be that the subprocess' output buffer is filling up... so it stops processing anything (i.e. hangs/waits) before you get a chance to read/emtpy the buffer (as you're doing the reading after the writing).  Just another hypothesis, though :)

        How much output would you expect when things don't work?

        As to diagnosing what's going on, strace would probably be a good tool.

[reply]
[d/l]

      ...and this has been an obvious problem for decades in Unix, and yet there is still no libc that allows a simple environment variable to influence the buffering mode used for stdout...

      - tye        

[reply]
Re: Diagnosing blocking io (or: finding the WHY of my Open2 woes).
by ikegami (Patriarch) on Nov 18, 2008 at 22:18 UTC
    I would have thought open2 set the handle to autoflush, but that might not be the case. What happens when you add 
    use IO::Handle qw( );  # For autoflush

...

my $pid = open2($JSREAD, $JSWRITE, ...);
$JSWRITE->autoflush(1);

...

    [download]

    By the way, isn't `which js` the same thing as just js?

[reply]
[d/l]
[select]
      $JSWRITE->autoflush(1); Changes nothing, unfortunately. I tried that already, along with other things like: 
      use Fcntl;

...

my $flags |= O_NONBLOCK;
fcntl($JSWRITE, F_SETFL, $flags);

...

      [download]
      And, yes "`which js`" is the same... I forget why I had that in there. Still doesn't work, but I'll update the open2() call in the original post to avoid confusion.
[reply]
[d/l]
[select]

        Your program works fine for me when I run a Perl script for the child. 

        my $pid = open2($JSREAD, $JSWRITE, q{perl -e"{ $_=<>; last if /^END$/;
+ ++$i; $m=qq{$i: $_}; warn $m; print $m; redo}"});

        [download]

        (The $|=1 isn't even necessary!)

        Seems to me the problem is in the js side. But I doubt it's in readline. Are you sure your Perl script reaches END? Are you sure END is causing the child to exit?

        By the way, you're adding newlines where they already exist in two different places.

[reply]
[d/l]
[select]
Re: Diagnosing blocking io (or: finding the WHY of my Open2 woes).
by zentara (Cardinal) on Nov 19, 2008 at 15:55 UTC
    Just for the sake of experimenting, you could try a sysread instead of the <> operator, and grab 1 byte at a time. 
      while (my $nread = sysread($JSREAD, my $buf, 1)) {
        print  $buf; 
       last if $nread == 0;
        }

    [download]
    I'm not really a human, but I play one on earth Remember How Lucky You Are
[reply]
[d/l]
      I commented out the line 
      my @output=<$JSREAD>;

      [download]
      And replaced it with that code, but but there was no observable difference.
[reply]
[d/l]
        Well, at least now you know that ikegami was right, it's not a Perl piping problem, your js program isn't outputting correctly. Here is a pty example you might want to try. 
        #!/usr/bin/perl -w
# Description: Fool a process into
# thinking that STDOUT is a terminal, when in fact
# it may be a file or a pipe. This can be useful
# with programs like ps and w on linux... which
# will trunc their output to the width of the
# terminal, and, if they cannot detect the terminal
# width, use a default 80 columns. Wouldn't it be
# nice to say "ps -aux | grep etcshadow", and get
# output that looks like when you just say "ps
# -aux"? Well, that's the idea.

#try ./pseudotty "xterm -e top"
#or  ./pseudotty top

use warnings;
use strict; 
use IO::Pty; 

die "usage: ptyexec command [args]\n" unless @ARGV;

my $pty = IO::Pty->new; 
my $slave = $pty->slave;
open TTY,"/dev/tty" or die "not connected to a terminal\n"; 
$pty->clone_winsize_from(\*TTY);
close TTY;

my $pid = fork(); die "bad fork: $!\n" unless defined $pid; 
if (!$pid) {
#  $slave->close();
  open STDOUT,">&=".$pty->fileno() or die $!;
  exec @ARGV;
}else{
  $pty->close();
     while (defined (my $line = <$slave>)) {
       print $line;
     }
}

#cleanup pty for next run
$pty->close();

        [download]
        I'm not really a human, but I play one on earth Remember How Lucky You Are
[reply]
[d/l]

Back to Seekers of Perl Wisdom