Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot

Re: Synchronizing STDERR and STDOUT

by nothingmuch (Priest)
on Sep 21, 2006 at 11:27 UTC ( #574106=note: print w/replies, xml ) Need Help??

in reply to Synchronizing STDERR and STDOUT

FWIW, why is TAPx::Parser wrapping around the stream? Why isn't a plumbing loop pulling from the stream and pushing to the parser?

Then you don't necessarily need nonblocking reads, etc at the IO level - you could push that down to POE or whatever.

XML::Parser has a non blocking interface which i've always liked due to it's simplicity - you just push strings when you have them, and it generates events. If you put in a partial string then the parser's state machine will simply be in that state waiting for more input.

This way you can have e.g. TAPx::Parser::Harness::Win32, *Socket, *POE, *Whatever, all reusing the parser without needing to model an iterator API around the various platform specific quirks.

Update: to clarify that last part - you only truely need non blocking IO if you need to parse multiple streams simultaneously, and as long as the parser has a push api flexible enough to be reentrant (multiple parsers instantiated and with their own state simultaneously) then there's no reason why it can't deliver callbacks in only when it's ready.

Update 2: POE::Filter::XML is written over XML::SAX::Expat::Incremental which is basically a SAX wrapper for the ExpatNB interface. That might be a nice example.

zz zZ Z Z #!perl

Replies are listed 'Best First'.
Re^2: Synchronizing STDERR and STDOUT
by Ovid (Cardinal) on Sep 21, 2006 at 13:39 UTC

    A couple of comments: this is for TAPx::Parser and one of the requirements is to have nothing which prevents it from installing in a fresh Perl install (i.e., no external dependencies), that means POE and friends are out of the question. Even if I make them optional, that doesn't solve my root problem :)

    As for the plumbing loop, while I do like that idea, this thing is rapidly becoming far more complex than desired. Too many layers of indirection/abstraction are going to make this unmanageable. Already I have one major design flaw because of this problem and it's slowing down development. I might consider this option if I have no choice, but for now, I just want to pull from a stream. However, comments that others are making are rapidly convincing me that the synchronization issue can't be solved downstream. I do, however, have ideas on how to solve that little nit.


    New address of my CGI Course.

      I was suggesting you remove a layer of abstraction - instead of keeping the stream loop inside of TAPx::Parser, let it remain outside so that it doesn't have to worry at all about blocking and synchronization and what not.

      The POE part is just to demonstrate that this approach (push parsing) works well in more situations than stream based iterators (that is, streams are usable with push parsers, but not vice versa).

      zz zZ Z Z #!perl
Re^2: Synchronizing STDERR and STDOUT
by nicholasrperez (Monk) on Sep 21, 2006 at 13:48 UTC
    You get ten points for pimpin' my code ;)

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://574106]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (2)
As of 2022-11-28 19:04 GMT
Find Nodes?
    Voting Booth?