I did find a parse_more method in XML::Parser::ExpatNB, which looked like it might do the trick, but never got around to actually seeing if it could parse unbalanced xml snippets.

UPDATE

I did some hacking on the ExpatNB solution, here's what I came up with. It would have been much better than stream mender I put together myself.

#!/usr/bin/env/perl

use strict;
use warnings;

use XML::Parser::Expat;
use Data::Dumper qw(Dumper);

my $parser = XML::Parser::ExpatNB->new();

$parser->setHandlers('Start' => \&sh,
                     'End'   => \&eh,
                     'Char'  => \&ch);

foreach my $snippet qw( < bro ke nx ml> con tent < /bro kenxm l> ) {
    print "Waiting for an event...\n";
    $parser->parse_more($snippet);
}

sub sh {
   print "A start element: ", Dumper($_[1]), "\n";
}

sub eh {
   print "An end element: ", Dumper($_[1]), "\n";
}

sub ch {
   print "Some Data: ", Dumper($_[1]), "\n";
}

1;
[download]

This is exactly what I was searching for... How it eluded me I cannot explain.

Many thanks!

-nuffin
zz zZ Z Z #!perl

OK, I've uploaded XML::SAX::Expat::Incremental and XML::SAX::ExpatNB to the CPAN. The first supports incremental parsing, and the second does adds nonblocking reads from a handle, both using the XML::SAX interface.

Thanks for your help!

-nuffin
zz zZ Z Z #!perl

Might not be perfect, but it just might be what you need.

I demonstrate this technique in one of my columns.

Oh, and apparently the XML::LibXML parser has a parse_chunk method which works similar to HTML::Parser's parse method. Try that too.

That's a wonderful column, merlyn. IIRC it was the first one by you that I've read (possibly the first about perl I've read ever).

Although it seems to provide me with a good solution, I think I will go with redhotpenguin's, as it will probably be more robust than HTML::Parser, and I already know for certain that it does exactly what I need.

Update: Appearantly my memory is pulling tricks on me, that was not so long ago. Nevertheless, it and this one I remember as mind opening, in terms of using one tool to get another's job done.

-nuffin
zz zZ Z Z #!perl

One of the major selling points of XML is that it is well structured: an XML parser can determine if the data is well-formed.

It sounds like the right way to go would be to use a buffer that captures the XML as it comes in, then when it gets the closing tag that matches the first opening tag, cut the buffer off and send it to be parsed.

Perhaps the reason you're having a hard time is because you're fighting all of the things that have been done to make XML in Perl easy and reliable.

If that's the case, I can just wait for </xml>, and then parse that string up.

But I need a SAX like interface, generating events for data, as it enters. I'm not concerned about the XML's tree at the end, because I need it's data parsed (as far as possible), as it arrives. The opening of an element concerns me before it's end is even on the data stream, let alone actually parsed.

-nuffin
zz zZ Z Z #!perl

Can't you just pipe the data to the parser as it comes, so the parser thinks it is just a regular file? Have a process that gets the data, then pipes it to the process that parses it.

Given

parser a
parser b

stream a
stream b
[download]

I would like to give parser a data from stream a, and parser b data from stream b, simultaneously, in the same process (no threads).

You can just use a select loop, and read nonblocking on your own, to get data from multiple streams at the same time.

What I need to get this done is parsers that can be told 'here is more data', and have them return even if that's not a complete XML document. They parse as much as they can given the current buffer which they keep. When more data will become available, they continue parsing.

Basically, instead of reading more from a handle, i'd like the parsers to return control to me, so I can give them more strings.

-nuffin
zz zZ Z Z #!perl