Before I start my ob XML::Twig pitch... ;--)

There are a couple of modules on CPAN that are specifically designed to process record-oriented XML:

• XML::RAX, RAX is described in RAX, an XML Database API. With XML::RAX you just go through your document, one record at a time:while ( $rec = $R->readRecord()) { print "Phone = ".$rec->getField('Phone')."\n"; }. Note that this works only if each record is "flat" (each element within the record is a single value)
• XML::SAX::Machines also has a record mode, which looks suspiciously like what XML::wig does... except it's SAX-based and integrates nicely with other SAX processing. I haven't tried this module (yet) but it looks interesting.

Finally XML::Twig was designed especially for this kind of problem. You can easily get rid of part of the document after you are done processing it: you can set a twig_handler for each ArgusFlowRecord, which will call a sub that has access to everything inside the element, then call purge or flush (if you need to output the updated element) and get your memory back./,/p>

See the tutorial, section 4.3 is what you are looking for (n ote that since I wrote the tutorial I added a field method that is equivalent to first_child->text but looks nicer.

For stream-based XML processing you should probably be looking at XML::SAX. XML::LibXML and XML::Parser also support this mode.

And mirod will almost certainly be along very soon to show you how to do it with XML::Twig :)

"The first rule of Perl club is you do not talk about Perl club."
-- Chip Salzenberg

#!/usr/bin/perl

use warnings;
use strict;
$|++;

use XML::LibXML;

my $parser = XML::LibXML->new();
my $closing_root_tag = '</ArgusFlowRecord>';
my $skip_past        = '<ArgusDataStream>';

{

  $XML::LibXML::skipXMLDeclaration = 1;

  local $/ = $skip_past;
  <>; # ignore declaration and stream open tag

  local $/ = $closing_root_tag;
  my $temp_chunk;

  while ( <> ) {
    $temp_chunk .= $_;
    my $dom;
    eval { $dom = $parser->parse_string( $temp_chunk ) };
    next if $@; # keep nibbling if XML is invalid
    undef $temp_chunk;
    print $dom->toString(), "\n"; # or do other processing
  }

}
[download]

use strict;
use XML::Simple;
use Data::Dumper;

while (<>) {

  next if /xml|ArgusDataStream/;
  my $xml=$_;
  $xml.=<> while $xml !~ m|</ArgusFlowRecord>|;
  my $xmlhash=XMLin($xml);
  print Dumper($xmlhash);  
  
}
__END__
$ xmlstuff.pl <xmlinput
$VAR1='
   Flow data set 1
';
$VAR1='
   Flow data set 2
';
[download]

<?xml version="1.0">
<ArgusDataStream>
<ArgusFlowRecord>
Flow data set 1
</ArgusFlowRecord>
<ArgusFlowRecord>
Flow data set 2
</ArgusFlowRecord>
</ArgusDataStream>
[download]

Update: this is basically the same as marcello's solution, and is probably better handled with an event based parser such as XML::Sax::PurePerl, as per jeffa's suggestion in the chatterbox.

CU
Robartes-

use constant END_TAG = qr#</ArgusDataStream>#;

my $xml = "";
while (<STDIN>) {
    my $line = $_;
    $xml .= $line;
    if ($line =~ END_TAG) {
        # Process the XML here...

        # Start again
        $xml = "";
    }
}
[download]