Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to determine the most efficient method of parsing large (1-3 meg) RRD (Round Robin Database) files after a dump to XML on Windows.

I'm using the RRDTool-OO module to retreive my data like so: 

  $rrd->dump();

[download]
The output is dumped to STDOUT in XML format.
Is it possible to parse STDOUT and store it in some data strcuture? My other concern is that this data structure will chew up a large amount of memory, so is it possibly to use pointers at all?

I'm assuming it's faster to parse STDOUT vs dumping the data to a file and then using XML-Parser to create and store my data structure.

Any ideas or suggestions would be greatly appreciated.

Replies are listed 'Best First'.
Re: Parsing STDOUT ?
by jZed (Prior) on Feb 07, 2006 at 22:38 UTC
    Perhaps something like this 
    use IO::Scalar;

my $str = IO::Scalar->new;
select *$str;
$rrd->dump();
select *STDOUT;
# now parse $str;

    [download]
[reply]
[d/l]
Re: Parsing STDOUT ?
by Old_Gray_Bear (Bishop) on Feb 07, 2006 at 22:50 UTC
    Purely as a matter of curiosity, why not use the example in the man-page? Once you have it in the file, use your XML took-kit of choice to rock'n'roll. 
    $rrd->dump()

    Available as of rrdtool 1.0.49.

    Dumps the RRD in XML format to STDOUT. If you want to dump it into
+ a file instead, do this:

        my $pid;

        unless ($pid = open DUMP, "-|") {
          die "Can't fork: $!" unless defined $pid;
          $rrd->dump();
          exit 0;
        }

        waitpid($pid, 0);

        open OUT, ">out";
        print OUT $_ for <DUMP>;
        close OUT;

    [download]

    ----
    I Go Back to Sleep, Now.

    OGB

[reply]
[d/l]
Re: Parsing STDOUT ?
by saintmike (Vicar) on Feb 07, 2006 at 22:39 UTC
    XML::Simple is your friend: 
    my $ref = XMLin([<xml file or string>] [, <options>]);

    [download]
     Never mind, that won't work from STDOUT, you'll have to tie first, as explained in the RRDTool::OO documentation:

    Dumps the RRD in XML format to STDOUT. If you want to dump it into a file instead, do this: 
        my $pid;

    unless ($pid = open DUMP, "-|") {
        die "Can't fork: $!" unless defined $pid;
        $rrd->dump();
        exit 0;
    }

    waitpid($pid, 0);

    open OUT, ">out";
    print OUT $_ for <DUMP>;
    close OUT;

    [download]
    Just replace the last couple of lines by 
        use XML::Simple;
    my $data = join '', <DUMP>;
    my $ref = XMLin($data);

    [download]
    and you'll have the data structure in $ref.
[reply]
[d/l]
[select]
      I seem to get this error on windows: 
      '-' is not recognized as an internal or external command,
operable program or batch file.

      [download]
      Any idea why?
[reply]
[d/l]

Back to Seekers of Perl Wisdom