Re: Dynamically breaking up a data set

I strongly suggest that you use an XML-processing CPAN package such as, say, XML::LibXML, and that you “query” the file using an XPath expression. If you select and use a module that is designed to work with large XML files, will provide you with the means of obtaining and iterating through a list of nodes that are extracted from the file, no matter how big the file may be. By using XPath expressions, you specify what you are interested in and do not have to write one-off Perl logic to navigate through the structure.

You might also be able to get even further along with your task by using so-called XSLT stylesheets (which are not at all like CSS stylesheets). Very sophisticated XML transformations and queries can be specified using no programming per se at all. You might even find that you can do the entire task this way, using command-line tools and actually eliminating Perl altogether.

Replies are listed 'Best First'.
Re^2: Dynamically breaking up a data set by Jenda (Abbot) on May 24, 2012 at 08:17 UTC
Erm .... sorry to bother you with details, but ... the OP is not querying any XML. He's not parsing any XML file and while I understand that someone twisted enough might like XSLT for XML-to-something transformation, in this case that's not gonna fly. There is no XML to start with! The data are in a database and the XML is to be produced. The data should be escaped though to prevent problems if the PACKINFO contains less-than signs or other stuff special to XML. Eg. like this: `use XML::Rules; my $parser=new XML::Rules(rules=>{}); ... $XMLContent .= $parser->toXML(packet_ref => {packID => [$foo], packInf +o => [$bar]}) . "\n"; # or, with formatting # $XMLContent .= $parser->toXML(packet_ref => {packID => [$foo], packI +nfo => [$bar]}, 0, ' ') . "\n";` [download] Jenda Enoch was right! Enjoy the last years of Rome.	[reply] [d/l]

Replies are listed 'Best First'.

Re^2: Dynamically breaking up a data set
by Jenda (Abbot) on May 24, 2012 at 08:17 UTC

Erm .... sorry to bother you with details, but ... the OP is not querying any XML. He's not parsing any XML file and while I understand that someone twisted enough might like XSLT for XML-to-something transformation, in this case that's not gonna fly. There is no XML to start with! The data are in a database and the XML is to be produced.

The data should be escaped though to prevent problems if the PACKINFO contains less-than signs or other stuff special to XML. Eg. like this:

use XML::Rules;
my $parser=new XML::Rules(rules=>{});
...
$XMLContent .= $parser->toXML(packet_ref => {packID => [$foo], packInf
+o => [$bar]}) . "\n";

# or, with formatting

# $XMLContent .= $parser->toXML(packet_ref => {packID => [$foo], packI
+nfo => [$bar]}, 0, '  ') . "\n";
[download]

Jenda
Enoch was right!
Enjoy the last years of Rome.

[reply]
[d/l]