Script to Parse 1 large xml file to many smaller files

LF has asked for the wisdom of the Perl Monks concerning the following question:

I am fairly new to perl and am trying to write a Perl script to parse 1 large xml file containing many records into many xml files containing 1 record each. The records are between <rdf: Description> tags.

So far, I have a script that reads in an infile and a directory and creates an outfile in the specified directory to which "print OUTFILE ("New Information"); can add new information (below)

Can anyone give me suggestions on any of the following?

1. reading from the infile and writing to the outfile

2. parsing the xml text using the <rdf: Description> tags

3. Saving the xml header (the <?xml version="1.0" encoding="UTF-8"?>) and attaching it to the beginning of each of the individual records

4. using the file name listed under <dc:identifier> as the outfile name instead of the record count?

Thanks!

#!/usr/bin/perl -w

# Sort out the input and output files

my ($infile, $dir) = @ARGV;
defined($infile) && defined($dir) || usage();
$infile && $dir || usage();
$record_count = 0;
$record_element = 'rdfescription';
$filename_element = 'dc:identifier';

#Create Infile
if (@ARGV > 0)
{
open(INFILE, "<$ARGV[0]") || die "Failed to open $ARGV[0]\n";
$infile = "INFILE";
}
else { $infile = "STDIN"; }

#Create Outfile
if (@ARGV > 1)
{
open(OUTFILE, ">$ARGV[1]/$record_count.xml") || die "Failed to open $A
+RGV[1]/$record_count.xml\n";
$outfile = "OUTFILE";
}
else { $outfile = "STDOUT"; }

else
print OUTFILE ("New Information");

#$stop = ((INFILE == $record_element) || (INFILE == eof));
#if ($stop)

close (INFILE);
close (OUTFILE);


sub msg {
print @_, "\n";
}

sub usage {
msg("Usage: $0 <file> <directory>");
exit(1);
}
[download]

Comment on Script to Parse 1 large xml file to many smaller files Download Code

Replies are listed 'Best First'.
Re: Script to Parse 1 large xml file to many smaller files by edan (Curate) on Aug 09, 2004 at 19:03 UTC
XML::Twig ? XML::Writer ? XML::Parser ? -- edan	[reply]
Re: Script to Parse 1 large xml file to many smaller files by murugu (Curate) on Aug 10, 2004 at 04:30 UTC
As mentioned by Edan, I think XML::Twig is an excellent option for this kind of thing. Below i have given some code which may u can modify for ur needs. The below code will create three files named as per dc:identifier element. For Further details see XML::Twig. use XML::Twig; undef $/; $x=<DATA>; ## i have assumed that dc:identifier as first child of rdf:Description +. $p=new XML::Twig(twig_roots=>{ 'rdf:Description'=>\&rdf}, pretty_print => 'indented'); $p->parse($x); sub rdf { my ($a,$b)=@_; my $filename=$b->first_child_text; open(FOUT,">$filename") or die("$filename not opened for writing\n +"); print FOUT q(<?xml version="1.0" encoding="UTF-8"?>); print FOUT $b->sprint; close(FOUT); } __DATA__ <?xml version="1.0" encoding="UTF-8"?> <ab:root> <rdf:Description> <dc:identifier>ab.xml</dc:identifier> <dc:abc>abc</dc:abc> </rdf:Description> <rdf:Description> <dc:identifier>bc.xml</dc:identifier> <dc:abc>bcd</dc:abc> </rdf:Description> <rdf:Description> <dc:identifier>ef.xml</dc:identifier> <dc:abc>efg</dc:abc> </rdf:Description> </ab:root> [download]	[reply] [d/l]