Sorry for the delay in responding -- I'm the only IT person in my company, and a few days after posting this question I got snowballed -- this is the first chance I've had to get back to my programming.

I'm not sure I can say something is "missing" from Twig. It is currently working just fine -- in my initial testing and such. The only "problem" I am having is a design issue.

The way version-2 of my system works, each data record is in a unique file; each file was plain ascii; each file had a data layout like this:

Everything worked fine until they started doing the cut/paste thing and dumping in full email text and/or web pages and/or resumes, etc. So I started tinkering about with encoding and such, with limited success.

Then I stumble on this whole XML concept, and realize that this is just what I've been needing to overcome some of the issues I was working with -- so now I'm moving my data to XML (and now it is v3.1), and the layout looks like:

I had a "VOC" file (a carryover from my old PickBASIC days) that had the layout of each "fieldname" -- and converting that to XML (esp using XML::Twig) was a beautiful thing. It now works like a champ, and I love it! There is one XML file (voc.xml) that is easy to read and modify and maintain; reading it into the program is almost too trivial; extracting the layout info is a piece of cake!

Now, I could do as some suggested, and that is have one file per database, and have the multiple records in it, and that'd be okay. I can see how that might be good (and how Twig could be exploited to make that work). But my one-record-per-file concept makes things work well for the web-based projects I am working on (limited record locking issues), so I don't want to walk away from it just yet.

The point that I was looking to get around was that I wanted to make a generic enough data entry and data display routine that would grab any of the records (no matter what the defined layout was) and edit it or display it. To do that, my thoughts were that I had to either: 1) read in and process the DTD, or 2) have Twig automagically read in and process the DTD. But XML::Twig doesn't deal with the DTD, so that was my problem.

So that was the basis of my comments and question. If XML::Twig processed the DTD and could return to me the "complete list" of fields to be expected, then nobody would have heard from me. I would have stuck it into each item, and that would have been that. As it is, I am now getting back to trying to figure out how to grab that info from either the standalong DTD file or the internal DTD info in the XML file.

And since I spent the time since I posted this original message putting out fires instead of working on the project, I am just today starting to look back at that question again.

I guess the bottom line question is: how do folks keep up with the full list of what fields are supposed to be in the XML file?

You can wrestle some information out of XML::Twig: you need to use the load_DTD option to read the external DTD, then get the list of elements in the dtd and their models using the model method. It is not perfect, and you then have to do some (simple) post-processing to get just the sub element names.

For example:

#!/usr/bin/perl -w
use strict;
use XML::Twig;

my $t= XML::Twig->new(load_DTD => 1, error_context => 1)->parsefile( '
+test.xml');

my @elts= $t->model;
print "all elements: ", join( ', ', @elts), "\n";

foreach my $elt (@elts)
  { my $model= $t->model( $elt);
    print "model of $elt: $model\n";
    my @subelts= grep { $_ && $_ ne 'PCDATA'} split( /[^\w:-]/, $model
+);  # keep only element names
    # if you want an array of unique element names you then need this
    my %subelts= map { $_ => 1 } @subelts;
    @subelts= sort keys %subelts;
    print "elements in $elt: ", join( ', ', @subelts), "\n";
  }
[download]

Does this help?