This is an interpreter error than than a script or module error. To be honest there is not much you can do on the programming side to fix it.

Your options boil down too,

1. There may be bad RAM on the system, highly unlikely but it is a possibility.
2. The library the interpreter is using. Upgrade the version of Perl or an extreme case downgrade the version of Perl that is installed. Don't really recommend downgrading though unless you have no other choice.
3. The OS is a problem. Try moving the program to another machine. Try it from another version of Windows or possibly a *NIX system. If you go to a new OS the interpreter may not have this bug at all.

1. I will, as soon as possible, run Memtst86 to verify the hardware. I agree, this is worthwhile, but unlikely.

2. I expect my Perl is not the most up-to-date, (not a long way away, but possibly somewhat crusty). I can certainly investigate this. Similarly the module, although I did download the module last autumn, I do not know how frequently this is updated. Again, I can investigate this.

3. Whilst I may be able to try this on other OS's, it will not solve my problem. The target OS is XP, that is a constraint.

If this is a size issue, I could possibly chop the XML buffer into a number of smaller buffers by traditional means, then call the processing routine repeatedly each invocation processing say 1000 records.

I create the object with an xml->abc parameter. Is there a formal destructor or reset method for this object or do I need to create a new object for each inquiry? The documentation at CPAN does not show a destructor.

Well, another option is to just use another module, I recommend, XML::Twig as another possibility. It can deal with large files very well because you can flush (clear from memory) the parts of the document do not need anymore. This would reduce the size of the document as well.

Just to update, memtest86 showed the RAM to be fine. The XPath I have is the latest available. My Perl core was slightly out of date, 5.8.7, I have updated to 5.8.8 this evening. The error persists.

Loading 32MB of XML into a DOM structure is bound to use quite a lot of memory, but it seems the XML::XPath->find() is buggy. In either case you'd be better of using something that doesn't build the whole structure, expecially if as you said in the other node you only need very little out of the large XML. XML::Twig or maybe XML::Rules ;-)

It's hard to give you an example without knowing the XML, but using XML::Rules you might start with:

my $parser = XML::Rules->new(
 rules => [
  'computers,and,other,tags,you do not care about at all' => 'skip',
  '^user' => sub {return $_[1]->{teamid} eq $CONFIG{teamid} },
  'user' => 'as array',
   # only the <user> tags containing the right attribute will be proce
+ssed
  '_default' => 'as is',
  'tags,with,no,attributes' => 'content',
 ]
);

my $data = $parser->parse($the_xml);
[download]

Thanks for the steers, I have been browsing the Twig and Rules documentation today. I will download the packs this evening and experiment with them.

The xml is very simple...

<users>
 <user>
  5-6 simple key's here
  <teamid>100</teamid>
 </user>
</users>
[download]

... there are many thousands of user tags, but I am only interested in the 100 - 150 records with the correct teamid.

It's a shame the teamid is a subtag and not an attribute, you will be unable to use the "start" rules. Which will make the script a bit slower, but not more memory hungry. In this case I think

my $parser = XML::Rules->new(
 rules => [
  _default => 'content',
  user => sub {
   return unless $_[1]->{teamid} = $_[4]->{parameters}{teamid};

   delete $_[1]->{_content}; # delete the textual context (whitespace)
   return '@user' => $_[1];
  },
  users => 'pass no content',
 ]
);

my $users = $parser->parsefile( $the_xml_file, {teamid => 100};
[download]

should give you the data for the right users only.

Jenda


Support Denmark! Defend the free world!

I don't have access to a system with XML::XPath installed as of this writing, when I get it I will update if necessary, but I believe the problem is that your xpath query might contain a syntax error. If $CONFIG{teamid} is a string, it needs to be in single quotes for a string equals predicate (the part in brackets). XML::XPath will sometimes fail spectacularly when given a bad query.

  $usernodeset = $xpu->find("/users/user[teamid='$CONFIG{teamid}']");
[download]

The teamid value is an integer. Reading the XPath spec, it should not be necessary to quote it. I can try of course!