BenHopkins has asked for the wisdom of the Perl Monks concerning the following question:

Hey monks! I grovel in the gravel and genuinely genuflect gratefulness.

I'm using XML::Twig (3.30) and want to replace all entities in the data with ascii equivalents, i.e. — with --.

I tried using a _default_ root handler like this: 

my $t = XML::Twig->new(
    twig_roots => {'nitf/body/body.head/hedline/hl1' => \&fix_hl1,
                   'nitf/body/body.head/hedline/hl2' => \&fix_hl2,
                    _default_ => \&fixup,
                  },
        twig_print_outside_roots => 1,
        keep_encoding => 1,
);

[download]
And the subroutine fixup looks like this: 
sub fixup {
    my ($tree, $elem) = @_;

    my $tag = $elem->tag;
    plog("in fixup:  tag = $tag");

    if ($fixup_tag{$tag}) {
        my $t = $elem->text;
        conv_chars(\$t);
        $elem->set_text($t);
    }

    $elem->print;
}

[download]
The fixup_tag hash has p, byttl, person, hl1, etc., otherwise I was changing higher level tags, like nitf and that removed all the internal tags (not a good thing).

Question: is there some way to apply a filter to just the data and leave the tagging and attributes alone.

I have a feeling that a good answer to the above will obviate what comes next:

Problem: after running this, the output is invalid. The input document has <nitf><head><title></title> followed by a bunch of <meta> tags. The output has (after the DOCTYPE) a <title></title>, the <meta> tags, then the <head>, with <title> and <meta> duplicated, then comes a copy of the whole mess, this time inside the <nitf></nitf> tags!

Obviously, I'm doing something wrong.

Anybody have an idea of what it is?

Replies are listed 'Best First'.
Re: XML::Twig question and problem
by mirod (Canon) on Jun 28, 2007 at 05:04 UTC
    Question: is there some way to apply a filter to just the data and leave the tagging and attributes alone.

    Look at the output_text_filter option, it filters only text and attribute values.

    For your other problem, you can't use _default with print_roots, that just messes up everything. The output_text_filter thing will take care of it. And if you still have a problem with the beginning of the document being duplicated, get the development version of the module, I fixed a bug with it that last week.

[reply]
      I saw that, but couldn't figure out how to use it.

      If I say 

      ...  output_text_filter => \%fixup,

      [download]
      and 
      sub fixup {
    my $t = $@;
    conv_chars(\$t); # subroutine that operates on a string
    return($t);
}

      [download]
      ...doesn't work.

      When I saymy $t = shift;, $t contains tag names.

      I'm missing something here.

[reply]
[d/l]
[select]

        OK, so here the problem is that the filter opions do not operate when using the twig_print_outside_roots. At least they are not applied to the data outside the roots. Which makes sense as you are supposed to only care about what's inside the roots, that's the whole point of the "roots mode".

        Below is a way of solving your problem. I must say I would not use it though. Instead I would do a pre or post processing of the data, using regexps or maybe Text::Unidecode, to replace the characters you don't want. I assume you want to replace those characters everywhere in the document, so most likely you don't need to use an XML aware tool.

        #!/usr/bin/perl

use strict;
use warnings;

use XML::Twig;

my $t = XML::Twig->new( twig_handlers => { hl1 => \&fix_hl1,
                                           hl2 => \&fix_hl2,
                                           _all_ => \&fixup,
                                         }, 
                         keep_encoding => 1,
                         keep_spaces => 1,
                     )
                 ->parse( \*DATA);



sub fixup
  { my( $t, $elt)= @_;
    foreach my $pcdata ($elt->children( '#PCDATA'))
      { $pcdata->set_text( conv_chars( $pcdata->text)); } 
    $elt->flush unless( $_->in( 'hl1') || $_->in( 'hl2'));
  } 

sub conv_chars { my $t= shift; $t=~ s{&#8212;}{--}g; return $t; }
sub fix_hl1 { $_->set_tag( 'new_hl1')->prefix( 'fixed hl1:'); }
sub fix_hl2 { $_->set_tag( 'new_hl2')->prefix( 'fixed hl2:'); }

__DATA__
<!DOCTYPE nitf [
<!ENTITY foo "bar"> ]>
<nitf><s><hl1>hl1 with a &foo; and an other &foo; and &#8212; and &#82
+12;</hl1>
         <p>text, no foo</p>
         <p>text, no foo but &#8212;</p>
         <p>text, with &foo;</p>
         <hl2>hl2 with a &foo; and an other &foo; and &#8212; and &#82
+12;</hl2>
         <hl2>hl2 no foo </hl2>
         <hl2>hl2 with a bold <b>&foo;</b>, <b>&#8212;</b> and an othe
+r &foo;</hl2>
      </s>
</nitf>

        [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom