Jointly parsing characters and entities in XML::Twig

eff_i_g has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Twiglets,

I'm working with a large XML document that has pairs of double quotes throughout. I'm trying to convert the bland--Unicode names follow--"QUOTATION MARK"/"QUOTATION MARK" pairs into appealing "LEFT DOUBLE QUOTATION MARK"/"RIGHT DOUBLE QUOTATION MARK" pairs. I thought this would be an easy task by using char_handler, but entities throw a wrench in the process.

For demonstration purposes (and ease of reading), the example code is replacing the double quote pairs with single quote pairs; observe:

This...

use warnings;
use strict;
use XML::Twig;

my $twig = XML::Twig->new(
    char_handler => sub {
        my ($txt) = @_;
        $txt =~ s/"(.*?)"/'$1'/g;
        return $txt;
    },
    ### Note: These are included for terminal ease.
    ### The problem still exists without them.
    pretty_print => 'indented',
    keep_encoding => 1,
);
$twig->parsestring(*DATA);
$twig->flush();

__DATA__
<root>
    <data>"A", "B&#x2014;C", "D"</data>
</root>
[download]

Yields...

<root>
  <data>'A', "B&#x2014;C', 'D"</data>
</root>
[download]

And I was expecting...

<root>
  <data>'A', 'B&#x2014;C', 'D'</data>
</root>
[download]

Is it possible to parse the content and the entities together in XML::Twig, or must I result to looking for partial matches and saving open and close states?

Please note: I'm doing a lot more than just swapping out quotes, so changing XML modules is probably not an option.

Thanks.

Comment on Jointly parsing characters and entities in XML::Twig Select or Download Code

Replies are listed 'Best First'.
Re: Jointly parsing characters and entities in XML::Twig by mirod (Canon) on Sep 12, 2008 at 16:31 UTC
You have it nearly right, you just have to replace `char_handler` by `text_output_filter`, and everything works OK. I strongly suspect (I have to check) that what happens is that during the parsing, XML::Twig receives 3 events for the `data` element, 1 before the entity, 1 for the entity, and 1 after the entity. As of now, the char_handler is thus called 3 times, which might be considered a bug, I'll look into it. By using `text_output_filter` the substitution is performed later, once the 3 texts have been merged. Does that solve your problem?	[reply]
Re^2: Jointly parsing characters and entities in XML::Twig by eff_i_g (Curate) on Sep 12, 2008 at 16:56 UTC
Or `output_text_filter`, rather? I was digging through that meaty POD yesterday and I obviously missed that one. Yes, this appears to have solved my problem. mirod, thanks for creating, maintaining, and supporting such a grand module! :)	[reply] [d/l]
Re^3: Jointly parsing characters and entities in XML::Twig by mirod (Canon) on Sep 12, 2008 at 17:15 UTC
Or output_text_filter, rather Oops! The sad thing is that I did a cut'n paste. My test code did not do the replace at all, but I didn't see it ;--( I'm glad I was close enough that you managed to get it right! Which is why you (and by that I mean... I) should always write tests as real tests, using Test::* and comparint the expected result to the real one. That will save you embarrassment, and will prevent a few bugs every now and then.	[reply]