in reply to Re^3: XML::Twig and file encoding
in thread XML::Twig and file encoding

ok sorry, brain cramp!

#!/usr/bin/perl --
use Path::Tiny qw/ path /;
use Data::Dump qw/ dd /;
use XML::Twig;

my $infile  = 'twiggy-test.xml';
my $outfile = 'twiggy-WHAT.xml';
dd( path( $infile )->slurp_utf8 );
getTwiggy( $infile , $outfile );
dd( path( $outfile )->slurp_raw );
sub getTwiggy { 
 
 # set up the XML parser:
 my $twig= XML::Twig->new( 
   comments => 'keep',
 #  keep_encoding => 1,
   keep_spaces=> 1,
   twig_handlers => { li => sub_ix_strings($infile),
                 
               },
 );
 
 $twig->parsefile($infile);

   print "  Saving $outfile...\n";
   open(NEW,">$outfile") || die("cannot open file $outfile for writing
+.\n");
   $twig->flush( \*NEW);
   close(NEW);

}

sub sub_ix_strings {
  my($file) = @_;
  return sub {    
   my($twig,$elt) = @_;
   $elt->set_att('test',"hello");
  }
}
__END__

[download]

OUTPUT (looks the same, but file is now ANSI not UTF-8):

<?xml version="1.0" encoding="UTF-8"?>
<reference id="ids_excal_142" xml:lang="en-us">
<title>The ISO character set</title>
<refbody>
<section><p>Each ISO character has its own value, except that lowercas
+e
characters are translated to uppercase. Two characters that have no
uppercase equivalent are: <ul>
<li test="hello">German small sharp s (ß - 0xDF)</li>
<li test="hello">Lowercase y diaeresis/umlaut (ÿ - 0xFF)</li>
</ul></p></section>
</refbody>
</reference>

[download]

So the output looks ok in my text editor (TextPad) but when I open it in an XML editor (Arbortext or Oxygen) the special characters don't display correctly.

Thanks! Scott

Replies are listed 'Best First'.
Re^5: XML::Twig and file encoding
by slugger415 (Monk) on Aug 01, 2014 at 17:11 UTC

    Also this is what's printed to STDOUT:

    "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<reference id=\"ids_exc
+al_142\" xml:lang=\"en-us\">\r\n<title>The ISO character set</title>\
+r\n<refbody>\r\n<section><p>Each ISO character has its own value, exc
+ept that lowercase\r\ncharacters are translated to uppercase. Two cha
+racters that have no\r\nuppercase equivalent are: <ul>\r\n<li test=\"
+hello\">German small sharp s (\xDF - 0xDF)</li>\r\n<li test=\"hello\"
+>Lowercase y diaeresis/umlaut (\xFF - 0xFF)</li>\r\n</ul></p></sectio
+n>\r\n</refbody>\r\n</reference>"

    [download]
[reply]
[d/l]
[reply]

        I see! This was the bit I was missing:

        open(NEW, '>:encoding(UTF-8)', $outfile)

        That seems to do the trick. Thanks for the tips! Really appreciate it.

[reply]
[d/l]

        hello, sorry to keep harping on this, but I continue to experience character mishaps when using XML::Twig.

        The files are UTF8. They contain things like smart quotes, n-dashes, special spaces, etc. When I resave them I set UTF8:

           open(NEW,'>:encoding(UTF-8)', $outfile)

        But then those special characters turn to gibberish. So "0-9" becomes "0–9"

        My Twig setup:

         my $twig= XML::Twig->new( 
   comments => 'keep',
   keep_encoding => 1,
   pretty_print => 'indented',
   twig_handlers => { ...}
);

        [download]

        Sorry to be slow here but I'm at a loss as to how to do this properly. (I did read the topics suggested but they seem to suggest encoding/decoding, and then it says not to do it if you don't want them encoded.)

        Thanks, Scott

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom