Take a look at the core Encode and Encode::Supported modules, specifically the encode and decode commands. The PerlIO::encoding module calls those commands internally.
Am I going to need to swap the bytes on every character I read in?
No way. Encode is set up to handle both Big-Endian and Little-Endian. You're having a problem with the Byte Order Mark because you specify an encoding of ucs2, which is an alias for the big-endian format UCS-2BE, but your BOM indicates that the data from the stream is in little-endian format. Perl doesn't like that, and it complains.
Here's a script which illustrates that what you want to do is possible:
#!/usr/bin/perl use strict; use warnings; binmode( STDOUT, ':utf8' ); my $big_endian_BOM = pack( 'n', 0xFEFF ); my $smiley = pack( 'n', 0x263A ); my $ack = pack( 'n', 0x0006 ); my $newline = pack( 'n', 0x000A ); my $data = $big_endian_BOM; $data .= "$smiley$newline$ack$smiley$newline$ack"; open( my $fh, '<:encoding(ucs2)', \$data ) or die $!; local $/ = $ack; while (<$fh>) { chomp; print; }
It prints two smileys, each on its own line. No errors or warnings.
In reply to Re: UCS2 Internationalization file parsing
by creamygoodness
in thread UCS2 Internationalization file parsing
by sfinster
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |