Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re: Module to read - modify - write text files in any unicode encoding

by moritz (Cardinal)
on May 19, 2008 at 21:32 UTC ( [id://687471]=note: print w/replies, xml ) Need Help??


in reply to Module to read - modify - write text files in any unicode encoding

Encode is "core" since perl 5.7.3 and handles pretty much any encoding you can think of. You should use it.

To write a file with a specific encoding, it's enough to do things like this:

open (my $file_handle, '>:encoding(UTF-8)', $filename) or die $!; print $file_handle $text_string; close $file_handle;

(Update: I didn't see you already used that, sorry for the noise.)

Guessing character encodings can be done with Encode::Guess, but it can never be done reliably.

I know of no module that combines encoding guessing with file slurping, so it might be worth the effort. But don't roll any encoding handling code by hand, it's all been done before and properly tested.

See perluniintro, perlunitut and perlunicode for details, I also wrote a short article on the subject.

(Update: Fixed article link, thanks for reporting Rudif.

  • Comment on Re: Module to read - modify - write text files in any unicode encoding
  • Download Code

Replies are listed 'Best First'.
Re^2: Module to read - modify - write text files in any unicode encoding
by Rudif (Hermit) on May 20, 2008 at 21:18 UTC
    moritz,
    thank you for the perldocs links and for your article Character Encodings in Perl
    (which I found even though the suffix .de was missing from the href in your post).

    I also read the excellent Unicode backgrounder on Joel on Software site.

    But don't roll any encoding handling code by hand, it's all been done before and properly tested.

    Very right, except that some of it might have been properly tested on a Mac or on Linux, and not at all on Windows. Caveat emptor.

    Rudif

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://687471]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (3)
As of 2024-04-25 06:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found