First of all, where does your source document come from? Which factors determine its encoding?

Ultimately the best course of action is to ensure that you're using utf8 everywhere (and advertising that to the viewing programs). You can tell the web browser that your content is in utf8 by sending an additional "charset" header attribute like this:

  use CGI;
  my $cgi = CGI->new;

  ...

  print $cgi->header(-type=>'text/html', -charset=>'utf-8');
[download]

Content-Type: text/plain; charset="utf-8"
[download]

I'd like to point out that in recent perls (version 5.8), under most circumstances, strings are encoded internally as utf8. So it makes sense to be consistent about that in the rest of your application.

Thank you to rhesa, graff & fraktalisman for your constructive input. "First of all, where does your source document come from? Which factors determine its encoding?" The content comes from an input box (form) and the cgi program writes this to a text file. What happens after that i dont know. I was able to modify the script to insert a MIME header as you suggested but it did not change anything. I dont know what determines its encoding. I think im going to have to get the programmer involved here and come back with some code and more info. thank you again to everyone.

I think getting your developer in here to discuss the details is a very good idea.

I've found on several occasions that it's necessary to manually upgrade form input to utf8. For some reason, CGI returns raw byte strings, and those might end up being upgraded to utf once more, resulting in lots of squiggly characters. I did this like so:

    use Encode;
    my $email_body = $cgi->param( 'email_body' );
    $email_body    = decode_utf8( $email_body );
[download]

After this, most conversions and display issues are a snap.

If you would prefer your email output to be iso-8859-1, use the Encode module on the text data, or PerlIO control on the output file handle, in order to assure that the text is written with that encoding; assuming the email text is going out via a file handle the easiest thing would probably be:

binmode $fh, ":encoding(iso-8859-1)";  # convert internal utf8 to lati
+n1 on output
[download]

You'll need to set the charset as unicode (you can do this with a meta tag in the head <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">) btw. you can find more info at unicode.org (check out the FAQs).

I've had similar problems recently when upgrading an existing content management system from Latin-1 to UTF-8. After extensive testing we have decided to display all website content in UTF-8 encoding, but send emails in Latin-1, because many freemail websites seem to ignore the charset directives.

If your provider happens to run an older Perl version, the Encode module might not support UTF-8 or it might not be there at all. In that case, have a look at the sub latin1 in the following example: Converting character encodings.