PoorLuzer has asked for the wisdom of the Perl Monks concerning the following question:

I have been handling some email related process recently, mainly with processing information from the raw mail headers.

What I want to understand are: 1. how to map strings like these: 

=?iso-8859-1?Q?Communiqu=E9?=
=?iso-8859-1?Q?Telef=F3nica?=
=?ISO-8859-1?Q?Montre=E1l?=
=?iso-8859-1?Q?Minist=E8re?=

[download]

to these: 

Communique
Telefonica
Montreal
Ministere

[download]

I get the same, unhelpful (does not help me get the desired mappings above), results regardless of whether I use: 

    use MIME::Words qw(:all);

    my $rawSub = $head->get( 'subject' );
    
    my $mailSubject = decode_mimewords( $rawSub );

[download]
or I use: 

    use MIME::WordDecoder;

    my $wd = supported MIME::WordDecoder "US-ASCII";

    ### Decode a MIME string (e.g., into Latin1) via the default decod
+er:
    my $str = $wd->decode( $rawSub );
    
    print $str, "\n";

[download]

For some reason that I am unable to understand, the following fails completely (the output shows me the unknown character designator '_' for any character!): 

    use MIME::WordDecoder;

    my $wd = new MIME::WordDecoder::US_ASCII;
    $wd->unknown( '_' ); # What to translate unknown characters to
    $wd->collapse( 1 ); # Collapse runs of unknown characters to a sin
+gle unknown
    
    print $wd->decode( $rawSub ), "\n";

[download]

2. How do I make Date::Manip::UnixDate completely ignore the timezone part of the date string?

Here is some code: 

    use Date::Manip;
    Date_Init( "ConvTZ=IGNORE", "TZ=GMT" ); # We don't want time conve
+rsions happening. GMailBackup ignores timezone conformance and keeps 
+timestamps as they were
    
    # If using Date::Transform, you can use '%g' that is short for %a,
+ %d %b %Y %H:%M:%S %z - Fri, 28 Apr 1995 17:23:15 EDT; For eg : Sat, 
+9 Feb 2008 17:04:08 -0330
    $mailDate = UnixDate( $mailDate , '%Y_%m_%Q-%H%M%S' ); # 2008_02_2
+0080209-170408

[download]

For this example, consider the date string:

Sat, 9 Feb 2008 17:04:08 -0380

The timezone in the string is clearly bogus (minutes value > 60); yet this happens in "real life".

As evident from the code, I don't need the timezone information anyways. So how do I "lop it off" (is it recommended?) reliably?

I would rather have someone tell me a magic flag that I am unaware of that makes the UnixDate ignore the timezone as "chopping it off" promises to complicate life even more (time zones can be in different formats. Eg : EET, -330, +710, etc)

Replies are listed 'Best First'.
Re: Transcoding MIME Strings
by zwon (Abbot) on Oct 09, 2009 at 18:24 UTC

    The following code produces desired output: 

    use strict;
use warnings;
use 5.010;
use MIME::WordDecoder;
use open ':encoding(utf8)';
use open ':std';

my $wd = supported MIME::WordDecoder "iso-8859-1";
for my $enc (
    '=?iso-8859-1?Q?Communiqu=E9?=', '=?iso-8859-1?Q?Telef=F3nica?=',
    '=?ISO-8859-1?Q?Montre=E1l?=',   '=?iso-8859-1?Q?Minist=E8re?='
  )
{
    say $wd->decode($enc);
}

__END__
Communiqué
Telefónica
Montreál
Ministère

    [download]

    Update: and MIME::Words gives me the same output: 

    use strict;
use warnings;
use 5.010;
use open ':utf8';
use open ':std';

use MIME::Words qw(:all);

my @encoded = (
    '=?iso-8859-1?Q?Communiqu=E9?=', '=?iso-8859-1?Q?Telef=F3nica?=',
    '=?ISO-8859-1?Q?Montre=E1l?=',   '=?iso-8859-1?Q?Minist=E8re?=',
);
for (@encoded) {
    say scalar decode_mimewords($_);
}

    [download]
[reply]
[d/l]
[select]
      I wanted : 

      Communique
Telefonica
Montreal
Ministere

      [download]

      instead of : 

      Communiqué
Telefónica
Montreál
Ministère

      [download]
      That is the problem I am trying to solve - I want the output to be in the "ASCII" character set.
[reply]
[d/l]
[select]

        You could transliterate into ASCII in a second step, e.g. using Text::Unidecode: 

        #!/usr/bin/perl

use strict;
use warnings;
use MIME::Words qw(:all);
use Text::Unidecode;

my @encoded = (
    '=?iso-8859-1?Q?Communiqu=E9?=', '=?iso-8859-1?Q?Telef=F3nica?=',
    '=?ISO-8859-1?Q?Montre=E1l?=',   '=?iso-8859-1?Q?Minist=E8re?=',
);
for (@encoded) {
    my $s = decode_mimewords($_);
    print unidecode($s), "\n";
}

__END__
Communique
Telefonica
Montreal
Ministere

        [download]
[reply]
[d/l]
Re: Transcoding MIME Strings
by PoorLuzer (Beadle) on Oct 10, 2009 at 02:12 UTC
    any insights into the second question of the timezone issue?
[reply]

Back to Seekers of Perl Wisdom