This little script parses the results of procmail's mail sorting, and reports how many messages went into each folder. I use it to test the effectiveness of my spam filter (crm114). Run it from cron daily, and it'll report on the distribution of yesterday's mail (by date). Oh - and it helped me see that crm114 had 0.2% false positives, and 0% false negative yesterday.

It works, but I'm not 100% happy with the use of write there - seems a little convuluted for me.

As always, comments and suggestions are very welcome!

Sample (compressed) output:

Mail report for Mar 15
    Folder        |     Messages    |     Bytes
------------------+-----------------+----------------
        * TOTAL * | 695   (100.00%) | 37053 (100.00%)
          In/spam | 226   (32.52% ) | 16071 (43.37% )
        In/mythtv | 140   (20.14% ) | 67511 (18.22% )
            INBOX | 100   (14.39% ) | 63448 (17.12% )
        In/lugnet | 32    ( 4.60% ) | 13740 ( 3.71% )
          In/Root | 21    ( 3.02% ) | 20213 ( 5.46% )
       In/seclist | 17    ( 2.45% ) | 56512 ( 1.53% )
...

Code follows:

#!/usr/bin/perl -w
#
# Version 1.0
# Reports on mail distribution into folders by parsing procmail.log
# Author: Dan Boger (perl@peeron.com)
# Date: 03/16/04
# Usage: Just update the location of the procmail log below, and run d
+aily via cron.
#        Will report on all the entries from *yesterday* (so schedule 
+to run after midnight

use strict;
use Date::Manip;

# where is the procmail log located?
my $procmail = "/home/dan/Mail/procmail.log";

# How should the results be sorted?  either Msgs or Bytes
my $sortmethod = "Msgs";

# what format should the report use?  read perlform for details
my $entry;
format STDOUT_TOP =
                 Folder                  |     Messages    |     Bytes
-----------------------------------------+-----------------+----------
+------
.
format STDOUT =
  @>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | @<<<< (@<<<<<<) | @<<<< (@<
+<<<<<)
  $entry->{Title}, $entry->{Msgs}, $entry->{MPercent}, $entry->{Bytes}
+, $entry->{BPercent}
.

# no user cofiguration below this line (currently)

# figure out what date we're looking for
my $date = &ParseDate("Yesterday");
my $year = &UnixDate($date, '%Y');
$date = &UnixDate($date, "%b %e");

open(LOG, $procmail) or die "Can't read $procmail: $!";

# scan for beginning of yesterday's mail
while (<LOG>) { # search up to the destination date
  last if /^From \S+ \w\w\w $date \d\d:\d\d:\d\d $year/o;
}

# sum up mail traffic
my %mbox;
my ($folder, $bytes);
while (<LOG>) {
  # check if we've past the target date
  if (/^From \S+ \w\w\w (... [ \d]\d)/ and $1 ne $date) {
    if (defined $folder) {
      # we already counted this email as if it was from
      # yesterday - remove it
      $mbox{$folder}->{Bytes} -= $bytes;
      $mbox{$folder}->{Msgs}  --;
      $mbox{"* TOTAL *"}->{Bytes} -= $bytes;
      $mbox{"* TOTAL *"}->{Msgs}  --;
    }
    last;
  }

  next unless /^\s*Folder: (\S+)\s+(\d+)/;
  ($folder, $bytes) = ($1, $2);
  $mbox{$folder}->{Bytes} += $bytes;
  $mbox{$folder}->{Msgs}  ++;
  $mbox{"* TOTAL *"}->{Bytes} += $bytes;
  $mbox{"* TOTAL *"}->{Msgs}  ++;
}

# done scanning
close LOG;

# print out results
print "Mail report for $date\n\n";

# sorted by either Msgs or Bytes
foreach (sort {$mbox{$b}->{$sortmethod} <=> $mbox{$a}->{$sortmethod}} 
+keys %mbox) {
  $entry = $mbox{$_};
  $entry->{Title} = $_;
  if ($mbox{"* TOTAL *"}->{Msgs} > 0) {
    $entry->{MPercent} = sprintf("%5.2f%%", $entry->{Msgs} / $mbox{"* 
+TOTAL *"}->{Msgs} * 100);
  } else {
    $entry->{MPercent} = "N/A";
  }
  if ($mbox{"* TOTAL *"}->{Bytes} > 0) {
    $entry->{BPercent} = sprintf("%5.2f%%", $entry->{Bytes} / $mbox{"*
+ TOTAL *"}->{Bytes} * 100);
  } else {
    $entry->{BPercent} = "N/A";
  }
  write;
}

[download]

-- zigdon

In reply to (z) Parse procmail.log to report on mail distribution by zigdon

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.