in reply to Most efficient (or fast) way to split a postfix mail log file?

I guess I just need a point the right direction or a suggestion of a particular function or tutorial so I don’t spend hours trying to do something that isn’t possible to do in the first place.
A big ++ for that alone. I tend to do just that - seek advice only after everything else has failed. Being a complete autodidact, it's sort of tradition. But seeking advice beforehand has - apart from learning being much lighter - a big sociable effect.
My plan is to write a pre-processing script that will split up the main mail log file into several files based on the sender email address.

Why? Did you hit any boundaries with pflogsumm? Is splitting even necessary?

Splitting up a mail log file is tricky, since the sender (envelope) address isn't the reference for log entries, afaik. Each mail gets assigned a message ID, and depending on the mailer, a processing identifier which might change through the delivery chain, depending on setup. If you have e.g. a sandwich setup, postfix delivering to spamassassin and a virus checker, then picking up the message again with a second instance of postfix, the internal IDs will likely change; you would then have to take that into account. Having all that done and producing a set of output files suitable for pflogsumm, you will have either rewritten or duplicated a large part of pflogsumm itself.

Since you didn't mention any constraints of your setup yet (processing speed? memory? disk space?) but only spoke of your plan, more information is needed to give good advice. What has been the outcome of using pflogsumm "as-is" so far?

--shmem

_($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                              /\_¯/(q    /
----------------------------  \__(m.====·.(_("always off the crowd"))."·
");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
  • Comment on Re: Most efficient (or fast) way to split a postfix mail log file?

Replies are listed 'Best First'.
Re^2: Most efficient (or fast) way to split a postfix mail log file?
by oltranzista (Initiate) on Mar 11, 2008 at 03:23 UTC
    Thanks for the reply shmem. I looked over the docs/man page for pflogsumm and didn't see any feature to split by "from" address or message id so I thought I would just feed it pre-split files.

    The reason I need to split these is we provide email list server services to multiple clients and on Lyris they are used to getting smtp reports for their individual lists. So I need to either write a program to parse this info out from scratch or find a third party program to bolt onto Sympa to provide this data.

    I suppose I could patch pflogsumm since it is also written in perl but I was wanting to do get this done in a less obtrusive way.

    We have a dedicated server running Debian Linux on a T1 line. The hard disk space isn't something my company is worried about me using with this new email server too so that shouldn't be an issue either.

    Thanks!
    Jason