I believe I understand your situation. You have an email filter which gets run by qmail for every incoming email. You believe that you cannot control when the emails come in or when qmail fires off your filter.

You want your filter to hold off running if there are a lot of other copies of itself already running and processing other emails.

I agree that the most elegant solution would be to file the emails in a separate folder and run a batch process regularly which processes the entire folder. However, this delays when you get your email delivered to your real mailbox as it depends on a polling mechanism.

Here is my module which uses LockFile::Simple to only allow one copy of a program to run at a time. This is similar to merlyn's highlander column which I just learned about. However, since I use this code regularly, mine is a bit more reusable. We obviously both watched the same movie, though.

package My::ThereCanBeOnlyOne; use strict; use LockFile::Simple; my $Lock; sub import { my $self = shift; my $name = shift || 'therecanbeonlyone'; my %args = @_; # Using /tmp allows an internal denial of service attack. my $lockfile = "/tmp/$name.pid"; my $locker = LockFile::Simple->make(-autoclean => 0, %args); $Lock = $locker->lock($lockfile, '%f') and return 1; open(LOCKFILE, "< $lockfile") or die __PACKAGE__."Unable to open $lockfile: $!"; my $other_pid = <LOCKFILE>; close(LOCKFILE); chomp($other_pid); die __PACKAGE__.": $name: $other_pid still running. $$ exiting\n"; } END { $Lock->release(); } 1;

At the top of your filter program you would write:

use My::ThereCanBeOnlyOne 'myprogram';
Replace 'myprogram' with the name of your program or resource you want to lock on. It is arbitrary.

This would use all of the LockFile::Simple defaults as far as retries, timeouts, expiration, etc. You can override any LockFile::Simple parameters, by simply including them at the end of the use statement like so:

use My::ThereCanBeOnlyOne 'myprogram', -hold => 0, -stale => 1, -max => 1;

Now comes the interesting idea. You want to limit the number of email filters to N. If you're willing to live with N as the absolute maximum and having it be more and more likely that a copy of the program will wait for other copies to finish as the number of running copies approaches N (say 6), then you could use:

use My::ThereCanBeOnlyOne 'myprogram'.int(rand(6));
This picks a random lock file slot from 0 to 5. If another running program already has that slot, we keep waiting until it finishes. You can see that the probability of a program waiting will increase the more programs are already running, but this will for sure limit you to 6 (or N) simultaneous copies.

This might not be reasonable for some applications, but for an email filter, I thought it might be appropriate.

-- Eric Hammond


In reply to Re: Maximum # of concurrent runs by esh
in thread Maximum # of concurrent runs by warthurton

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.