I believe I understand your situation. You have an email filter which gets run by qmail for every incoming email. You believe that you cannot control when the emails come in or when qmail fires off your filter.
You want your filter to hold off running if there are a lot of other copies of itself already running and processing other emails.
I agree that the most elegant solution would be to file the emails in a separate folder and run a batch process regularly which processes the entire folder. However, this delays when you get your email delivered to your real mailbox as it depends on a polling mechanism.
Here is my module which uses LockFile::Simple to only allow one copy of a program to run at a time. This is similar to merlyn's highlander column which I just learned about. However, since I use this code regularly, mine is a bit more reusable. We obviously both watched the same movie, though.
package My::ThereCanBeOnlyOne; use strict; use LockFile::Simple; my $Lock; sub import { my $self = shift; my $name = shift || 'therecanbeonlyone'; my %args = @_; # Using /tmp allows an internal denial of service attack. my $lockfile = "/tmp/$name.pid"; my $locker = LockFile::Simple->make(-autoclean => 0, %args); $Lock = $locker->lock($lockfile, '%f') and return 1; open(LOCKFILE, "< $lockfile") or die __PACKAGE__."Unable to open $lockfile: $!"; my $other_pid = <LOCKFILE>; close(LOCKFILE); chomp($other_pid); die __PACKAGE__.": $name: $other_pid still running. $$ exiting\n"; } END { $Lock->release(); } 1;
At the top of your filter program you would write:
Replace 'myprogram' with the name of your program or resource you want to lock on. It is arbitrary.use My::ThereCanBeOnlyOne 'myprogram';
This would use all of the LockFile::Simple defaults as far as retries, timeouts, expiration, etc. You can override any LockFile::Simple parameters, by simply including them at the end of the use statement like so:
use My::ThereCanBeOnlyOne 'myprogram', -hold => 0, -stale => 1, -max => 1;
Now comes the interesting idea. You want to limit the number of email filters to N. If you're willing to live with N as the absolute maximum and having it be more and more likely that a copy of the program will wait for other copies to finish as the number of running copies approaches N (say 6), then you could use:
This picks a random lock file slot from 0 to 5. If another running program already has that slot, we keep waiting until it finishes. You can see that the probability of a program waiting will increase the more programs are already running, but this will for sure limit you to 6 (or N) simultaneous copies.use My::ThereCanBeOnlyOne 'myprogram'.int(rand(6));
This might not be reasonable for some applications, but for an email filter, I thought it might be appropriate.
-- Eric Hammond
In reply to Re: Maximum # of concurrent runs
by esh
in thread Maximum # of concurrent runs
by warthurton
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |