ginseng has asked for the wisdom of the Perl Monks concerning the following question:

I inherited a problem when I became webmaster at an established ISP. (More than just one, but I have this one in mind at the moment.) Like many ISP's we use Matt Wright's formmail.pl, making it available to customers. This script takes the contents of a web form and converts it to an email, sent to a recipient mentioned in a type=hidden tag.

Lately, with the increased difficulty of finding open relays, spammers and worse have resorted to locating servers with formmail.pl installed. They structure a GET request that specifies the content of an email, and its recipient. The web server brings in the request and gladly sends it out, provided that the most basic requirements are met.

This has happened to me, or at least to one of the servers for which I'm responsible. As it turns out, I didn't have a spammer relaying through my web server; I had a credit card fraud. The jerk has a worn-out scheme sending emails to AOL subscribers, telling them their credit card number was lost, etc, and please go to the new billing center and re-enter their data. I would love to think that no one is naive enough to fall for this, but if it were me, I'd guess that AOL is a good place to look for people naive enough to fall for this.

Of course, in the last week since this particular jerk started relaying through my web server, 56 others have probed for this hole.

My solution to the problem involved looking at how the hole has been plugged in the newest release, and I'm afraid I'm not impressed. (Go figure.) Basically, Matt has added an array of "allowable recipients"; if the specified recipient isn't in the list, it isn't sent.

This is fine if you have one web site hosted on a server, but when you have hundreds, many with contract designers and frequent changes, it gets just a bit unwieldy trying to keep such a list up-to-date, or trusting that each domain's local copy of the script is correctly configured. (I have it symlinked to prevent the need to maintain a hundred or more copies.)

There was a previous effort to control it based on the referrer, as shown here:

{snip} 

sub check_url {

    # Localize the check_referer flag which determines if user is vali
+d.     #
    local($check_referer) = 0;

    # If a referring URL was specified, for each valid referer, make s
+ure    #
    # that a valid referring URL was passed to FormMail.              
+       #

    if ($ENV{'HTTP_REFERER'}) {
        foreach $referer (@referers) {
            if ($ENV{'HTTP_REFERER'} =~ m|https?://([^/]*)$referer|i) 
+{
                $check_referer = 1;
                last;
            }
        }
    }
    else {
        $check_referer = 1;
    }

    # If the HTTP_REFERER was invalid, send back an error.            
+       #
    if ($check_referer != 1) { &error('bad_referer') }
}

[download]
(code from http://worldwidemart.com/scripts/cgi-bin/download.cgi?s=formmail&c=txt&f=FormMail.pl. Copyright 1995-2001 Matt Wright)

What I don't get is that the referer is ignored if not provided. Also, it still requires @referers to contain a list of all domains using the script, another unwieldy maintenance job.

My solution was a kludgy hack: 

    local($check_referer) = 0;
    my $refhost = $ENV{'HTTP_REFERER'};
    $refhost =~ s/^.*\/\///g;
    $refhost =~ s/\/.*$//g;
    my $refip = `dnsip $refhost`;
    chomp $refip;
    $refip =~ s/\s+$//g;
    print STDERR "[" . localtime() . "] [formmail referer check] \$ENV
+{'REMOTE_ADDR'} = $ENV{'REMOTE_ADDR'} [$ENV{'REMOTE_HOST'}] \$ENV{'HT
+TP_REFERER'} = $ENV{'HTTP_REFERER'} \$refhost = $refhost \$refip = $r
+efip.\n";
    foreach $referer (@referers) {
      ($refip eq $referer) and $check_referer = 1;
    }

    # If the HTTP_REFERER was invalid, send back an error.            
+       #
    if ($check_referer != 1) { &error('bad_referer') }

[download]

This does a dns lookup (I use Daniel Bernstein's programs whenever possible) of the referrer and compares that to the list of acceptable IP addresses in @referer. (Side note: I've never understood why the HTTP standard misspelled referrer. Anyone know?) Notice that it also fails if a referrer is not provided. (When is the referrer not available? I'd rather support the fewest calls possible; this strikes me as the method.) Also, it logs all formmail requests in the error log, with the client's address and the referring domain.

However, this method has a fatal flaw: it is trivial to forge the referrer. Matt Wright's solution does solve this, by restricting the recipient list. I thought hard about doing the same thing; but (a) I don't want the headache of taking care of this script forever and ever with customers, and (b) many of the customers have the form sent "off-site", to a domain hosted elsewhere, and need control over the recipient list.

So my question, wise ones, is how to fix this? (Whoa, broad question. I'll try to narrow things down a bit.)

First, are there secure alternatives to this script, preferably that don't involve telling my customers they need to rewrite all their forms?

Second, what would you do? Throw it away and start from scratch? (I'm considering it...) Patch and hack, keeping a step ahead of the spammers and frauds? Limit recipients and deal with the administrative tasks?

Third, if you were saddled with this legacy, maybe even considering a complete (but mostly compatible) rewrite, how would you prevent creating a port 80 relay?

Replies are listed 'Best First'.
Re: Port 80 Mail Relay (Spammers Welcome?)
by mirod (Canon) on Sep 13, 2001 at 14:47 UTC

    I am afraid I don't have a solution, but I wuld just like to point out that if someone comes up with a patched or a rewrite of that script it would be nice to contact davorg so he can include it in his Not Matt's Scripts archive, so it's available to as many people as possible. Of course sending the patch/rewrite to Matt would also be nice, although I have no idea how responsive he is to that kind of help.

[reply]
[reply]
Re: Port 80 Mail Relay (Spammers Welcome?)
by blakem (Monsignor) on Sep 13, 2001 at 14:58 UTC
    uggh, been there done that... The issue is fundamental to the way the script works, so I don't know if there is really a good way to patch this one up *and* retain backwards compatability.

    However, when I inherited a similiar situation, I did manage to stop the script kiddies. Luckilly the entire site was dynamic (SSI), so we put an extra field in all the FORM pages. This value of this hidden field changed every hour and was valid for two hours. If a request came in that didn't have a valid value, we didn't send the mail.

    We managed to get rid of the real-world problem, even though theoretically it wasn't any more secure. It does require slight modification to the submission page, so it might not be what you need, but it did work fine for us.

    -Blake

[reply]
      whoa, I like this. It can be extended.

      I don't like SSI (and fortunately only one existing customer was using it, so I only support it on one site :) but I could require ... what, an include of some kind? ... on the form page that generates a key. if the key is not there, the mail just doesn't go.

      I can quickly and easily migrate all existing forms using a perl script, and require all new forms to use the key generator. Instructions can be integrated with the existing "how to".

      I will admit first though, btrott has written a replacement program called stamp that I'm looking at right now. It looks *really* good, though it's not backward compatible. (Thanks to crazyinsomniac who pointed it out to me.)

[reply]
        I wasn't advocating SSI, just pointing out that the submission forms weren't static html. You've got the basic idea though... a carefully crafted perl -pi -e 's///' on the existing forms, along with an hourly cron job to update the "secret" key was all it took. Not a great solution, but perhaps good-enough for the short term.

        -Blake

[reply]
[d/l]
Re: Port 80 Mail Relay (Spammers Welcome?)
by projekt21 (Friar) on Sep 13, 2001 at 14:53 UTC

    How do you get the recipient address?

    a) via the form (this opens spam)
    b) via the script (hardcoded/database/file)

    I used to hardcode the recipient address into formmail.pl for every customer as a quick hack to avoid abuse. It works fine.

    Another approach could be entering all possible recipients into a file or database and let formmail check that. It will reduce administration effort.

    Sorry, no other idea at the moment.

    p.s.: I noticed some spammers check our formmail for open relaying by sending a mail to some obscure hotmail (or other) account. I got that mail as I overwrote the recipient, but those spammers never tried again.

    alex pleiner <alex@zeitform.de>
    zeitform Internet Dienste

[reply]
Re: Port 80 Mail Relay (Spammers Welcome?)
by jepri (Parson) on Sep 13, 2001 at 14:54 UTC
    Since we are perl coders here, how about creating a site where the clients can log in and add/remove their email addresses to the recipients list. That way they get what they want and you don't waste your own time.

    I faced the same problem and ended up using the recipients list. Other options are to dump the form entries to disk and then ship the file to the client in a batch

    I never really understood the purpose of formmail anyway. That's what hotmail is for.

    When I want someone to send me an email, I hand out my email address, I don't direct them to a form mail program.

    I realise as the sysadmin you don't always get a choice for what clients can do.

    ____________________
    Jeremy
    I didn't believe in evil until I dated it.

[reply]
(ichimunki) Re: Port 80 Mail Relay (Spammers Welcome?)
by ichimunki (Priest) on Sep 13, 2001 at 16:27 UTC
    You've gotten some good suggestions so far. I don't think I like the SSI method very much because it is more complicated than it needs to be (perhaps opening more security issues than it solves). Simplicity is your friend when it comes to security.

    With that in mind, I like the suggestion to create a place for your users to get on the recipients list. As long as this area is secure, this is an improvement over simply restricting email TO the domains you host (which is the easiest solution, and therefore probably the best one). If you have clients who "must" have email to off-site domains or whatever, they can introduce forwarding rules via procmail. Or they be shown how to modify formmail.pl so that this vulnerability does not affect it, but they get email where they need it to go.
[reply]
Re: Port 80 Mail Relay (Spammers Welcome?)
by bastard (Hermit) on Sep 14, 2001 at 00:29 UTC
    Unfortunately the webmail I just wrote was under contract so i cant share the code. What I can share is the theory behind what I wrote.

    It doesn't take alot to write an webmailer. (A week or so).

    These are the modules i used. (ones that i hadn't custom written for this script). use Email::Valid;
    use Mail::Mailer;
    use Mail::Sendmail;

    To prevent abuse, (a very bad thing from here) I set it up so that all the form takes other than the to-be-emailed fields is a hidden "code" field. This field describes a directory in the data directory that contans a config file and display template. Also every field to be sent is descibed in the config file (for data validation and required fields), and any that arent, don't get sent, so people can't make their own version of the form to send wierd things.

    The destination email address is hard coded into the config file preventing open use of the script. The one i just wrote supports multiple email address each assigned a name in the config file. The names can be specified in the form, they would get translated by whats in the config file. (multiple names are ok).

    A script could probably be built to manage the config files and templates. The config files should probably also be xml, but I was lazy and didn't want to install the expat stuff.

    With this system, the referrer is not needed (besides, it can be spoofed), and it sidesteps alot of other problems. (and probably creates a few new ones, but I think overall its more secure.)

[reply]
Re: Port 80 Mail Relay (Spammers Welcome?)
by shotgunefx (Parson) on Sep 14, 2001 at 01:00 UTC
    I have any idea that won't make it any more secure but may help. Most spammers are sending the same message over and over in rapid succession. One way would be to modify the script to log the ip and keep track of how many submissions you have received in a given period and if it's more than $x amount in a given time frame, add the IP to a dbm file or such and refuse or forward to an admin account emails from that IP. (I know IP ne 'user')

    Another thing, is that spammers usually send the same message body over and over. Perhaps have a dbm file that uses a MD5 hash of each message for a key and check to see if it has been sent before. If over an allowable amount, forward to an admin account.

    Far from perfect solutions but it may help or at least make it more difficult to use this hole efficiently.

    -Lee

    "To be civilized is to deny one's nature."

[reply]

Back to Seekers of Perl Wisdom