in reply to State-of-the-art in Harvester Blocking

What are some of the better techniques for doing this, with minimal annoyance to real customers, and without making the site inaccessible to people with text-only browsers or visual impairments?

Try some social pressure. Once you're reasonably certain that you've detected a spider, arrange to direct it to a page that lays out the terms of use for the site. Include some sort of spider-proof way to get unbanned (e.g., manually copy/paste this URL). The spider will keep getting the same page, until the spider's author/runner notices. At that point, they might think "oops, busted" or they might work around you. If the former, great. If the latter, you'll have to give some thought about whether you want to get into an arms race.

  • Comment on Re: State-of-the-art in Harvester Blocking

In Section Seekers of Perl Wisdom