in reply to Re: (kudra: mailto is not opt-in) Re: Re: Perl Programs that can retrieve email addresses from web pages
in thread Reaped: Perl Programs that can retrieve email addresses from web pages

No, robots.txt only specifies pages on a server, and even theough it's a standard, some broken peices of cruft programming don't care for it. Wget cares, though. ;)

--
$Stalag99{"URL"}="http://stalag99.keenspace.com";

  • Comment on Re: Re: (kudra: mailto is not opt-in) Re: Re: Perl Programs that can retrieve email addresses from web pages