in reply to NCSA Uses Perl to Compare Google/Yahoo
Please do not write to Google to request permission to "meta-search" Google for a research project, as such requests will not be granted.It also looks like the spider doesn't sleep between requests. Presumably the author made a considered choice that because Google and Yahoo are so robust, it was acceptable to fire off a huge number of requests all at once, in contravention of standard spidering netiquette (Google and Yahoo don't do that to your server). If you ever write a spider, please don't do that.
Perhaps the author was not a native perl coder?
sub main ? No hashes? Lots of subscripted array elements? Could it be... C?
|
---|