A Comparison of the Size of the Yahoo! and Google Indices

Interesting article at http://vburton.ncsa.uiuc.edu/indexsize.html (note, I'm not affiliated with it in any way). What makes this a CUFP is that they used Perl (well, they say PERL) to do their experiments. By looking at the code, it seems they aren't familiar with LWP, but as long as it works, it can be useful. ;-)

2005-08-16 jdporter moved to News. CUFP is a place for posting code.

Comment on A Comparison of the Size of the Yahoo! and Google Indices

Replies are listed 'Best First'.
Re: A Comparison of the Size of the Yahoo! and Google Indices by jhourcle (Prior) on Aug 16, 2005 at 03:32 UTC
Useful to show that any values can be shown with a bad experiment, it seems. I've heard complaints that they've introduced significant bias by only using English words, but my issue is that they've only counted links, and haven't bothered to actually look at them. If it weren't against Google's Terms of Service to search against them in the manner that was done so, (I would assume with Yahoo, as well, but I haven't looked), I'd be interested to know if for any of the searches they did, if Yahoo contained any pages that Google didn't list Again, that might just show different search algorithms, but because Google tends to return as much as it can, it might be an indicator of Yahoo is larger or not. I'll just wait to see if this thing passes vetting in a scientific journal ... I'm guessing not.	[reply]

Replies are listed 'Best First'.

Re: A Comparison of the Size of the Yahoo! and Google Indices
by jhourcle (Prior) on Aug 16, 2005 at 03:32 UTC

Useful to show that any values can be shown with a bad experiment, it seems.

I've heard complaints that they've introduced significant bias by only using English words, but my issue is that they've only counted links, and haven't bothered to actually look at them.

If it weren't against Google's Terms of Service to search against them in the manner that was done so, (I would assume with Yahoo, as well, but I haven't looked), I'd be interested to know if for any of the searches they did, if Yahoo contained any pages that Google didn't list

Again, that might just show different search algorithms, but because Google tends to return as much as it can, it might be an indicator of Yahoo is larger or not.

I'll just wait to see if this thing passes vetting in a scientific journal ... I'm guessing not.

[reply]