in reply to Speeding up HTML parsing

Not an answer, but keep in mind that what you're doing is technically in violation of Google's Terms of Service.

Replies are listed 'Best First'.
Re: Re: Speeding up HTML parsing
by sulfericacid (Deacon) on Apr 21, 2004 at 03:13 UTC
    What I am doing really isn't against their Terms of Service in the least, I'm just not doing it the right way yet. It will not let me parse them until I setup the DPI for it, but that's another story. I'm more concerned on how to increase the speed so I can begin adding the other ~8+ engines to the list.


    "Age is nothing more than an inaccurate number bestowed upon us at birth as just another means for others to judge and classify us"

    sulfericacid
[reply]
      No Automated Querying
      You may not send automated queries of any sort to Google's system without express permission in advance from Google.

      You are sending automated queries to Google, by the letter of their TOS that's forbidden. You may get away with it, you may get your IP (or worse your ISP's netblock) blocked by Google. Simply pointing out that Google technically forbids any non-browser access not through their official API. Go right ahead and live on the edge (as long as you're not on my netblock :).

[reply]

In Section Seekers of Perl Wisdom