Even if this is not for spam, hitting a web server with 2000 requests every minute (that is over 33 a second) will put a substantial load on the server unless sized to handle that traffic. Perhaps working with the site owner to find a mutually agreeable mechanism to query that quantity of data would be in order.
| [reply] |
If you're going to hit a webserver 2000 times a minute, each minute expect to get banned really quickly. If this works for you, it won't work for very long.
| [reply] |
Ideally, the way you should do this is start a relationship with the web site's API, which, when asked nicely, would tell you when a particular user was most recently changed (assuming you're looking for updates to that user).
Once you had that information, you'd be able to get details on just the users that were changed, saving you and the web server lots of wasted cycles.
Alex / talexb / Toronto
"Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds
| [reply] |
so how could I do that
Maybe SMOP
what is the possibility of its speed be
Maybe 42
all I'm new and please keep calm with me
No thanks spammer, go spam somewhere else
| [reply] |