in reply to Anyone know why I can't scrape this page?
Regarding the first update: Thought you meant a 500 Internal Server error at first (very confusing :P). From the Mechanize FAQ:
My Mech program gets these 500 errors. A 500 error from the web server says that the program on the server side died. Probably the web server program was expecting certain inputs that you didn't supply, and instead of handling it nicely, the program died. Whatever the cause of the 500 error, if it works in the browser, but not in your Mech program, you're not acting like the browser.
It takes a helluva long time to connect to the site using a normal browser, so I do not think the problem is your script. Either that or the javascript on the page is severely interfering with Mech. If it is the latter case, I suggest WWW:Selenium (I heard it knows how to interpret javascript. Take the suggestion with a grain of sugar.)
Regarding the second update: Post thine code.
I'm so adjective, I verb nouns!
chomp; # nom nom nom
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: Anyone know why I can't scrape this page?
by lv211 (Beadle) on Sep 07, 2008 at 17:48 UTC | |
by Lawliet (Curate) on Sep 07, 2008 at 18:17 UTC | |
by lv211 (Beadle) on Sep 07, 2008 at 18:32 UTC | |
by Lawliet (Curate) on Sep 07, 2008 at 18:54 UTC |