WWW::Mechanize and long links

InfiniteSilence has asked for the wisdom of the Perl Monks concerning the following question:

I want to be able to grab the next few pages from my safari subscription at orielly.com using WWW::Mechanize but I can't seem to get past the first page link:

perl -MWWW::Mechanize -e "$a=WWW::Mechanize->new(
); $a->get('http://safari.oreilly.com'); @links=$a->links();$a->follow
+_link(url=>$links[5]->[0]);"
#result
Unexpected field value http://safari.oreilly.com at (eval 5) line 1
[download]

Anybody know a work-around?

Celebrate Intellectual Diversity

Comment on WWW::Mechanize and long links Download Code

Replies are listed 'Best First'.
Re: WWW::Mechanize and long links by zengargoyle (Deacon) on Nov 02, 2003 at 07:34 UTC
i feel your pain. Safari used to be nice, but they changed the interface and it's often to slow to be usable if you read fast. so much so that i've stopped using it. best advice, use a browser with tabs and when you start every page open the next in a background tab. do that and keep all of the tabs open so you can flip back if you need. next best advice would be to setup squid or some other proxy cache. configure the site with a week or two expiration so you can quickly browse what you've recently seen without suffering the wait.	[reply]
Re: WWW::Mechanize and long links by Enlil (Parson) on Nov 02, 2003 at 05:40 UTC
Wouldn't what you are proposing against the Terms of the agreement where it mentions that you agree: - not to use "Web spiders" or any other automated retrieval mechanisms when using the Service other than what is provided by the Service; -enlil	[reply]
Re: Re: WWW::Mechanize and long links by InfiniteSilence (Curate) on Nov 02, 2003 at 13:44 UTC
Well, your point about automated retrieval may be true but I think the 'spirit of the law' is that the rule is designated to keep people from hitting their servers too hard. Anyway, the reason I want to be able to do this is I really don't have the time to sit and read the books online, so I made some changes to the code in node 300762 (see my scratchpad) to be able to simply read the text. I then record the stuff onto tape and listen to it while driving. It would be nice if I could automatically load the next portion of text, trim the file to a suitable length to fit onto the tape, and have everything ready so that I could pop in a tape, hit record, and eat breakfast. Celebrate Intellectual Diversity	[reply]