in reply to Re: A case where Mechanize works, LWP doesn't
in thread A case where Mechanize works, LWP doesn't

Ok. Interesting results on the Sniffer trace. It appears Mechanize is handling a redirect while LWP is not. Also the encapsulation of the POST on Mechanize is application/x-www-form-urlencoded while LWP just has variable: value in separate lines. So it appears Mechanize is doing much more work under the covers. I'll see how I can manipulate LWP to mimic Mechanize.
  • Comment on Re^2: A case where Mechanize works, LWP doesn't

Replies are listed 'Best First'.
Re^3: A case where Mechanize works, LWP doesn't
by Corion (Patriarch) on Feb 10, 2010 at 16:33 UTC

    Why don't you just use WWW::Mechanize? It's based on LWP, so basically to make LWP::UserAgent behave like WWW::Mechanize means just taking the code of WWW::Mechanize.

[reply]
      Great questions. The reason is two fold.

      1. Limit change. The existing PERL installation doesn't use Mechanize and I would rather not introduce it primarily to keep the politics under control.

      2. POE integration. I have seen much on LWP and POE but nothing on Mechanize and POE. I'd rather go with a more proven route.

      Correct me if I am wrong (with a URL, ISDN or other tangible reference) especially on #2

[reply]

        As WWW::Mechanize ISA LWP::UserAgent, most of the techniques used to make LWP::UserAgent cooperate with POE will work with WWW::Mechanize as well.

[reply]

        As a last resort, you could try LWP::Coro with POE (I'm not saying it works), or fork a separate process to do the downloading. Try that only if you really can't solve the original problem of doing the query without Mechanize, and if you really can't get rid of POE.

[reply]
      Actually found out this isn't the case. Mechanize has blocking calls. POE is more a cooperative scheduler along the lines of what Windows 3.1 was. So POE won't work well with Mechanize like it will with LWP. So I am back to using the sniffer to figure out how to code LWP with enough of Mechanize so the pages will return.
[reply]

In Section Seekers of Perl Wisdom