in reply to Stateful Browsing and link extraction with AnyEvent::HTTP
in thread Async DNS with LWP
I think it shouldn't be too hard to push the results into a WWW::Mechanize object when they are available. WWW::Mechanize will then do the cookie extraction etc. and if you're using raw LWP, you're extracting the cookies yourself anyway. You then need to override/capture the request that WWW::Mechanize (or LWP) generates when you ->get or follow a link. This request is then again handed off to AnyEvent::HTTP.
I'm not sure that it makes much sense to rewrite WWW::Mechanize to be based on AnyEvent::HTTP, because you will need asynchronicity all over the place anyway.
You could look into spawning threads or simply spawning external processes to handle your requests, but if you're already looking into asynchronous resolvers, you're either prematurely optimizing the wrong end of the task or the overhead from launching threads or processes will eat into your time/latency budget.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^2: Stateful Browsing and link extraction with AnyEvent::HTTP
by jc (Acolyte) on Oct 05, 2010 at 09:28 UTC | |
by Corion (Patriarch) on Oct 05, 2010 at 09:35 UTC | |
by jc (Acolyte) on Oct 05, 2010 at 11:30 UTC | |
by Corion (Patriarch) on Oct 05, 2010 at 11:57 UTC | |
by jc (Acolyte) on Oct 05, 2010 at 12:51 UTC | |
by jc (Acolyte) on Oct 05, 2010 at 14:05 UTC | |
|