in reply to LWP POST to a form on a 'secondary web page'

Can LWP only be used for shallow screen scraping of directly addressable web pages, and not for deeper navigation and extraction?
LWP should do the job. The only place I've gotten stuck using LWP was when the web server is using NTLM authentication.
LWP has some support for NTLM, but it seems to not work with newer implementations.

You will need to post some code to get good help though.

non-Perl: Andy Ford

  • Comment on Re: LWP POST to a form on a 'secondary web page'

Replies are listed 'Best First'.
Re^2: LWP POST to a form on a 'secondary web page'
by ady (Deacon) on Dec 25, 2006 at 20:35 UTC
    Tnx Andy,
    I got the NTLM authorization to work;
    I'm trying LWP partly out of curiosity and partly because I expect it to be faster than Win32::IE::Mechanize.
    I've updated the node with the relevant code.
    It runs as is, and does what I want it to do, but it's not as efficient as it could be, because the POST of the field values to the 'second page' doesn't work, and thus the database retrieves all values (ie doesn't filter by the fields in the form on the page)
    Best regards,
    allan
[reply]

In Section Seekers of Perl Wisdom