Re^4: Scraping an ASP form I don't have any control over

I did some google searching before I posted here and found several other people trying to do something similar but they were using a completely different module. I've made progress with Mechanize though, so I think I'll stick with it.

Comment on Re^4: Scraping an ASP form I don't have any control over

Replies are listed 'Best First'.
Re^5: Scraping an ASP form I don't have any control over by 1nickt (Canon) on Apr 15, 2017 at 20:26 UTC
Use Firefox / Firebug to see what fields you are really posting. ASP adds many hidden fields. Go to Network > Post , right click and copy as POST args, try with LWP. Or copy as cURL request and try with cURL. If you don't get the results you expect, there's probably. JavaScript involved in posting the form and Mech won't be able to do the job. Then you'll need to use PhantomJS or Selenium. The latter can be controlled by Perl. Hope this helps! The way forward always starts with a minimal test.	[reply]

Replies are listed 'Best First'.

Re^5: Scraping an ASP form I don't have any control over
by 1nickt (Canon) on Apr 15, 2017 at 20:26 UTC

Use Firefox / Firebug to see what fields you are really posting. ASP adds many hidden fields. Go to Network > Post , right click and copy as POST args, try with LWP. Or copy as cURL request and try with cURL. If you don't get the results you expect, there's probably. JavaScript involved in posting the form and Mech won't be able to do the job. Then you'll need to use PhantomJS or Selenium. The latter can be controlled by Perl.

Hope this helps!

The way forward always starts with a minimal test.

[reply]