LWP::UserAgent how to download captcha and access web

liuweichuan has asked for the wisdom of the Perl Monks concerning the following question:

1. LWP::UserAgent is as a object for browser 2. open url and download captcha, and captcha info wait for input, like $x = <> 3. requirst post to login. my issue is i can't see the captcha info. my $url="http://caigou.51book.com/caigou/captcha"; $Lwp->post($url); open FILE,">abc.jpg" or die "$!"; my $outcome=get ($url); binmode(FILE); print FILE $outcome; ..... $

Comment on LWP::UserAgent how to download captcha and access web

Replies are listed 'Best First'.
Re: LWP::UserAgent how to download captcha and access web by Corion (Patriarch) on Jul 22, 2016 at 07:29 UTC
In what way do you mean you "can't see the captcha info"? Maybe the captcha information is not directly in the HTML but loaded via Javascript or an iframe element. You will have to look at the page source to find out. As a captcha is usually applied to prevent automation it is also very likely that the captcha server does not respond to naive Perl scripts. Maybe you need to convert your Perl script to behave more like a browser. See for example Wireshark or LWP::Debug how to find out what your browser sends and what LWP sends. Also see WWW::Mechanize to make behaving like a browser easier.	[reply]

Replies are listed 'Best First'.

Re: LWP::UserAgent how to download captcha and access web
by Corion (Patriarch) on Jul 22, 2016 at 07:29 UTC

In what way do you mean you "can't see the captcha info"? Maybe the captcha information is not directly in the HTML but loaded via Javascript or an iframe element. You will have to look at the page source to find out.

As a captcha is usually applied to prevent automation it is also very likely that the captcha server does not respond to naive Perl scripts. Maybe you need to convert your Perl script to behave more like a browser. See for example Wireshark or LWP::Debug how to find out what your browser sends and what LWP sends. Also see WWW::Mechanize to make behaving like a browser easier.

[reply]