Perl_Love has asked for the wisdom of the Perl Monks concerning the following question:

hi all! LWP is work:
my $url='http://www.xmrc.com.cn/net/enterprise/login.aspx'; my $ua=LWP::UserAgent->new(agent=>'Mozilla/5.0 (Windows NT 10.0; rv:46 +.0) Gecko/20100101 Firefox/46.0',timeout=>10); my $cookie_jar=HTTP::Cookies->new(); $ua->cookie_jar($cookie_jar); $ua->request(POST $url, [ '__VIEWSTATE'=>'/wEPDwUKLTQwMDQ2MTU5OGQYAQUeX19Db250cm9sc1JlcXVpcmVQb3 +N0QmFja0tleV9fFgEFF2N0bDAwJEJvZHkkY3RsMDAkZ2V0cHdk', 'ctl00$Body$ctl00$UsernameTextBox'=>'UserName', 'ctl00$Body$ctl00$PasswordTextBox'=>'PassWord', 'ctl00$Body$ctl00$LoginButton'=>'登录' ] ); my $newreq=$ua->request(GET 'http://www.xmrc.com.cn/net/enterprise/Res +ume.aspx?TalentID=95438746')->as_string; print $newreq;
but AnyEvent::HTTP is not work, can you help me? thx!
use Coro; use AnyEvent::HTTP; my $url='http://www.xmrc.com.cn/net/enterprise/login.aspx'; my $cookie_jar={}; my %headers={ 'Host'=>'www.xmrc.com.cn', 'User-Agent'=>'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Ge +cko/20100101 Firefox/44.0', 'Accept'=>'text/html,application/xhtml+xml,application/xml;q=0 +.9,*/*;q=0.8', 'Accept-Language'=>'zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3', 'Accept-Encoding'=>'gzip, deflate', 'Referer'=>'http://www.xmrc.com.cn/net/enterprise/login.aspx', 'Content-Type'=>'application/x-www-form-urlencoded', #'keepalive'=>'1', }; http_post( "$url", '__VIEWSTATE'=>'/wEPDwUKLTQwMDQ2MTU5OGQYAQUeX19Db250cm9sc1JlcX +VpcmVQb3N0QmFja0tleV9fFgEFF2N0bDAwJEJvZHkkY3RsMDAkZ2V0cHdk', 'ctl00$Body$ctl00$UsernameTextBox'=>'UserName', 'ctl00$Body$ctl00$PasswordTextBox'=>'PassWord', 'ctl00$Body$ctl00$LoginButton'=>'登录', cookie_jar=>$cookie_jar, headers=>%headers, ,timeout=>10,cb=>Coro::rouse_cb ); my @rouse=Coro::rouse_wait; print $rouse[0];
Hello ,afoken , Corion , hippo : AnyEvent::HTTP cannot work ,the reason is that http_post login HTML information only,but could not simulate the login website same as LWP example . I know the pass parameters method of LWP POST ,such as UserName ,PassWord , LoginButton . But I do not know AnyEvent::HTTP is how to transfer the information to the website. I created headers from HttpFox . Could you help to make a simple AnyEvent: : HTTP transfer example for reference ? Even though I am a novice,I like Perl very much ! Thanks a lot ~ hi , ikegami : Thank you for guidance ! you give the example is http_get, I can ues http_get to obtain the Websit. And I think if I want to log on to a website, this situation should be submitted it with http_post. For example of LWP
$ua->request(POST $url, [ '__VIEWSTATE'=>'/wEPDwUKLTQwMDQ2MTU5OGQYAQUeX19Db250cm9sc1JlcXVpcmVQb3 +N0QmFja0tleV9fFgEFF2N0bDAwJEJvZHkkY3RsMDAkZ2V0cHdk', 'ctl00$Body$ctl00$UsernameTextBox'=>'UserName', 'ctl00$Body$ctl00$PasswordTextBox'=>'PassWord', 'ctl00$Body$ctl00$LoginButton'=>'登录' ] );
I do not know how the parameters between the http_post from AnyEvent::HTTP , Kindly ask for help ,thanks a lot ~

Replies are listed 'Best First'.
Re: Help, AnyEvent::HTTP no work!
by Corion (Patriarch) on May 09, 2016 at 07:29 UTC

    The easy approach is to read the documentation of AnyEvent::HTTP again. It documents your error situation and the remedy in detail because it's a very common error. Also, mixing Coro and AnyEvent makes very little sense in your case. Your code is IO/network-bound and won't benefit from the additional paralelization possibility that Coro opens up for you.

    Also note that writing a password cracker is not really a great use of your and our time, and might even be illegal depending on your jurisdiction.

      Mixing Coro and AnyEvent makes very little sense in your case.

      I have no idea why you say that.

      Making a fixed number of HTTP requests in parallel:

      use AE qw( ); use AnyEvent::HTTP qw( http_get ); use Coro qw( async ); use Coro::Channel qw( ); sub make_request { my ($url, $cb) = @_; http_get($url, ..., sub { ... $cb->(); }, ); } my $num_workers = 10; my $q = Coro::Channel->new(); my @threads; for (1..$num_workers) { push @threads, async { while (my $url = $q->get()) { my $cv = AE::cv(); make_request($url, $cv); $cv->recv(); } } } while (my $url = get_next_url()) { $q->put($url); } $q->shutdown(); $_->join() for @threads;

      Sure, you can do it without Coro too.

      use AE qw( ); use AnyEvent::HTTP qw( http_get ); sub make_request { my ($url, $cb) = @_; http_get($url, ..., sub { ... $cb->(); }, ); } my $max_requests = 10; my $active_requests = 0; my $cv = AE::cv(); while (my $url = get_next_url()) { while ($active_requests == $max_requests) { $cv->recv(); $cv = AE::cv(); } ++$active_requests; make_request($url, sub { --$active_requests; $cv->send(); }); } while ($active_requests) { $cv->recv(); $cv = AE::cv(); }

      Whatever floats your boat.

Re: Help, AnyEvent::HTTP no work!
by afoken (Chancellor) on May 09, 2016 at 06:39 UTC
    not work

    Ah, the ever so detailed error descriptions. The only one who is NOT working here is you. Show some effort by at least posting the error messages, preferably also add returned output and expected output. Please edit your posting now, and mark the update.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re: Help, AnyEvent::HTTP no work!
by hippo (Archbishop) on May 09, 2016 at 08:27 UTC

    Compare:

    my $ua=LWP::UserAgent->new(agent=>'Mozilla/5.0 (Windows NT 10.0; rv:46.0) Gecko/20100101 Firefox/46.0',timeout=>10);
    with
    'User-Agent'=>'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0',
    and explain why you have created such different headers?


    "It doesn't work" is as useful to a developer as "I'm ill" is to a medic. Be specific.

      Hello , afoken , Corion , hippo : AnyEvent::HTTP cannot work ,the reason is that http_post login HTML information only,but could not simulate the login website same as LWP example . I know the pass parameters method of LWP POST ,such as UserName ,PassWord , LoginButton . But I do not know AnyEvent::HTTP is how to transfer the information to the website. I created headers from HttpFox . Could you help to make a simple AnyEvent::HTTP transfer example for reference ? Even though I am a novice,I like Perl very much ! Thanks a lot ~
Re: Help, AnyEvent::HTTP no work!
by ikegami (Patriarch) on May 09, 2016 at 16:11 UTC

    I'm guessing you want to make web requests in parallel.

    If you want to continue using Coro and/or AnyEvent, you'll find using LWP::Protocol::AnyEvent::http simpler.

    use Coro qw( async ); use Coro::Channel qw( ); use LWP::Protocol::AnyEvent::http qw( ); use LWP::UserAgent qw( ); my $num_workers = 10; my $q = Coro::Channel->new(); my @threads; for (1..$num_workers) { push @threads, async { my $ua = LWP::UserAgent->new(); $ua->protocols_allowed([qw( http https )]); while (my $url = $q->get()) { my $response = $ua->get($url); ... } } } while (my $url = get_next_url()) { $q->put($url); } $q->shutdown(); $_->join() for @threads;

    However, I'd personally use Net::Curl::Multi (on unix systems).