in reply to My Mechanize script cannot work ?

WWW::Mechanize does not execute any JavaScript. You will have to look at the returned JavaScript and then look what the next URL to fetch will be, and fetch that URL then from the script :

use strict;
use WWW::Mechanize;
my $mech = WWW::Mechanize->new();
$mech->quiet(1);
$mech->get("http://www.thesite.com/sample.asp");
my $content = $mech->content();
$mech->content =~ qr(\.location = "(http://.*?)")
  or die "Couldn't find anything matching '.location = \"http://...\" 
+in $content";
my $new_location = $1;
print $new_location;
$mech->get($new_location);
print $content;

[download]

If you want to see the "real" behaviour in your browser as well, you have to switch off JavaScript in your browser.

Perl does not yet have a real JavaScript interpreter with the browser DOM, but there is JavaScript.pm and Scripting::JavaScript, both by Claes Jacobson.

perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ;    # The  
$d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider
($c = $d->accept())->get_request(); $c->send_response( new   #in the
HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' #  web

[download]

Replies are listed 'Best First'.
Re: Re: My Mechanize script cannot work ?
by donno20 (Sexton) on Aug 07, 2003 at 10:43 UTC
    Thank you. I manage to get to the right login page.
    However I get this return whil submit the form.
    Unexpected field value http://www.thesite.com/members/login.asp at (eval 5) line 1
    What is going on ? I dump the filled in form and return:

    uid=<UNDEF> (hidden)
    tuid=username
    password=pass123 (password)
    cb= (hidden)
    vcid=dummy (hidden)
    submit1=Login now (submit)

    Thanks again. ^_^
[reply]

      Without seeing your code, it is very hard to guess what happens there. I guess from your description that the script prints the message Unexpected field value http://www.thesite.com/members/login.asp at (eval 5) line 1 while running. I don't know what might cause this message to be printed, so I guess that it has something to do with how you are passing form values to the $mech object. 

      perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ;    # The  
$d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider
($c = $d->accept())->get_request(); $c->send_response( new   #in the
HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' #  web

      [download]
[reply]
[d/l]
Re: Re: My Mechanize script cannot work ?
by donno20 (Sexton) on Aug 08, 2003 at 06:09 UTC
    The code is pretty simple as the following 
    use WWW::Mechanize;
my $mech = WWW::Mechanize->new();
$mech->get("http://www.thesite.com/members/login.asp");
my $content = $mech->content();

$mech->form_number(1);
$form = $mech->current_form();

$mech->field('username', 'username');
$mech->field('password', 'pass123');
print $form->dump();

$result = $mech->submit();

    [download]
    So any comments ?
[reply]
[d/l]

      try this...

      # ...
my $mech = WWW::Mechanize->new();
$mech->agent('Mozilla/4.0 (compatible; MSIE 4.01; Windows 98)');   # l
+ie through your teeth.
# ...

      [download]
[reply]
[d/l]

      As far as I can see, there is nothing wrong with the script, so the problem possibly lies with the HTML from that site. Save the HTML of the site into a file (with $mech->getstore("http://www.thesite.com/members.login.asp", "login.html") and then try to reproduce the problem with a page as small and simple as possible. To load a page ino WWW::Mechanize, set the content like this :

        open HTML, "<", "login.html"
    or die "Couldn't read 'login.html' : $!\n";
  my $html = do { local $/; <HTML> };
  $mech->{content} = $html;
  $mech->_reset_page();
  if ( $mech->is_html ) {
        $mech->{forms} = [ HTML::Form->parse($mech->{content}, "http:/
+/www.thesite.com/members/") ];
        $mech->{form}  = $self->{forms}->[0];
        $mech->_extract_links();
    };

      [download]

      This is all I can do for you at the moment and I will be away over the weekend.

      perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ;    # The  
$d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider
($c = $d->accept())->get_request(); $c->send_response( new   #in the
HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' #  web

      [download]
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom