waiterm has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to download some statistics from http://www.statistics.gov.uk/STATBASE/tsdataset.asp?vlnk=7174 and all I want to get WWW::Mechanize to do is click on the button which says 'Display Complete Dataset'. I think this may in fact be some kind of cookie/session problem, because I've tried several different ways and they all result in me being redirected to the 'Customise Dataset' page unless I browse to the page I want using IE6. Here's one of the methods I've tried: 
$url = 'http://www.statistics.gov.uk/STATBASE/tsdataset.asp?vlnk=7174'
+;
$mech = WWW::Mechanize->new();
$mech->agent_alias( 'Windows IE 6' );
$mech->get($url);

$mech->click(B3,1,1);

$_ = $mech->content();
print $_;

[download]
This doesn't work because the button is disguised as a clickable image. Have also tried: 
$mech->submit_form(
        form_name => 'form1',
);

[download]
And i tried a couple of other things. I've also tried chaing the referer, and alias, neither work, I also tried cacthing the cookie in a cookie jar without success. If it is a cookie problem, why is Perl not caching this automatically for me? I would really appreciate some help on this as it's driving me completely crazy. Please email rob@hostpipe.co.uk with suggestions as am having trouble typing tonight, damaged hand playing rugby...oops!

Replies are listed 'Best First'.
Re: WWW::Mechanize issues, possibly cookie problem
by PodMaster (Abbot) on Sep 28, 2004 at 22:24 UTC
    Try the following code, automagically generated by WWW::Mechanize::Shell (and notice how it uses strict and warnings, quotes values properly ... ) as shown after the program (in <readmore>) 
    #!C:\pache\Perl\bin\perl.exe -w
use strict;
use WWW::Mechanize;
use WWW::Mechanize::FormFiller;
use URI::URL;

my $agent = WWW::Mechanize->new( autocheck => 1 );
my $formfiller = WWW::Mechanize::FormFiller->new();
$agent->env_proxy();

  $agent->get('http://www.statistics.gov.uk/STATBASE/tsdataset.asp?vln
+k=7174');
    $agent->form(1) if $agent->forms and scalar @{$agent->forms};
  $agent->form_number(3);
  { local $^W; $agent->current_form->value('B3', '1,1'); };
  $agent->click('B3');
  print $agent->content,"\n";

    [download]
    Read more... (136 kB)
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom