My obstacle now seems to be that the site (adwords.google.com) checks for a referring URL (or something else I can't figure out). How do I mimic that? Otherwise I am not able to get the page I need to read/parse, because I get the following message (on the first result):

Sorry, we're unable to process this request. There are two possible ca
+uses for this error: 

You accessed this page via a bookmark instead of entering through the 
+AdWords Select homepage 
You have disabled cookies in your browser.

[download]

So I need to mimic the referring URL, and perhaps there are some other things google is doing to keep people from coming to their pages from a non-traditional way. This is what my script looks like right now: 
#!/usr/lib/perl -w

use strict;

use LWP::UserAgent;
use Crypt::SSLeay;
use HTTP::Request::Common;
use HTTP::Cookies;
use LWP::Simple;
use LWP::Debug qw(+);

my $ua = LWP::UserAgent->new;

$ua->cookie_jar(HTTP::Cookies->new(file => "cookie_jar", autosave => 1
+));
$ua->timeout(300);
   
my $req = POST 'https://adwords.google.com/select/main',
             ['login.userid'       => 'test@test.com', 
              'login.password'     => 'testabc',
              'cmd'             => 'LoginValidation',
              'login'            => 'Login'
             ];

my $res = $ua->request($req);
 
    unless ($res->is_success) {
        print "Login Failed: : ". $res->status_line .  "\n";
        return 0;
    }

my $b = $res->as_string;

print "\n\n\nThe 1st result is:\n$b\n";

my $req = HTTP::Request->new(GET=>'https://adwords.google.com/select/m
+ain?cmd=CampaignManagement&campaignid=0&timeperiod=simple&timeperiod.
+simpletimeperiod=today');
my $res = $ua->request($req);

my $c = $res->as_string;

print "\n\n\nThe 2nd result is:\n$c\n";

[download]


Michael Jensen
michael at inshift.com
http://www.inshift.com

In reply to Mimic referring URL in LWP? by inblosam

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.