in reply to Mimic referring URL in LWP?

HTTP::Headers has a referer method (yes, it's mis-spelt in the HTTP standard, too). It does exactly what you expect in that it sets the Referer header in the HTTP request.

You should be able to tweak the HTTP::Headers object that is used by your request: 

$req->headers->referer("http://www.perlmonks.org/");

[download]

In fact, according to the HTTP::Message documentation, all unknown methods are fobbed off to the appropriate HTTP::Headers object, so the following also works: 

$req->referer("http://www.perlmonks.org/");

[download]

Couldn't be much more simple than that. :)

Paul Fenwick
Perl Training Australia

Replies are listed 'Best First'.
Re: Re: Mimic referring URL in LWP?
by inblosam (Monk) on Jun 20, 2002 at 03:14 UTC
    I tried the "referer" suggestions, but doesn't seem to help. I placed it after both my post and my get, like this: 
    my $req = POST 'https://adwords.google.com/select/main',
             ['login.userid'       => 'test@test.com', 
              'login.password'     => 'testabc',
              'cmd'             => 'LoginValidation',
              'login'            => 'Login'
             ];
$req->referer("https://adwords.google.com/select/");

my $req = HTTP::Request->new(GET=>'https://adwords.google.com/select/m
+ain?cmd=CampaignManagement&campaignid=0&timeperiod=simple&timeperiod.
+simpletimeperiod=today');
$req->referer("https://adwords.google.com/select/main");

    [download]
    Is the referrer page the one that brought me to the page I am posting to or getting (that was my assumption)?
    Also, in my headers returned from my "Post" I get this:
    Client-SSL-Warning: Peer certificate not verified

    [download]

    I didn't know if that was an issue or not. Something they are looking for I am not sending them, but I can't figure it out. Also, I realized the response from my "Get" is not the same as the first, which seems a little odd:
    Your session has expired. Please return to the AdWords Select homepage
+ and login again. 

(This is a security precaution to prevent someone from gaining access 
+to your account if you forget to log-out.)

    [download]
    Is the log in working then? But somehow my cookie shows the session is timed out? Thanks!

    Michael Jensen
    michael at inshift.com
    http://www.inshift.com
[reply]
[d/l]
[select]
      It seems fairly clear that Google either does not expect or does not want automated scripts to access this particular facility. You may find it worthwhile asking them if they have a more programmer-friendly method of accessing the information that you're after, as that will save a lot of hassle trying to reverse-engineer the whole process.

      Judging from the session-expired message, I would judge that google requires you to go through the whole login process to get a valid cookie, as even if these don't expire on the client, they do on their server.

      It's very common for sites to stop accepting old cookies, particularly when money's involved. They want to avoid the situation of having cookies stored on a public computer, and a potential third-party accessing the content in question.

      If you were dealing with un-encrypted HTTP sessions, then you could use tcpdump/ethereal to log and examine what's happening "under-the-hood". However, since your connection is proceeding via SSL, that's not an easy option.

      Paul Fenwick
      Perl Training Australia

[reply]

In Section Seekers of Perl Wisdom