How to get password protected content via LWP - by emulating my own browser (that can view that content)

Prat824 has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks I am trying to get content of a url that I can view on my browser for some time (till the cookie expires I guess) but need to get through LWP. I have copied the cookies.txt file from mozilla profiles and using that to pretend that LWP was used for authntication. --------------------------------

my $url = 'http://www.princetonreview.com/schools/college/CollegeAdmis
+sions.aspx?iid=1023828&uidbadge=%07';

my $browser = LWP::UserAgent->new;

$browser->agent('Mozilla/4.76 [en] (Win98; U)');

my $cookie_jar = HTTP::Cookies::Mozilla->new(qw( autosave 1 file cooki
+es.txt ) );    

$browser->cookie_jar( $cookie_jar );

my $response = $browser->get( $url );

print $response->content;
[download]

--------------------------------- The resulting content is always the same authentication page asking for username and password but not the real page. As you can see I am not using authentication in my code but want to make use of already authenticated session that I started on mozilla. Any help would be appreciated. Thanks

Comment on How to get password protected content via LWP - by emulating my own browser (that can view that content) Download Code

Replies are listed 'Best First'.
Re: How to get password protected content via LWP - by emulating my own browser (that can view that content) by Lawliet (Curate) on Sep 03, 2008 at 21:49 UTC
"Want to make use of already authenticated session that I started on mozilla." Is there a specific reason for this? If not, try the credentials() method? `use LWP::UserAgent qw(); my $ua = LWP::UserAgent->new; my $netloc = 'http://www.buddhism-dict.net/cgi-bin/xpr-dealt.pl:80'; $ua->credentials( $netloc, 'CJK-E and Buddhist Dictionaries', # basic realm 'guest', # user '', # empty pw ); my $response = $ua->get($netloc);` [download] Example taken from Rosetta Code. I'm so adjective, I verb nouns! chomp; # nom nom nom	[reply] [d/l]
Re^2: How to get password protected content via LWP - by emulating my own browser (that can view that content) by Prat824 (Acolyte) on Sep 03, 2008 at 23:31 UTC
Thanks Lawliet No - there is no specific reason, but I tried authentication using LWP and failed so thought of using the mozilla cookiestore to utilized the already existing session. as regards your suggestion, do you know from where I could get the basic realm for my site? Thanks Prat	[reply]
Re^3: How to get password protected content via LWP - by emulating my own browser (that can view that content) by Lawliet (Curate) on Sep 03, 2008 at 23:36 UTC
I believe it is what appears in between the quotes when the popup box asks for the password when you try to navigate to the page in a browser.</baddirections> Something like `Enter your username and password for 'realm'` I'm so adjective, I verb nouns! chomp; # nom nom nom	[reply] [d/l]
Re^4: How to get password protected content via LWP - by emulating my own browser (that can view that content) by Anonymous Monk on Sep 04, 2008 at 06:57 UTC
Re: How to get password protected content via LWP - by emulating my own browser (that can view that content) by Lawliet (Curate) on Sep 04, 2008 at 18:33 UTC
I was mistaken in my previous comment on this node. I think it might be best for you to use WWW::Mechanize to fill out the html form and submit it. Depending on the form however, you may want to check out WWW::Selenium - which is used for handling javascript. I'm so adjective, I verb nouns! chomp; # nom nom nom	[reply]
Re: How to get password protected content via LWP - by emulating my own browser (that can view that content) by Anonymous Monk on Sep 04, 2008 at 07:03 UTC
As you can see I am not using authentication in my code but want to make use of already authenticated session that I started on mozilla. Any help would be appreciated. Thanks What exactly is the hold up?	[reply]