cfm links and lwp

webadept has asked for the wisdom of the Perl Monks concerning the following question:

Hey Monks,

I know one of you had to run into this at some time. I'm very versed in the LWP mod but this one is throwing me on two seperate sites. Here' s the demo link.

I've tried

use LWP::UserAgent;
$ua = new LWP::UserAgent;
$ua->agent('Mozilla/5.0'); 
my $cookies;
$ua->cookie_jar($cookies);

  $r = new HTTP::Request('GET', $url);
  $res = $ua->request($r);
  $this = $res->{_content};
[download]

and

use LWP::Simple;

  my $this = get $url;
  die "Couldn't get $url" unless defined $content;
[download]

both run and get nada :-(

So to see what the heck was going on I used both wget and lynx to see the page and got nothing as well.

I know nothing about Cold Fusion and what it does in the back ground. Anyone here able to grab the HTML from that Link? I'm out of ideas here. thanks,

webadept

Every day someone is doing what someone else said is impossible. _{edited by ybiC: convert text URL to PM-style link, to eliminate lateral browser-window scrolling}

Comment on cfm links and lwp Select or Download Code

Replies are listed 'Best First'.
Re: cfm links and lwp by Aristotle (Chancellor) on Aug 26, 2003 at 01:20 UTC
I get redirected to their homepage from following that link. `curl -I` says `HTTP/1.1 302 Moved Temporarily Server: Microsoft-IIS/5.0 Date: Tue, 26 Aug 2003 01:20:46 GMT Connection: close Set-Cookie: CFID=225704;expires=Thu, 18-Aug-2033 00:49:17 GMT;path=/ Set-Cookie: CFTOKEN=17470125;expires=Thu, 18-Aug-2033 00:49:17 GMT;pat +h=/ Set-Cookie: JSESSIONID=80301275331061860846968;path=/ Expires: Thu, 01 Dec 1994 16:00:00 GMT location: / Cache-Control: no-cache="set-cookie,set-cookie2" Content-Type: text/html; charset=UTF-8` [download] Obviously the server is expecting some cookie or something else you haven't supplied. The URL alone is not enough. That the page is written in ColdFusion is a red herring. Makeshifts last the longest.	[reply] [d/l]
Re: Re: cfm links and lwp by webadept (Pilgrim) on Aug 26, 2003 at 06:12 UTC
Just goes to show what staring at your own problem for too long can do to your eyes (and your mind) :-/ Well.. don't that just beat all. I ran my script hitting the front english link first and picking up the cookie and from there it all works fine. Thanks for opening my eyes.. should have checked my cooking in mozilla first. Oh well.. Many thanks for the help/slap. :-) webadept Every day someone is doing what someone else said is impossible.	[reply]
Re: cfm links and lwp by Cody Pendant (Prior) on Aug 26, 2003 at 03:50 UTC
You really don't like to use `or die "$!"` [download] do you? First thing I'd do is sprinkle my code liberally with that magic incantation. Secondly, I think you may have mistaken the cookie syntax for LWP. My (working) LWP script which needs to accept cookies has this: `$ua->cookie_jar(HTTP::Cookies->new(file => "cookies.txt", autosave => 1));` [download] `($_='kkvvttuubbooppuuiiffssqqffssmmiibbddllffss') =~y~b-v~a-z~s; print` [download]	[reply] [d/l] [select]
Re: cfm links and lwp by Anonymous Monk on Aug 26, 2003 at 11:25 UTC
What are you doing? Ask yourself, what are you doing? (or rather what is your program doing). It's pretending to be a browser, that's what. Think about that for a second, you're pretending to be a browser. Now be the browser ;) WWW::Mechanize helps you be the browser.	[reply]