in reply to Re: is winproxy the problem? send additional header info?
in thread LWP, extract_cookies, etc.

Here is the full LWP debug log from running my script. The problem is that it doesn't seem that the second post is using the cookie because the page is being directed to a default sign up page.

One thing I can see is the "no proxy" message, but I don't know what that means or any resolution. THANKS!!

use perl5::site_perl::5.005::LWP::UserAgent;    #perl2exe
use perl5::site_perl::5.005::HTTP::Request;    #perl2exe
use perl5::site_perl::5.005::HTTP::Message;    #perl2exe
use perl5::site_perl::5.005::HTTP::Headers;    #perl2exe
use perl5::site_perl::5.005::URI;    #perl2exe
use perl5::site_perl::5.005::URI::Escape;    #perl2exe
use /usr/libdata/perl/5.00503/overload.pm;    #perl2exe
use perl5::site_perl::5.005::HTTP::Response;    #perl2exe
use perl5::site_perl::5.005::HTTP::Status;    #perl2exe
use perl5::site_perl::5.005::HTTP::Date;    #perl2exe
use /usr/libdata/perl/5.00503/Time/Local.pm;    #perl2exe
use perl5::site_perl::5.005::LWP;    #perl2exe
use perl5::site_perl::5.005::LWP::Debug;    #perl2exe
use perl5::site_perl::5.005::LWP::Protocol;    #perl2exe
use perl5::site_perl::5.005::LWP::MemberMixin;    #perl2exe
use perl5::site_perl::5.005::HTTP::Request::Common;    #perl2exe
use perl5::site_perl::5.005::HTTP::Cookies;    #perl2exe
use perl5::site_perl::5.005::HTTP::Headers::Util;    #perl2exe
LWP::UserAgent::new: ()
use perl5::site_perl::5.005::URI::http;    #perl2exe
use perl5::site_perl::5.005::URI::_server;    #perl2exe
use perl5::site_perl::5.005::URI::_generic;    #perl2exe
use perl5::site_perl::5.005::URI::_query;    #perl2exe
LWP::UserAgent::request: ()
HTTP::Cookies::add_cookie_header: Checking www.sitedomain.com for cook
+ies
HTTP::Cookies::add_cookie_header: - checking cookie path=/
HTTP::Cookies::add_cookie_header:  - checking cookie LASTVISIT=%7Bts+%
+272002%2D05%2D22+23%3A02%3A55%27%7D
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFGLOBALS=HITCOUN
+T%3D53%23LASTVISIT%3D%7Bts+%272002%2D05%2D22+23%3A02%3A55%27%7D%23TIM
+ECREATED%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFTOKEN=60606369
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie OLDDATE=%7Bts+%27
+2002%2D05%2D22+12%3A56%3A16%27%7D
HTTP::Cookies::add_cookie_header:    expired
HTTP::Cookies::add_cookie_header:  - checking cookie CFID=2772192
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie PGSID=60570720020
+513131253
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header: Checking .sitedomain.com for cookies
HTTP::Cookies::add_cookie_header: Checking sitedomain.com for cookies
HTTP::Cookies::add_cookie_header: Checking .com for cookies
LWP::UserAgent::send_request: POST http://www.sitedomain.com/shared/ge
+tuser.cfm
LWP::UserAgent::_need_proxy: Not proxied
use perl5::site_perl::5.005::LWP::Protocol::http;    #perl2exe
use perl5::site_perl::5.005::Net::HTTP;    #perl2exe
use /usr/libdata/perl/5.00503/mach/IO/Socket.pm;    #perl2exe
use /usr/libdata/perl/5.00503/mach/IO/Handle.pm;    #perl2exe
use perl5::site_perl::5.005::Net::HTTP::Methods;    #perl2exe
LWP::Protocol::http::request: ()
use perl5::site_perl::5.005::i386-freebsd::HTML::HeadParser;    #perl2
+exe
use perl5::site_perl::5.005::i386-freebsd::HTML::Entities;    #perl2ex
+e
use perl5::site_perl::5.005::i386-freebsd::HTML::Parser;    #perl2exe
LWP::Protocol::collect: read 209 bytes
LWP::UserAgent::request: Simple response: Found
LWP::UserAgent::request: ()
HTTP::Cookies::add_cookie_header: Checking www.sitedomain.com for cook
+ies
HTTP::Cookies::add_cookie_header: - checking cookie path=/
HTTP::Cookies::add_cookie_header:  - checking cookie LASTVISIT=%7Bts+%
+272002%2D05%2D22+23%3A02%3A55%27%7D
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFGLOBALS=HITCOUN
+T%3D53%23LASTVISIT%3D%7Bts+%272002%2D05%2D22+23%3A02%3A55%27%7D%23TIM
+ECREATED%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFTOKEN=60606369
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie OLDDATE=%7Bts+%27
+2002%2D05%2D22+12%3A56%3A16%27%7D
HTTP::Cookies::add_cookie_header:    expired
HTTP::Cookies::add_cookie_header:  - checking cookie CFID=2772192
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie PGSID=60570720020
+513131253
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header: Checking .sitedomain.com for cookies
HTTP::Cookies::add_cookie_header: Checking sitedomain.com for cookies
HTTP::Cookies::add_cookie_header: Checking .com for cookies
LWP::UserAgent::send_request: POST http://www.sitedomain.com/developer
+s/index.cfm?sid=11247720020523182832&CFID=2772192&CFTOKEN=60606369
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 411 bytes
LWP::Protocol::collect: read 1736 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 4096 bytes
LWP::Protocol::collect: read 44 bytes
LWP::Protocol::collect: read 2081 bytes
HTTP::Cookies::extract_cookies: Set cookie CFGLOBALS => HITCOUNT%3D54%
+23LASTVISIT%3D%7Bts+%272002%2D05%2D23+18%3A28%3A33%27%7D%23TIMECREATE
+D%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23
HTTP::Cookies::extract_cookies: Set cookie LASTVISIT => %7Bts+%272002%
+2D05%2D23+18%3A28%3A33%27%7D
HTTP::Cookies::extract_cookies: Set cookie OLDDATE => %7Bts+%272002%2D
+05%2D22+23%3A02%3A55%27%7D
HTTP::Cookies::extract_cookies: Set cookie PGSID => 112477200205231828
+32
LWP::UserAgent::request: Simple response: OK
LWP::UserAgent::request: ()
HTTP::Cookies::add_cookie_header: Checking www.sitedomain.com for cook
+ies
HTTP::Cookies::add_cookie_header: - checking cookie path=/
HTTP::Cookies::add_cookie_header:  - checking cookie LASTVISIT=%7Bts+%
+272002%2D05%2D23+18%3A28%3A33%27%7D
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFGLOBALS=HITCOUN
+T%3D54%23LASTVISIT%3D%7Bts+%272002%2D05%2D23+18%3A28%3A33%27%7D%23TIM
+ECREATED%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFTOKEN=60606369
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie OLDDATE=%7Bts+%27
+2002%2D05%2D22+23%3A02%3A55%27%7D
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFID=2772192
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie PGSID=11247720020
+523182832
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header: Checking .sitedomain.com for cookies
HTTP::Cookies::add_cookie_header: Checking sitedomain.com for cookies
HTTP::Cookies::add_cookie_header: Checking .com for cookies
LWP::UserAgent::send_request: POST http://www.sitedomain.com/developer
+s/devedit.cfm
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 188 bytes
LWP::UserAgent::request: Simple response: Found
LWP::UserAgent::request: ()
HTTP::Cookies::add_cookie_header: Checking www.sitedomain.com for cook
+ies
HTTP::Cookies::add_cookie_header: - checking cookie path=/
HTTP::Cookies::add_cookie_header:  - checking cookie LASTVISIT=%7Bts+%
+272002%2D05%2D23+18%3A28%3A33%27%7D
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFGLOBALS=HITCOUN
+T%3D54%23LASTVISIT%3D%7Bts+%272002%2D05%2D23+18%3A28%3A33%27%7D%23TIM
+ECREATED%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFTOKEN=60606369
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie OLDDATE=%7Bts+%27
+2002%2D05%2D22+23%3A02%3A55%27%7D
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie CFID=2772192
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header:  - checking cookie PGSID=11247720020
+523182832
HTTP::Cookies::add_cookie_header:    it's a match
HTTP::Cookies::add_cookie_header: Checking .sitedomain.com for cookies
HTTP::Cookies::add_cookie_header: Checking sitedomain.com for cookies
HTTP::Cookies::add_cookie_header: Checking .com for cookies
LWP::UserAgent::send_request: POST http://www.sitedomain.com/developer
+s/signup.cfm?sid=60570720020513131253&CFID=2772192&CFTOKEN=60606369
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 524 bytes
LWP::Protocol::collect: read 1736 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1380 bytes
LWP::Protocol::collect: read 1329 bytes
HTTP::Cookies::extract_cookies: Set cookie CFGLOBALS => HITCOUNT%3D55%
+23LASTVISIT%3D%7Bts+%272002%2D05%2D23+18%3A28%3A35%27%7D%23TIMECREATE
+D%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23
HTTP::Cookies::extract_cookies: Set cookie LASTVISIT => %7Bts+%272002%
+2D05%2D23+18%3A28%3A35%27%7D
HTTP::Cookies::extract_cookies: Set cookie PGSID => 605707200205131312
+53
LWP::UserAgent::request: Simple response: OK
worked

[download]

I appreciate all your help!

Michael Jensen
michael at inshift.com
http://www.inshift.com

Replies are listed 'Best First'.
holy cow it is working!
by inblosam (Monk) on May 24, 2002 at 09:31 UTC
    Well, I'm happy to say the script is working like a charm. I think the last thing we had to change was get a new session id from the website and I did that real simply with:

    #!/usr/bin/perl -w
#see IlyaM, I am being a good boy!
use strict;
use LWP::Simple;
my $page = get 'http://www.sitedomain.com/login.cfm';
$page =~ /sid" VALUE="(\w+)/;
my $sid = $1;
print $sid;

    [download]

    Thanks to everyone (especially IlyaM!) for their help!

    Michael Jensen
    michael at inshift.com
    http://www.inshift.com
[reply]
[d/l]
[reply]

In Section Seekers of Perl Wisdom