is winproxy the problem? send additional header info?

Replies are listed 'Best First'.
Re: is winproxy the problem? send additional header info? by IlyaM (Parson) on May 23, 2002 at 19:50 UTC
I don't think your problem is releated to headers. Anyway you can add any header to the request using following code `$request->header(Header => 'Value');` [download] In you code you will have to replace `$ua->request(POST ...);` [download] with `my $request = POST ...; $request->header(Header => 'Value'); ... $us->request($request);` [download] BTW you don't have to insert header User-agent. LWP sends it itself. See documentation for method `agent` in LWP::UserAgent. P.S. Post full debug logs here. Maybe you have overlooked anything important there. -- Ilya Martynov (http://martynov.org/)	[reply] [d/l] [select]
Full LWP debug log by inblosam (Monk) on May 24, 2002 at 01:32 UTC
Here is the full LWP debug log from running my script. The problem is that it doesn't seem that the second post is using the cookie because the page is being directed to a default sign up page. One thing I can see is the "no proxy" message, but I don't know what that means or any resolution. THANKS!! use perl5::site_perl::5.005::LWP::UserAgent; #perl2exe use perl5::site_perl::5.005::HTTP::Request; #perl2exe use perl5::site_perl::5.005::HTTP::Message; #perl2exe use perl5::site_perl::5.005::HTTP::Headers; #perl2exe use perl5::site_perl::5.005::URI; #perl2exe use perl5::site_perl::5.005::URI::Escape; #perl2exe use /usr/libdata/perl/5.00503/overload.pm; #perl2exe use perl5::site_perl::5.005::HTTP::Response; #perl2exe use perl5::site_perl::5.005::HTTP::Status; #perl2exe use perl5::site_perl::5.005::HTTP::Date; #perl2exe use /usr/libdata/perl/5.00503/Time/Local.pm; #perl2exe use perl5::site_perl::5.005::LWP; #perl2exe use perl5::site_perl::5.005::LWP::Debug; #perl2exe use perl5::site_perl::5.005::LWP::Protocol; #perl2exe use perl5::site_perl::5.005::LWP::MemberMixin; #perl2exe use perl5::site_perl::5.005::HTTP::Request::Common; #perl2exe use perl5::site_perl::5.005::HTTP::Cookies; #perl2exe use perl5::site_perl::5.005::HTTP::Headers::Util; #perl2exe LWP::UserAgent::new: () use perl5::site_perl::5.005::URI::http; #perl2exe use perl5::site_perl::5.005::URI::_server; #perl2exe use perl5::site_perl::5.005::URI::_generic; #perl2exe use perl5::site_perl::5.005::URI::_query; #perl2exe LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking www.sitedomain.com for cook +ies HTTP::Cookies::add_cookie_header: - checking cookie path=/ HTTP::Cookies::add_cookie_header: - checking cookie LASTVISIT=%7Bts+% +272002%2D05%2D22+23%3A02%3A55%27%7D HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFGLOBALS=HITCOUN +T%3D53%23LASTVISIT%3D%7Bts+%272002%2D05%2D22+23%3A02%3A55%27%7D%23TIM +ECREATED%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFTOKEN=60606369 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie OLDDATE=%7Bts+%27 +2002%2D05%2D22+12%3A56%3A16%27%7D HTTP::Cookies::add_cookie_header: expired HTTP::Cookies::add_cookie_header: - checking cookie CFID=2772192 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie PGSID=60570720020 +513131253 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: Checking .sitedomain.com for cookies HTTP::Cookies::add_cookie_header: Checking sitedomain.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: POST http://www.sitedomain.com/shared/ge +tuser.cfm LWP::UserAgent::_need_proxy: Not proxied use perl5::site_perl::5.005::LWP::Protocol::http; #perl2exe use perl5::site_perl::5.005::Net::HTTP; #perl2exe use /usr/libdata/perl/5.00503/mach/IO/Socket.pm; #perl2exe use /usr/libdata/perl/5.00503/mach/IO/Handle.pm; #perl2exe use perl5::site_perl::5.005::Net::HTTP::Methods; #perl2exe LWP::Protocol::http::request: () use perl5::site_perl::5.005::i386-freebsd::HTML::HeadParser; #perl2 +exe use perl5::site_perl::5.005::i386-freebsd::HTML::Entities; #perl2ex +e use perl5::site_perl::5.005::i386-freebsd::HTML::Parser; #perl2exe LWP::Protocol::collect: read 209 bytes LWP::UserAgent::request: Simple response: Found LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking www.sitedomain.com for cook +ies HTTP::Cookies::add_cookie_header: - checking cookie path=/ HTTP::Cookies::add_cookie_header: - checking cookie LASTVISIT=%7Bts+% +272002%2D05%2D22+23%3A02%3A55%27%7D HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFGLOBALS=HITCOUN +T%3D53%23LASTVISIT%3D%7Bts+%272002%2D05%2D22+23%3A02%3A55%27%7D%23TIM +ECREATED%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFTOKEN=60606369 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie OLDDATE=%7Bts+%27 +2002%2D05%2D22+12%3A56%3A16%27%7D HTTP::Cookies::add_cookie_header: expired HTTP::Cookies::add_cookie_header: - checking cookie CFID=2772192 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie PGSID=60570720020 +513131253 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: Checking .sitedomain.com for cookies HTTP::Cookies::add_cookie_header: Checking sitedomain.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: POST http://www.sitedomain.com/developer +s/index.cfm?sid=11247720020523182832&CFID=2772192&CFTOKEN=60606369 LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 411 bytes LWP::Protocol::collect: read 1736 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 4096 bytes LWP::Protocol::collect: read 44 bytes LWP::Protocol::collect: read 2081 bytes HTTP::Cookies::extract_cookies: Set cookie CFGLOBALS => HITCOUNT%3D54% +23LASTVISIT%3D%7Bts+%272002%2D05%2D23+18%3A28%3A33%27%7D%23TIMECREATE +D%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23 HTTP::Cookies::extract_cookies: Set cookie LASTVISIT => %7Bts+%272002% +2D05%2D23+18%3A28%3A33%27%7D HTTP::Cookies::extract_cookies: Set cookie OLDDATE => %7Bts+%272002%2D +05%2D22+23%3A02%3A55%27%7D HTTP::Cookies::extract_cookies: Set cookie PGSID => 112477200205231828 +32 LWP::UserAgent::request: Simple response: OK LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking www.sitedomain.com for cook +ies HTTP::Cookies::add_cookie_header: - checking cookie path=/ HTTP::Cookies::add_cookie_header: - checking cookie LASTVISIT=%7Bts+% +272002%2D05%2D23+18%3A28%3A33%27%7D HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFGLOBALS=HITCOUN +T%3D54%23LASTVISIT%3D%7Bts+%272002%2D05%2D23+18%3A28%3A33%27%7D%23TIM +ECREATED%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFTOKEN=60606369 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie OLDDATE=%7Bts+%27 +2002%2D05%2D22+23%3A02%3A55%27%7D HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFID=2772192 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie PGSID=11247720020 +523182832 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: Checking .sitedomain.com for cookies HTTP::Cookies::add_cookie_header: Checking sitedomain.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: POST http://www.sitedomain.com/developer +s/devedit.cfm LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 188 bytes LWP::UserAgent::request: Simple response: Found LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking www.sitedomain.com for cook +ies HTTP::Cookies::add_cookie_header: - checking cookie path=/ HTTP::Cookies::add_cookie_header: - checking cookie LASTVISIT=%7Bts+% +272002%2D05%2D23+18%3A28%3A33%27%7D HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFGLOBALS=HITCOUN +T%3D54%23LASTVISIT%3D%7Bts+%272002%2D05%2D23+18%3A28%3A33%27%7D%23TIM +ECREATED%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFTOKEN=60606369 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie OLDDATE=%7Bts+%27 +2002%2D05%2D22+23%3A02%3A55%27%7D HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie CFID=2772192 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: - checking cookie PGSID=11247720020 +523182832 HTTP::Cookies::add_cookie_header: it's a match HTTP::Cookies::add_cookie_header: Checking .sitedomain.com for cookies HTTP::Cookies::add_cookie_header: Checking sitedomain.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: POST http://www.sitedomain.com/developer +s/signup.cfm?sid=60570720020513131253&CFID=2772192&CFTOKEN=60606369 LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 524 bytes LWP::Protocol::collect: read 1736 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1380 bytes LWP::Protocol::collect: read 1329 bytes HTTP::Cookies::extract_cookies: Set cookie CFGLOBALS => HITCOUNT%3D55% +23LASTVISIT%3D%7Bts+%272002%2D05%2D23+18%3A28%3A35%27%7D%23TIMECREATE +D%3D%7Bts+%272002%2D05%2D20+23%3A47%3A29%27%7D%23 HTTP::Cookies::extract_cookies: Set cookie LASTVISIT => %7Bts+%272002% +2D05%2D23+18%3A28%3A35%27%7D HTTP::Cookies::extract_cookies: Set cookie PGSID => 605707200205131312 +53 LWP::UserAgent::request: Simple response: OK worked [download] I appreciate all your help! Michael Jensen michael at inshift.com http://www.inshift.com	[reply] [d/l]
holy cow it is working! by inblosam (Monk) on May 24, 2002 at 09:31 UTC
Well, I'm happy to say the script is working like a charm. I think the last thing we had to change was get a new session id from the website and I did that real simply with: `#!/usr/bin/perl -w #see IlyaM, I am being a good boy! use strict; use LWP::Simple; my $page = get 'http://www.sitedomain.com/login.cfm'; $page =~ /sid" VALUE="(\w+)/; my $sid = $1; print $sid;` [download] Thanks to everyone (especially IlyaM!) for their help! Michael Jensen michael at inshift.com http://www.inshift.com	[reply] [d/l]
Re: holy cow it is working! by IlyaM (Parson) on May 24, 2002 at 10:52 UTC