LWP Cookies in Perl 5.8

dataDrone has asked for the wisdom of the Perl Monks concerning the following question:

Just a quick question here.... Is anyone else having troubles getting Cookie data from an Apache server using Perl 5.8 on Win2K (yes Activestate build 804 see below for full details) I have a program that tries to log into a web server ( POST Method ) and should get back a Cookie with the "Secret handshake" to be used in following interaction with the web server. I get the confirmation document ( in the http content ) but an empty cookie. I'm at wit's end because the code works fine on Linux (RedHat 7.3 based, Perl version 5.6.1)and I'd like to not have two versions of the program. Both platforms use LWP 5.69, both use HTTP::Cookies version 1.30. Should I be at all concerned that perl -V reports osvers=4.0 while Microsoft has a habit of refering to Win2K as WIN NT Version 5.0? The offensive code is as follows.

use strict;
use LWP;
use HTTP::Cookies;
#use XML::DOM;
use XML::Twig;
use Data::Dumper;

my $oper = shift;
chomp($oper);
my $debug = shift;
my $user = 'username';
my $pword = 'userpassword';


my $ua = LWP::UserAgent->new;
  $ua->cookie_jar( {} );        # A place to hold the cookies from ETS
+S BO's

  $ua->agent("UPIXweb/1.0 ");

  # Create a request
  my $req = HTTP::Request->new(POST => 'http://etss.btv.ibm.com/xmes/p
+ack/usermgr');
  $req->content_type('application/x-www-form-urlencoded');
  $req->content("operation=authenticateUser&userid=$user"."&password=$
+pword");

C:\>perl -V
Summary of my perl5 (revision 5 version 8 subversion 0) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=define usemultip
+licity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -Gf -W3 -MD -DNDEBUG -O1 -DWIN32 -D_CON
+SOLE -DNO_STRICT -DHAVE_DES_FCRYPT  -DPE
RL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_REA
+DFIX',
    optimize='-MD -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64
+', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -release  -libpath:"C:\
+Perl\lib\CORE"  -machine:x86'
    libpth="C:\Program Files\Microsoft Visual Studio\VC98\mfc\lib" "C:
+\Program Files\Microsoft Visual Studio\V
C98\lib" "C:\Perl\lib\CORE"
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
+  comdlg32.lib advapi32.lib shell32.lib
ole32.lib oleaut32.lib  netapi32.lib uuid.lib wsock32.lib mpr.lib winm
+m.lib  version.lib odbc32.lib odbccp32.l
ib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool
+.lib  comdlg32.lib advapi32.lib shell32.
lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib wsock32.lib mpr.lib 
+winmm.lib  version.lib odbc32.lib odbccp
32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
    gnulibc_version='undef'
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release  -l
+ibpath:"C:\Perl\lib\CORE"  -machine:x86'



Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL
+_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
  Locally applied patches:
        ActivePerl Build 804
  Built under MSWin32
  Compiled at Dec  1 2002 23:15:13
  @INC:
    C:/Perl/lib
    C:/Perl/site/lib
    .
[download]

Somebody get me out of these "Twisty little passages, all alike".... Thanks, The Drone

Comment on LWP Cookies in Perl 5.8 Download Code

Replies are listed 'Best First'.
Re: LWP Cookies in Perl 5.8 by hardburn (Abbot) on Jun 26, 2003 at 16:07 UTC
I think `perl -V` will return the OS it was orginally built under, not what is currently running. ActivteState building under NT 4.0 makes sense, since there are still a good chunk of NT4 servers out there that wouldn't be able to run a version built on Win2k (just a guess). ---- I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident. -- Schemer Note: All code is untested, unless otherwise stated	[reply] [d/l]
Re: LWP Cookies in Perl 5.8 by dataDrone (Acolyte) on Jun 27, 2003 at 01:52 UTC
Ok so.... FYI, I upgraded to ActiveState build 806 of perl 5.8.0 and still the same problem. I'm guessing that it's something in the actual cookie header that is thwarting HTTP::Cookies's attempts to process the cookie. I've successfully traversed the maze all the way to Wit's End. here is the test case code and the exact output ( first on Windows Perl 5.8.0 build 806, then on RedHat Linux Perl 5.6.1 then again on Windows perl 5.6.1 build 631. #!/usr/bin/perl -w use strict; use LWP; use HTTP::Cookies; use Data::Dumper; my $oper = shift; my $user = 'user'; my $pword = 'pasword'; my $cj = HTTP::Cookies->new; my $ua = LWP::UserAgent->new; $ua->cookie_jar( $cj ); # A place to hold the cookies from ET +SS BO's $ua->agent("PerlWebBot/1.0 "); # Create a request my $req = HTTP::Request->new(POST => 'http://xmes.mydomain.com/pack/ +usermgr'); $req->content_type('application/x-www-form-urlencoded'); $req->content("operation=authenticateUser&userid=$user"."&password=$ +pword"); # Pass request to the user agent and get a response back my $res = $ua->request($req); # Check the outcome of the response if ($res->is_success) { print "Got something back...\n"; } else { print "Unable to contact server \n"; } print "LWP Version -> " , $LWP::VERSION , "\n"; print "Cookie Cutter -> " , $HTTP::Cookies::VERSION , "\n"; print $res->headers()->as_string() , "\n\n"; print Dumper($ua->cookie_jar() ) ; ----------------------------------------------------- results: D:\code>wipit.pl Got something back... LWP Version ->5.69 Cookie cutter -> 1.30 Connection: close Date: Fri, 27 Jun 2003 01:49:30 GMT Server: Apache/1.3.14 (Unix) tomcat/1.0 Content-Type: text/plain Client-Date: Fri, 27 Jun 2003 05:49:36 GMT Client-Peer: 9.61.8.22:80 Client-Response-Num: 1 Client-Transfer-Encoding: chunked Set-Cookie: member=QkFERVJEICB8KioqKioqKip8MTA1NjY3ODU3MXxXRVNUfDE0MzI +2fEVTVDVFRFR8VVN8ZW58OS42Ni4xNzQuMjAzfDc 0MTAzOHw=; expires=Fri, 27-Jun-2003 02:09:31 GMT; path=/; domain=btv.i +bm.com; $VAR1 = bless( { 'COOKIES' => { '.btv.ibm.com' => { '/' => {} } } }, 'HTTP::Cookies' ); Linux:----------- Got something back... LWP Version ->5.69 Cookie cutter -> 1.30 Connection: close Date: Fri, 27 Jun 2003 01:29:41 GMT Server: Apache/1.3.14 (Unix) tomcat/1.0 Content-Type: text/plain Client-Date: Fri, 27 Jun 2003 01:29:28 GMT Client-Peer: 9.61.8.22:80 Client-Response-Num: 1 Client-Transfer-Encoding: chunked Set-Cookie: member=QkFERVJEICB8KioqKioqKip8MTA1NjY3NzM4MnxXRVNUfDE0ODI +4fEVTVDVFRFR8VVN8ZW58OS42Ni4xNzUuMTk3fDc 0MTAzOHw=; expires=Fri, 27-Jun-2003 01:49:42 GMT; path=/; domain=btv.i +bm.com; $VAR1 = bless( { 'COOKIES' => { '.btv.ibm.com' => { '/' => { 'member' +=> [ + 0, + 'QkFERVJEICB8KioqKioqKip8MTA1NjY3Nz M4MnxXRVNUfDE0ODI4fEVTVDVFRFR8VVN8ZW58OS42Ni4xNzUuMTk3fDc0MTAzOHw=', + undef, + '1', + undef, + '1056678582' + ] } } } }, 'HTTP::Cookies' ); Window Perl 5.6.1 ---------------------- Got something back... LWP Version ->5.51 Cookie cutter -> 1.14 Connection: close Date: Fri, 27 Jun 2003 01:36:56 GMT Server: Apache/1.3.14 (Unix) tomcat/1.0 Content-Type: text/plain Client-Date: Fri, 27 Jun 2003 01:43:09 GMT Client-Peer: 9.61.8.22:80 Set-Cookie: member=QkFERVJEICB8KioqKioqKip8MTA1NjY3NzgxN3xXRVNUfDMyMjI +3fEVTVDVFRFR8VVN8ZW58OS42Ni4xNzUuMTR8NzQ xMDM4fA==; expires=Fri, 27-Jun-2003 01:56:57 GMT; path=/; domain=btv.ibm.com; $VAR1 = bless( { 'COOKIES' => { '.btv.ibm.com' => { '/' => { 'member' +=> [ + 0, + 'QkFERVJEICB8KioqKioqKip8MTA1NjY3NzgxN3xXRVNU fDMyMjI3fEVTVDVFRFR8VVN8ZW58OS42Ni4xNzUuMTR8NzQxMDM4fA==', + undef, + '1', + undef, + '1056679017' + ] } } } }, 'HTTP::Cookies' ); [download]	[reply] [d/l]
Re: Re: LWP Cookies in Perl 5.8 by dataDrone (Acolyte) on Jun 27, 2003 at 02:34 UTC
Using use LWP::Debug qw(+); i get the following, but i'm still not seeing any smoking gun here. LWP::UserAgent::new: () LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking etss.btv.ibm.com for cookie +s HTTP::Cookies::add_cookie_header: Checking .btv.ibm.com for cookies HTTP::Cookies::add_cookie_header: Checking btv.ibm.com for cookies HTTP::Cookies::add_cookie_header: Checking .ibm.com for cookies HTTP::Cookies::add_cookie_header: Checking ibm.com for cookies HTTP::Cookies::add_cookie_header: Checking .com for cookies LWP::UserAgent::send_request: POST http://etss.btv.ibm.com/xmes/pack/u +sermgr LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 376 bytes HTTP::Cookies::extract_cookies: Set cookie member => QkFERVJEICB8KioqK +ioqKip8MTA1NjY4MDQxNnxXRVNUfDE2OTM3fEVTV DVFRFR8VVN8ZW58OS42Ni4xNzQuMjAzfDc0MTAzOHw= LWP::UserAgent::request: Simple response: OK Got something back... LWP Version ->5.69 Cookie cutter -> 1.30 Connection: close Date: Fri, 27 Jun 2003 02:20:16 GMT Server: Apache/1.3.14 (Unix) tomcat/1.0 Content-Type: text/plain Client-Date: Fri, 27 Jun 2003 06:20:22 GMT Client-Peer: 9.61.8.22:80 Client-Response-Num: 1 Client-Transfer-Encoding: chunked Set-Cookie: member=QkFERVJEICB8KioqKioqKip8MTA1NjY4MDQxNnxXRVNUfDE2OTM +3fEVTVDVFRFR8VVN8ZW58OS42Ni4xNzQuMjAzfDc 0MTAzOHw=; expires=Fri, 27-Jun-2003 02:40:16 GMT; path=/; domain=btv.i +bm.com; [download]	[reply] [d/l]