in reply to Cookie Not Being Set

Maybe I am missing something but the $sessionid part of the URL doesn't seem to come from a cookie. It seems to be autogenerated on arrival at www.amazon.com. The 404 might be generated by using an unacceptable sessionid. I looked at my amazon.com cookie and the value was different to the sessionid in my urls.

From where I stand it appears like the following is the best approach to take is to go in through the front door at http://www.amazon.com/, get redircted to http://www.amazon.com/exec/obidos/subst/home/home.html/xxx-xxxxxxx-xxxxxxx (where x's are a valid sessionid) and use HTML::Parser and LWP::UserAgent to follow the links through to where you want to go. I know it is a bit of work but I don't see a way around it.

--blm-- If you don't like this post can you please /msg me

Replies are listed 'Best First'.
Re: Re: Cookie Not Being Set
by jonjacobmoon (Pilgrim) on Sep 28, 2002 at 18:53 UTC
    Been done and tried.

    It may not come from a cookie, but it is in the cookie and must match the cookie. I get a session-id and append it to the URL so I can have a valid Amazon URL. That is not the problem. The problem, as I see it, it that the cookie is not in the result object even though it appears to be in the request object. Where did it go?


    I admit it, I am Paco.
      the cookie is not in the result object

      Why would you expect it to be? The server isn't required to send the cookie (or even a SetCookie header) with each response. (This is a nit, but "response" is a better term to use than result.)

      Where did it go?

      You saved it in your cookie jar. That's where it is. Your browser will send the cookie along with subsequent requests so long as it hasn't expired. That's why you see it in the request object.

      -sauoq
      "My two cents aren't worth a dime.";
      
        I understand that, but I assume that if the cookie made it, then I would get a cookie back. I do not. Now, maybe my assumation is wrong, but I still have to wonder why this doesn't work on this site. I get redirected and lose the session id in the process. The script does work on other sites, so I am curious if anyone has an idea of what they might be doing differently.


        I admit it, I am Paco.
      The problem, as I see it, it that the cookie is not in the result object even though it appears to be in the request object.

      You say that the cookie "appears to be" in the request object. Have you verified that it is? If so, how?

        I should clarify that, sorry if that was confusing. If I change the code slightly, so it is thus:
        #!/usr/local/bin/perl -w use strict; use LWP::UserAgent; use HTTP::Cookies; use HTTP::Request::Common qw(POST GET);; use HTTP::Headers; use HTML::Parser; use URI; my $cururi; my $url; my @urls; my $sessionid; my $browser = LWP::UserAgent->new; my $email = 'xxxxxxxx'; my $password = 'xxxxxx'; my $req; my $res; #$browser->cookie_jar(HTTP::Cookies->new(file => 'cookie_jar', autosav +e => 1)); my $cookie_jar = HTTP::Cookies->new(file => 'cookie_jar', autosave => +1); $browser->cookie_jar($cookie_jar); my $initurl = "http://www.amazon.com/exec/obidos/flex-sign-in/ref=pd_n +fy_gw_si/"; &browserEmulation($initurl); open(CJ,"cookie_jar"); while (<CJ>) { chomp; if ($_ =~ /session-id\=\"(\d*-\d*-\d*)\"/) { $sessionid = $1; } } close(CJ); $url = "http://www.amazon.com/exec/obidos/flex-sign-in-done/$sessionid +"; #$res = $browser->simple_request(POST "$url", # { # 'email' => $email, # 'action' => 'sign-in checked', # 'next-page' => 'recs/instant-recs-sign-in-standard.ht +ml', # 'password' => $password, # 'method' => 'get', # 'opt' => 'oa', # 'page' => 'recs/instant-recs-register-standard.h +tml', # 'response' => 'tg/stores/static/-/goldbox/index/', # }); $req = POST $url, [ 'email' => $email, 'action' => 'sign-in checked', 'next-page' => 'recs/instant-recs-sign-in-standard.html', 'password' => $password, 'method' => 'get', 'opt' => 'oa', 'page' => 'recs/instant-recs-register-standard.html', 'response' => 'tg/stores/static/-/goldbox/index/', ]; $cookie_jar->add_cookie_header($req); $res = $browser->request($req); print $req->as_string(); print "\n\n=========================================================== +========\n\n"; print $res->as_string();
        When I print out the request and response objects I see the cookie in the request, yet no cookie is sent back in the response or appears to get through. Perhaps the server is doing something there, if so it is strange.