Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

how would you use HTML::Treebuilder or HTML::Parser to get a form from a cgi page? and also how would i extract the value of the action command? Thank you.

Replies are listed 'Best First'.
Re: ethixx
by lhoward (Vicar) on Jun 03, 2000 at 17:17 UTC
    Use the HTTP::Cookies module in conjunction with your LWP::UserAgent. I frequently use it like this to give the UserAgent a "normal browserish" looking store for cookies (the cookie jar used in association w/ LWP::UserAgent will receive and send cookies just like a normal browser would):
    use HTTP::Cookies; my $agent = new LWP::UserAgent; my $co=new HTTP::Cookies(file=>'./stored.cookies',autosave=>1); $agent->cookie_jar($co);
    There are sevearl methods in that module for manipluating cookies/the cookie jar.
      i'd just like to point out that when the cookie_jar in LWP::UserAgent is used in this manner, it will automatically grab and store the cookies from all HTTP::Responses returned by the UserAgent. it will also automatically add cookies to all HTTP::Request objects that pass through it.

      so, if you set a UserAgent's cookie jar, there is typically no need to manipulate the cookie jar manually (i.e. with extract_cookies and add_cookie_header).

Answer: Getting cookies from a HTTP response
by Corion (Patriarch) on Jun 03, 2000 at 17:32 UTC

    Note: All links here don't link directly to the documentation but to the CPAN search engine for the documentation, sorry for the inconvenience !

    I haven't used HTTP::Request and LWP::UserAgent much, but by looking into the documentation of HTTP::Message and of HTTP::Response, it seems to me as if this (untested) code should work :

    require LWP::UserAgent; require HTTP::Message; $ua = new LWP::UserAgent; $request = new HTTP::Request('GET', 'file://localhost/etc/motd'); $response = $ua->request($request); # or $response = $ua->request($request, '/tmp/sss'); # or $response = $ua->request($request, \&callback, 4096); print $response->headers_as_string, "\n"; print $response->headers->header( "Set-Cookie" ), "\n"; sub callback { my($data, $response, $protocol) = @_; # your stuff here, like status etc. }
    Hope that helps a bit, but maybe some of the other monks with more experience will step in ;). I'm also not really sure about the Set-Cookie header - if noone here provides an authoritative answer, you might like to check the according RFC, that would be RFC 2616, available here and RFC 2817.

    Another interesting thing for programmers that have to work with RFCs is the RFCsearch engine, written in Perl :)