You must have looked in perldoc lwpcook, which is very good. That is further than most people get.

When you say, "I could not get it to work", what do you mean? The best code snippet I could provide would be straight out of the documentation you read. Perhaps the site doesn't use basic authentication? The more information you provide (errors, warnings), the easier it is to help you.

The same document has a suggestion for retrieving multiple pages from a site. (HTTP is generally a connectionless protocol -- at least, for the task you're doing, so there's no need to keep a connection alive.)

To use cookies, have a look at HTTP::Cookies and LWP::UserAgent. They have good examples in there. You might also look in Super Search for previous questions on the same topic... someone brought this up a few months back.

Update: Fixed a typo, thanks to dws.

In reply to Re: Access to protected Sites by chromatic
in thread Access to protected Sites by mak

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.