mak has asked for the wisdom of the Perl Monks concerning the following question:

Hello,
I am a new perl programmer and dont have much exprience with it. What I need to login to web site download couple of files. I checked out the documentation on LWP under "Access to protected Documents" but I could not get it to work. I would really appreciate it if some can show me a code snippet that will do described above. All I need is to login to the site. BTW, once I logged in, I need to go to the other pages in the site to get the documents. How would I keep the connection alive? Do I need to login for each download? How would I send cookies to the servet?

Thanks in advance,
Mike

Replies are listed 'Best First'.
Re: Access to protected Sites
by chromatic (Archbishop) on May 14, 2001 at 03:32 UTC
    You must have looked in perldoc lwpcook, which is very good. That is further than most people get.

    When you say, "I could not get it to work", what do you mean? The best code snippet I could provide would be straight out of the documentation you read. Perhaps the site doesn't use basic authentication? The more information you provide (errors, warnings), the easier it is to help you.

    The same document has a suggestion for retrieving multiple pages from a site. (HTTP is generally a connectionless protocol -- at least, for the task you're doing, so there's no need to keep a connection alive.)

    To use cookies, have a look at HTTP::Cookies and LWP::UserAgent. They have good examples in there. You might also look in Super Search for previous questions on the same topic... someone brought this up a few months back.

    Update: Fixed a typo, thanks to dws.

[reply]
Re: Access to protected Sites
by Daddio (Chaplain) on May 14, 2001 at 04:05 UTC

    In the Q&A section of the Monastery, there is a question very similar: How can I login to get data from a webpage?. That may be a good place to start. I provided a number of links there to modules and other resources and questions.

    Also, if you post the code which you tried but could not get to work, it would give us a starting point from which to start helping you.

    D a d d i o

[reply]
Re: Access to protected Sites
by bobione (Pilgrim) on May 14, 2001 at 03:01 UTC
    I am not completly sure of me but I think that login on website (server protection) can be easely bypass. You just have to add login and pass on the URL.
    ex: http://mylogin:mypass@www.domain.com
    I will be happy if someone confirm my post.

    BoBiOne KenoBi ;)

[reply]
      That method will only work with certain types of .htaccess restriction. If plain forms are used (with CGIs), the data is send in different ways...
      Like noted before, there are loads of tutorials on using LWP to log into CGI-based forms.

      Greetz
      Beatnik
      ... Quidquid perl dictum sit, altum viditur.
[reply]
      I just tried that but it does not work. Any more ideas?

      Thanks,
      Mike
[reply]
Re: Access to protected Sites
by cajun (Chaplain) on May 14, 2001 at 12:21 UTC
[reply]

Back to Seekers of Perl Wisdom