marto9 has asked for the wisdom of the Perl Monks concerning the following question:

Hi everyone. My question is simple. How to get the content of a website with htaccess? This is what I have for a website without htaccess. 
use LWP::UserAgent;
use HTTP::Request;
$ua = new LWP::UserAgent;
$ua->timeout(5);
$ua->agent("Mozilla/6.0");
$url = "http://www.website.com";
$req = HTTP::Request->new('GET',$url);
$res = $ua->request($req);
$content = $res->content;

[download]

Replies are listed 'Best First'.
Re: Get content from site with htaccess
by kyle (Abbot) on Aug 11, 2008 at 19:41 UTC
[reply]
[d/l]

      Damnit, kyle! Slow down. You're making me look bad today.

[reply]
        WOW
[reply]
Re: Get content from site with htaccess
by Your Mother (Archbishop) on Aug 11, 2008 at 19:42 UTC

    ikegami's right that .htaccess doesn't really enter into it but I'll take a guess that you're talking about Basic Authentication set within an .htaccess file. If so, you probably want credentials.

    (Update: slower than kyle and reading less well too, the one I provided is from WWW::Mechanize; and though I recommend it, you asked about LWP::UserAgent instead.)

[reply]
[reply]
[d/l]
Re: Get content from site with htaccess
by ikegami (Patriarch) on Aug 11, 2008 at 19:37 UTC
    The presence of .htaccess doesn't prevent the above from working. What specific setting are you trying to work around?
[reply]
Re: Get content from site with htaccess
by marto9 (Beadle) on Aug 11, 2008 at 19:43 UTC
    To clearify. I'd like to get the content from a website that is secured with htaccess. So you need to give up a username and password. edit: Thx everyone. Kinda dumb from me to not look up that page.
[reply]

      To clarify though, "The presence of .htaccess doesn't" mean the site is password protected. You can also get BasicAuth to function from the main apache configs (it doesn't need to occur in an .htaccess file at all). Also, BasicAuth isn't unique to apache and .htaccess is.

      Also also: I wouldn't say it's dumb at all, no. It's hard to know how to ask the question until you know how to ask the question.

      -Paul

[reply]

Back to Seekers of Perl Wisdom