downloading a text file from http using LWP

asr055 has asked for the wisdom of the Perl Monks concerning the following question:

i tried to download a text file (the file does not have an extension in the url) using LWP::Simple mirror method and saved the output file locally on my machine. when i accessed the downloaded local file, the content of it has html tags and if statments etc. how do i convert this back into the text file that i see in teh web browser?

the content of the file when viewed in web browser is:

/vobs/arm_development_suite/.@@/main/5
[download]

what i see in the downloaded file is:

<HTML>
<!-- File: muserreg/login1.html -->
<!-- Action: ll -->
<HEAD>
<TITLE>Compass - Login Required</TITLE>
<SCRIPT LANGUAGE="JavaScript">
function SubmitCheck( obj )
[download]

any help is much appreciated. thanks

CODE tags added by Arunbear

Comment on downloading a text file from http using LWP Select or Download Code

Replies are listed 'Best First'.
Re: downloading a text file from http using LWP by davidrw (Prior) on Sep 13, 2005 at 22:27 UTC
Since the content you've gotten from LWP::Simple is a "Login Required" message, you either did not retrieve the right url or you need to authenticate (login in first to get session/cookie??) in some way or another. WWW::Mechanize (which uses LWP) can be a very easy way to do this .. Basically your goal is to mimic the steps you would take with IE or Firefox, so keep in mind anything you do there you have to have perl do too..	[reply]
Re: downloading a text file from http using LWP by InfiniteSilence (Curate) on Sep 13, 2005 at 21:33 UTC
You might try to look up HTML::TokeParser::Simple on CPAN to remove the HTML from the file. Also, try and use <code> tags around your code in posts. Celebrate Intellectual Diversity	[reply]