in reply to Re^2: Script Timeout Settings?
in thread Script Timeout Settings?

It sounds like you want to use LWP to grab these webpages ( assuming they are viewable via a browser) and run your analysis routines that way. Otherwise you would need to get a copy of the file you want to analyze, or read it off the server somehow, and analyze it that way.Net::FTP comes to mind, but I would say its poor practice to have your webpages setup in an FTP folder (just personal opinion there).

Anyhow, sounds like a super fun project and I hope you find your answers.
Grygonos

Replies are listed 'Best First'.
Re^4: Script Timeout Settings?
by ChrisJ-UK (Novice) on Jul 26, 2004 at 16:50 UTC

    Thanks for the mention of LWP.

    I'm currently using the Lynx browser to grab the page before analysing it. Would I gain any advantage from LWP?

    I appreciate the need to be aware of robots.txt but the crawler only goes over one site and they have OK'ed the crawling of it. I wasn't aware of a specific module for this though so I'll look into it if I expand this project or begin a new one.

    Thanks again everyone, this really is a great place.

    Chris.

[reply]

In Section Seekers of Perl Wisdom