Re^2: Best way to recursively grab a website

Yeah, sorry. I added the background of the problem, just so people who are familiar with Subversion, understand where I am coming from.

I have looked at LWP::Simple, but the mirror function looks like it is per-url, not per website?

I could use wget, but I want to do it all in perl, in case wget is not available.

Thanks.

Walking the road to enlightenment... I found a penguin and a camel on the way..... Fancy a yourname@perl.me.uk? Just ask!!!

Comment on Re^2: Best way to recursively grab a website

Replies are listed 'Best First'.
Re^3: Best way to recursively grab a website by inman (Curate) on Mar 29, 2005 at 11:04 UTC
LWP ships with a number of example applications / utilities such as GET, POST etc. One of these utilities is lwp-mirror. From the documentation: This program can be used to mirror a document from a WWW server. The document is only transfered if the remote copy is newer than the local copy. If the local copy is newer nothing happens.	[reply]
Re: Best way to recursively grab a website by b10m (Vicar) on Mar 29, 2005 at 12:26 UTC
Or lwp-rget This program will retrieve a document and store it in a local file. It will follow any links found in the document and store these documents as well, patching links so that they refer to these local copies. This process continues until there are no more unvisited links or the process is stopped by the one or more of the limits which can be controlled by the command line arguments. -- b10m All code is usually tested, but rarely trusted.	[reply]