Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I know I just saw something like this recently, but I cannot for the life of me find it. I need a tool that will create a static view of a dynamic site (like a snapshot). The goal is to be able to take this snapshot of a site and then be able to display it on a laptop without an internet connection. This tool I saw would somehow create plain HTML files from dynamically generated pages. Does anyone know where I can find a tool like this?

Replies are listed 'Best First'.
Re: static view of dynamic site
by suaveant (Parson) on May 11, 2001 at 22:26 UTC
    What you are looking for, I believe, is something to spider a site similar to perl monks or slashdot and store it as it is currently in flat files? You should look into LWP or even the wget command to download the pages of a site recursively (spider). Of course, this doesn't work with forms and such...

    There was a thread on something similar to this a while ago... here
                    - Ant

[reply]
Re: static view of dynamic site
by Beatnik (Parson) on May 11, 2001 at 22:37 UTC
[reply]
Re: static view of dynamic site
by jepri (Parson) on May 11, 2001 at 22:21 UTC
    wget

    ____________________
    Jeremy
    I didn't believe in evil until I dated it.

[reply]
      Could you expand on that a little bit?
[reply]
        yes

        ____________________
        Jeremy
        I didn't believe in evil until I dated it.

[reply]
Re: static view of dynamic site
by mr.nick (Chaplain) on May 12, 2001 at 01:08 UTC
    offtopic

    Just a mild comment: recently I rewrote the software that generates my homepage (http://www.mrnick.binary9.net) and encountered a situation where I needed to take snapshots of my own pages.

    The system works by a file-system based method. In the root directory of my page lives the .cgi. Each "section" of my homepage is a directory under that root with specially named files; like "blurb.html", or "title.txt" or "blurb.cgi". The main cgi scans the directory and does stuff with it. Those directories can also have subdirectories which can, of course, have more.

    I found that doing all the scanning, reading of files and such was taking more and more time to execute. I also realized that there is no reason to regenerate the page if nothing has changed.

    So I wrote some simple snapshot functions that compared the date of all the files in the directory with the snapshot's date. If anything had changed, it would regen the snapshot, if not, then just pump it back at the user.

    I thought it interesting. Go ahead and take a look at it; I've already taken up enough space here.

    The code. Look for chk_snapshot() and mk_snapshot().

[reply]
Re: static view of dynamic site
by Banky (Acolyte) on May 12, 2001 at 22:04 UTC
    Make sure the software you write or use contains sleeps so you aren't inundating the site with requests.
[reply]

Back to Seekers of Perl Wisdom