... and what a spider does is that it looks at a specified page, and finds all the links on it. It then follows all of those links (at least the ones local to the site), and downloads them as well. you keep repeating this process until you have all the pages (min you have to keep track of which pages you have to prevent an infinite loop). But there are already module for that (though sometimes its nice to know what your modules are doing).
                - Ant