Spidering websites is a difficult and complex task. It is also a problem that has been solved many times before. My suggestion would be to do some research to find and then re-use or extend a previously written spider. e.g. MOMspider

Also check out the SearchTools page on Robots and Spiders for inspiration and links. The Indexing Robot Crawlers Checklist will be useful if you decide to write your own code.