in reply to Re: Infinite loop prevention for spider
in thread Infinite loop prevention for spider
The solution, then, is to start spidering with a large inventory of items (ie, a shovel, perhaps some miscellaneous treasure). As you spider each page, drop one of your inventory items into that page. Then when you visit a page again, you can tell which one it is by which inventory item is there.
Oh, and make sure your spider has a lantern, or else it is likely to be eaten by a grue...
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
| A reply falls below the community's threshold of quality. You may see it by logging in. |