in reply to Re: Site Crawler
in thread Site Crawler

I am indeed just crawling my own site. I thought I made that pretty clear, but I guess I could been clearer. I am aware of the issues you raised.

Replies are listed 'Best First'.
Re: Re: Re: Site Crawler
by Elian (Parson) on Jun 13, 2002 at 23:00 UTC
    Yup, it was unclear. (I'm presuming you've got a lot of dynamic content, otherwise crawling this way's kinda pointless)
[reply]

In Section Seekers of Perl Wisdom