There is a good deal of information online and some interesting examples at O'Reilly from two books I found very helpful "Spidering Hacks" and "Perl & LWP". They have saved me a great deal of time.