Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: question about LWP and mysterious skipping over of pages

by ikegami (Patriarch)
on Aug 24, 2004 at 21:18 UTC ( [id://385511]=note: print w/replies, xml ) Need Help??


in reply to question about LWP and mysterious skipping over of pages

Do you see a line "Player\tSport\tSeason\t..." in STATS2 for the skipped loop? If not, that means @sports doesn't get populated as you expect. Either cause the page you downloaded wasn't as you expected it, or because it wasn't downloaded at all. Try checking if $page is defined (and turning on warnings). If that's the problem, you could retry a download that failed.

Replies are listed 'Best First'.
Re^2: question about LWP and mysterious skipping over of pages
by dchandler (Sexton) on Aug 24, 2004 at 21:45 UTC
    I was thinking something along those lines as well. If it were the case that the page wasn't coming in okay, would something like: unless ($page) { redo;} to just repeat the loop be good?
      yup, but you may want to limit the number of times you retry, and maybe pause a second before retrying.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://385511]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2024-03-29 09:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found