Re: Re: LWP::Simple Seems to Hang on get

Sorry, I was unclear. The program has never finished execution and only stops when I break execution. The main page is local, and from that I extract a list of IDs. The while loop takes each ID and then calls each subroutine with the ID. Each subroutine loads a separate page from Yahoo with the ID in the URL.

I updated the code to use the real Yahoo index page instead of my local copy. The local file never changed while I was running the script and contains over 500 rows.

I hope this clarifies things.

Comment on Re: Re: LWP::Simple Seems to Hang on get

Replies are listed 'Best First'.
Re: LWP::Simple Seems to Hang on get by Abigail-II (Bishop) on Nov 06, 2003 at 23:08 UTC
Perhaps you are impatient? The page contains 680 players, and you sleep for 15 seconds for each player. So, even if the fetching and scanning takes no time at all, it's still going to take 2 hours and 50 minutes for your program to finish. I took the sleep out, and your program seems to be running fine. Abigail	[reply]
Re: Re: LWP::Simple Seems to Hang on get by svsingh (Priest) on Nov 07, 2003 at 00:01 UTC
I realize that the script will take a while to run and that's why I added the `print STDERR` statement. Thinking the problem was that maybe those statements weren't printing immediately, I set `$\|=1`. It doesn't seem to be helping. I'm rerunning the code and it's stuck again, this time at the second iteration. Anyway, thanks for your help. I'm now using LWP::Simple::getstore to save the files locally. I hope that by running the script against local files, I'll be able to bypass whatever's holding up the get calls.	[reply] [d/l] [select]