LWP:: Speed

carterniall has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks
Using LWP::UserAgent I am doing a 'get' to return the contents of some online KML files.
Trouble is that the $ua->get() can take up to 20secs for each get and I am doing around seven of them. This means that my whole process runs fairly slow.
Is there a faster method of returning the contents of a URL?
Many thanks,
Niall

Comment on LWP:: Speed

Replies are listed 'Best First'.
Re: LWP:: Speed by BrowserUk (Patriarch) on Jul 25, 2008 at 08:29 UTC
Depending upon your bandwidth, that of the server and the servers performance, you may be able to cut your overall time by running the 7 downloads concurrently. See LWP::Parallel Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re: LWP:: Speed by pjotrik (Friar) on Jul 25, 2008 at 08:15 UTC
There's nothing Perl can do about the speed of a web server you don't control :-) Neither can you fasten the network speed - you might experiment with compressed communication, but I don't expect that to be of any help. But forking and downloading multiple files at once will help a lot.	[reply]
Re^2: LWP:: Speed by Fletch (Bishop) on Jul 25, 2008 at 12:05 UTC
Unless of course the problem is that the other side's CPU bound and you've now multiplied the demand it's under . . . :) The cake is a lie. The cake is a lie. The cake is a lie.	[reply]
Re: LWP:: Speed by poolpi (Hermit) on Jul 25, 2008 at 14:35 UTC
Can you access some KMZ files because they're zipped? KML hth, PooLpi 'Ebry haffa hoe hab im tik a bush'. Jamaican proverb	[reply] [d/l]