LWP:Simple too slow

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I'm using LWP:Simple to retrieve a small xml file (<2kb) from a web server that requires authentication, but it's unbelievably slow.... it takes about 25 seconds!

use LWP::Simple;
$URL='http://'.USER.':'.$PASSWORD.'@'.$IPADD.':'.$PORT.'/cgi-bin/status.xml';
$myxml = get($URL);

I can do the equivalent in PHP using file_get_contents(), which takes a blink of the eye. Any ideas why LWP:Simple is being so slow?

Comment on LWP:Simple too slow

Replies are listed 'Best First'.
Re: LWP:Simple too slow by Corion (Patriarch) on Sep 16, 2009 at 06:55 UTC
Usually, LWP is not slow. Try looking at what LWP sends and where it hangs, using LWP::Debug (resp. the debugging callbacks outlined there. There you should see what is taking your script so long.	[reply]
Re: LWP:Simple too slow by whereiskurt (Friar) on Sep 16, 2009 at 11:43 UTC
Anon: In my experience spoofing the wget user agent seems to help. Some sites reduce the quality of service for different User Agents. Maybe try: `$ua->agent('Mozilla/4.0 (compatible; MSIE 5.0; Windows 95)');` [download] It could be your issue is at the protocol/server level. KPH	[reply] [d/l]