Retrieving Data from URL

arunhorne has asked for the wisdom of the Perl Monks concerning the following question:

Monks,

I have a URL http://ca.expasy.org/cgi-bin/get-sprot-fasta?Q9PKH8 that results in text/plain output. I want to store this to a file. How can I go about doing this using Perl... the URL will be generated many times with different parameters hence need for automation.

Thanks, Arun

Comment on Retrieving Data from URL Download Code

Replies are listed 'Best First'.
Re: Retrieving Data from URL by lshatzer (Friar) on May 13, 2002 at 14:54 UTC
Check out LWP::Simple, it has a nice function `getstore`, where it will store the results in a file.	[reply]
•Re: Retrieving Data from URL by merlyn (Sage) on May 13, 2002 at 14:55 UTC
LWP is your friend. Use LWP. Install it, then `perldoc lwpcook` for many examples of usage. -- Randal L. Schwartz, Perl hacker	[reply]
Incase You Need to Use a Proxy with LWP by arunhorne (Pilgrim) on May 13, 2002 at 15:40 UTC
Thanks, LWP worked brilliantly. Just as an aside for anyone looking at this thread in the future, I am behind a firewall and so had to add the following code to make it all work. `use LWP::Simple qw(&getstore &is_success $ua);` The above imports the $ua variable and the subs I needed, $ua (User Agent, see LWP::UserAgent) is used to set proxy settings. `# Initialise the proxy $ua->proxy(http => 'http://wwwcache.ex.ac.uk:8080'); #$ua->proxy_authorization_basic( 'username', 'password' );` [download] The above actually sets the proxy. I didn't need any proxy auth, but you might, hence the line is there but commented out. Once this is done the subs like `getstore` worked great. Hope this helps in the future. Best wishes and thanks again, Arun	[reply] [d/l] [select]