using variables in lwp::simple

diomedea has asked for the wisdom of the Perl Monks concerning the following question:

i have an issue in lwp::simple that im trying to understand, i have read the cpan and googled for similar problems but have reached a dead end.... if i do this: (use strict and -wT on shebang line btw)

my $status = getstore("http://www.myurl.coml?data" , "file.txt");
[download]

I get the file i expect back in its entirety, however if i do this

my $uri = "http://www.myurl.coml?data";
my $file = "file.txt";
my $status = getstore($uri, $file);
[download]

I only get the first few lines of the file back. Im i making some simple schoolboy error? If not, is this likely to be an issue with lwp or more likely a server side issue (should I use user agent to get more details?) your advice would be appreciated!

Comment on using variables in lwp::simple Select or Download Code

Replies are listed 'Best First'.
Re: using variables in lwp::simple by elsiddik (Scribe) on Sep 25, 2007 at 11:47 UTC
`#!/usr/bin/perl use strict; use warnings; my $url = "http://www.myurl.coml?data"; my $file = "file.txt"; # get the data use LWP::Simple; is_success(getstore($url, $file)) or die "cant get the data\n";` [download] i dont know if its the right thing to do -- im also still learning, The world is a dangerous place. Not because of the people who are evil; but because of the people who don't do anything about it --Albert Einstein	[reply] [d/l]
Re: using variables in lwp::simple by Gangabass (Vicar) on Sep 25, 2007 at 12:15 UTC
You can try to install HTTP::Recorder to see whats happens with your request.	[reply]
Re: using variables in lwp::simple by graff (Chancellor) on Sep 26, 2007 at 04:33 UTC
I tried to replicate the behavior you reported with a couple simple one-liners, and a url that seemed similar enough in nature to your made-up example: `perl -MLWP::Simple -e 'getstore("http://www.perlmonks.org/index.pl?nod +e_id=640887","test1.html")' perl -MLWP::Simple -e '($u,$f)=("http://www.perlmonks.org/index.pl?nod +e_id=640887","test2.html"); getstore($u,$f)'` [download] But I got the full page each time -- no problem. If you try it that way and also get full data each time (no problem), then I would expect there to be some other, seemingly unrelated difference between your two earlier script versions. (I can't imagine what sort of server-side problem could be involved, except some bizarre coincidence.)	[reply] [d/l]
Re: using variables in lwp::simple by diomedea (Sexton) on Sep 26, 2007 at 09:19 UTC
hi monks, many thanks for you advice, I seem to have solved the problem by USEing URI and constructing the query that way instead of concatenating the url string. for info, I got this to work; `use Strict; use warnings; use URI; my $infile = "search_str"; my $savedfile; my $url = URI->new('http://www.mysite.com/cgi_bin/getthis'); $url->query($infile); my $status = getstore($url, $savedfile);` [download] it is based on the helpful o'reilley perl and lwp book which I have just stumbled across!! ;+) thanks	[reply] [d/l]