HTTP Headers Problem

gr0f has asked for the wisdom of the Perl Monks concerning the following question:

I've got an interesting problem for you LWP guys out there. A certain part of the script I'm writing involves uploading a file to a website via form upload. Normally this wouldn't be that much of a problem, but it's the way that the website decided to go about it that's causing the problem. For some reason, a command to their script must be placed in the URL, and the rest through the normal POST methods for the file upload to work correctly. Here is the HTML:


<form name="UploadPage" action="www.page.com/app?cmd=upload" method="P
+OST" enctype="multipart/form-data">
[download]

This of course works fine in the bowser, but as you guessed it, doesn't work in LWP. Once executed, the website gave me an error with regards to an incorrect byte count. I can only assume that LWP is ignoring the: cmd=upload command in the URL, and not taking it into account or something. Since this does use the POST method, that's understandable. I think just adding the extra bytes to the header count would solve the problem, I'm not sure though. Either way, I'm not experienced enough in using LWP to manipulate the HTTP Headers module in such a fine way. I'd appreciate any input. Thanks.

-Tim

Comment on HTTP Headers Problem Download Code

Replies are listed 'Best First'.
Re: HTTP Headers Problem by tachyon (Chancellor) on Aug 27, 2001 at 18:05 UTC
Can you specify the website so I can test a script or post the code you are using? I presume you are using LWP::UserAgent and HTTP::Request something like this: `use LWP::UserAgent; use HTTP::Request::Common qw(POST); $ua = new LWP::UserAgent; $req = POST ( 'http://www.page.com/app?cmd=upload', Content_Type => 'form-data', Content => [ name => 'value', name => 'value', file => ["/path/to/file.txt"], ] ); $response = $ua->request($req);` [download] Update Website remains down. This code seems to check out OK. `$filename = "c:/test.pl"; use HTTP::Request::Common qw(POST); use LWP::UserAgent; my $ua = new LWP::UserAgent; my $req = POST ('http://www.environet.gov.on.ca/dwws-app/DWWSApp?cmd=U +ploadSubmPage.uploadSubm', Content_Type => 'form-data', Content => [ works_id => '2353245', lab_id => '456436', filename => [$filename] ] ); print $req->as_string; # looks OK my $res = $ua->request($req); print $res->as_string;` [download] cheers tachyon s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print	[reply] [d/l] [select]
Re: Re: HTTP Headers Problem by gr0f (Acolyte) on Aug 27, 2001 at 18:19 UTC
Sure can, refer to the above reply to tomhukins...	[reply]
Re: Re: HTTP Headers Problem by gr0f (Acolyte) on Aug 27, 2001 at 23:35 UTC
Ok, I still seem to be getting the same error, but at least this time I can see the headers. Here is a header that was generated for one of the files I uploaded: `POST http://www.environet.gov.on.ca/dwws-app/DWWSApp?cmd=UploadSubmPag +e.uploadSubm Content-Length: 11052 Content-Type: multipart/form-data; boundary=xYzZY --xYzZY Content-Disposition: form-data; name="works_id" 210000586 --xYzZY Content-Disposition: form-data; name="lab_id" 48 --xYzZY Content-Disposition: form-data; name="filename"; filename="210000586-2 +10001130.txt" Content-Length: 10761 Content-Type: text/plain` [download] And this is the error that the website sent me back: `Malformed line after disposition: Content-Length: 10761` [download] Arrrgh, I hate this site ;) It's given me nothing but problems... -Tim	[reply] [d/l] [select]
Re: HTTP Headers Problem by tomhukins (Curate) on Aug 27, 2001 at 17:40 UTC
Without seeing your code, it's difficult to guess exactly what is causing problems for you. Can you post a small piece of code that reproduces your problem? Off the top of my head, some problems you might run into are: Not encoding your data as multipart/form-data The ? character before cmd=upload being escaped The action value being specified incorrectly. Most browsers will cope with this beginning with www.page.com, but really you should use http://www.page.com Also, you might find LWP+cgi file upload problems helpful.	[reply]
Re: Re: HTTP Headers Problem by gr0f (Acolyte) on Aug 27, 2001 at 18:15 UTC
No problem. Here is the actual HTML used in the page (I had to pull the other from memory since the website seems to be down ;), and the Perl code: `<FORM METHOD=POST ACTION="http://www.environet.gov.on.ca/dwws-app/DWWS +App?cmd=UploadSubmPage.uploadSubm" ENCTYPE="multipart/form-data"> <input type="hidden" name="works_id" value="3253245"> <input type="hidden" name="lab_id" value="2435325"> <input type="file" name="filename"> </FORM>` [download] And the Perl code responsible: `$FileName = ("C:\\Scripts\\moe\\swap\\$LotNumber.txt"); my $request = POST ('http://www.environet.gov.on.ca/dwws-app/DWWSApp?c +md=UploadSubmPage.uploadSubm', Content_Type=> 'form-data', Content => [works_id=>'2353245', lab_id=>'456436', filename=>[$FileName]]);` [download] The '?' character being escaped sounds like it may be significant. Anyways, hope this helps... -Tim	[reply] [d/l] [select]

Update