HTTP Headers Problem

gr0f has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to write a script to automatically upload reports in the form of XML files that I generate at work. The website uses form based file upload. Unfortunately, I've been having quite a few problems with it. Here is the code I'm using:

    $request = POST ('http://www.environet.gov.on.ca/dwws-app/DWWSApp?
+cmd=UploadSubmPage.uploadSubm',
                     Content_Type => 'form-data', Content =>
                            [
                                works_id=>'300002167',
                                lab_id=>'48',
                                filename=>['260015626-220006440.txt']
                            ]
                    );

    print OutFile $request->as_string;
    
    $response = $ua->request($request, 'output2.html');
[download]

and the header it generates:


POST http://www.environet.gov.on.ca/dwws-app/DWWSApp?cmd=UploadSubmPag
+e.uploadSubm
Content-Length: 3059
Content-Type: multipart/form-data; boundary=xYzZY

--xYzZY

Content-Disposition: form-data; name="works_id"



300002167

--xYzZY

Content-Disposition: form-data; name="lab_id"



48

--xYzZY

Content-Disposition: form-data; name="filename"; filename="260015626-2
+20006440.txt"

Content-Length: 2769

Content-Type: text/plain
[download]

This seems just dandy to me, however the server spits back an error that says 'Malformed line after disposition: Content-Length 2769'. It would appear that it doesn't like the next line. Why I'm not sure... Any suggestions on a course of action? Thanks, Tim.

Comment on HTTP Headers Problem Select or Download Code

Replies are listed 'Best First'.
Re: HTTP Headers Problem by vladb (Vicar) on Jul 11, 2002 at 15:16 UTC
The HTTP headers don't look right to me actually.. Shouldn't it be as follows: `POST /dwws-app/DWWSApp?cmd=UploadSubmPage.uploadSubm HOST: www.environet.gov.on.ca Content-Length: 3059 * * * THE REST * * ` [download] I know that some servers may also require HTTP version included with the POST header, like so: `POST /dwws-app/DWWSApp?cmd=UploadSubmPage.uploadSubm HTTP/1.1 * * * * ` [download] Update:* you can get more info on HTTP protocol at the w3.org site. On my home PC (Win2000) I have a great HTTP dump tool which allows you to have a closer look at HTTP traffic. You could get a free Win32 TCPDUMP utility here. This is the one I use. _____________________ `# Under Construction` [download]	[reply] [d/l] [select]
Re: Re: HTTP Headers Problem by gr0f (Acolyte) on Jul 11, 2002 at 15:36 UTC
Hmm, well, I'm not an expert at this so you may be right. I posted this problem several months ago and one guy confirmed that the headers looked ok. At any rate, it just seems to be the last line it doesn't like. Unless of course, if you are correct about the header, it's spawning other problems as well. Does anyone know how I might be able to get a hold of a sample header produced by a web browser such as IE or Mozilla? They don't have a problem with the site.	[reply]
View browser's headers by BorgCopyeditor (Friar) on Jul 11, 2002 at 17:56 UTC
You can get a view of the headers a browser is sending by pointing your form to: `http://www.delorie.com:81/some/url.html` The site also has a utility for viewing server headers. HTH, BCE --Your punctuation skills are insufficient!	[reply] [d/l]
Re: View browser's headers by mikeirw (Pilgrim) on Jul 11, 2002 at 18:28 UTC
Re: Re: View browser's headers by gr0f (Acolyte) on Jul 11, 2002 at 19:04 UTC
Re: HTTP Headers Problem by hossman (Prior) on Jul 11, 2002 at 22:32 UTC
Discounting issues with the "Host" header (which in my experience, a lot of webservers will let you slide on provided you specify a fully qualified URL in the "POST/GET" line) you have pointed out something very interesting... The HTTP::Request::Common documentation provides a direct example of using the POST method to create a multipart request that includes a local file upload -- as of perl5.6.1 this example does not show the Content-Length header within the file piece of the multipart message. To my knowledge, no RFC relating to MIME Multipart Messages forbids the use of a Content-Length header within a single sub-part ... it's just redundent. Bottom line: I don't know why the POST method is adding that header, but it shouldn't be causing your www server / CGI to complain. Do you know what is parsing the request on the server end? Update: After looking over the docs some more, and skimming the code for the POST method, it looks like you might be able to workarround this problem by setting `$HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1;` prior to calling POST. This is designed to make the HTTP::Request delay in reading the file untill it accutally needs to send it across the wire -- either because you don't want to load the whole thing in memory when you call POST, or because it's an "inifinte" file like "/dev/audio". In any case, using it causes the Request to be devoid of all Content-Length headers (which could cause other problems). Another thing to consider: you could try building up the content of your request using something like MIME::Lite and then set it in your request, ala...`$request->content($msg->as_string)`	[reply] [d/l] [select]
Re: Re: HTTP Headers Problem by gr0f (Acolyte) on Jul 12, 2002 at 14:46 UTC
Success! Setting $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1 eliminated the problem. My thanks for your help hossman, you don't know how much easier you've made my life: No more manual file uploading :) Peace.	[reply]


good chemistry is complicated, and a little bit messy -LW
	PerlMonks