Re: SOLVED: HTTP-POST with IO::Socket -- Header problem

I think it is because you send one too many linefeeds in your @head and @data:

    "Content-Type: application/octet-stream",
    "",
    "", # this one is too much
[download]

Perosonally, I wonder what's wrong with using LWP::UserAgent resp. HTTP::Request::Common or WWW::Mechanize to automate a web form.

Comment on Re: SOLVED: HTTP-POST with IO::Socket -- Header problem Select or Download Code

Replies are listed 'Best First'.
Re^2: SOLVED: HTTP-POST with IO::Socket -- Header problem by Anonymous Monk on Jun 23, 2011 at 09:58 UTC
No, because when I merge the header-array with the data via `join("\r\n", @head).$data` the $data variable has no "\r\n" at the beginning, so the header-array needs to have 2 line breaks so the paket as a whole looks like this: `[...] Content-Type: multipart/form-data; boundary=".$boundary." --".$boundary." Content-Disposition: form-data; name=\"username\" [...]` [download] PS: If you had read my first post you would know that all these nice comfortable modules have the problem that they copy the file completely into RAM while uploading it. You can circumvent that by setting "$HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1", but then the speed drops drastically - from 5mb/s to about 100kb/s. And you can't set the buffersize anywhere to increase that speed. Best regards, Kay	[reply] [d/l] [select]
Re^3: SOLVED: HTTP-POST with IO::Socket -- Header problem by Corion (Patriarch) on Jun 23, 2011 at 10:13 UTC
I'm sorry that I didn't read your original post closely enough. It would seem to me that a very simple approach to testing where the bottleneck lies would be in patching/replacing the subroutine `HTTP::Request::Common::form_data` to read the data in chunks larger than 2048 bytes. Unfortunately, `->form_data` is very large and monolithic and there is no easy way to change it other than copying it into your source code and replacing it: `use HTTP::Request::Common; sub my_post_file { my $bufsize = 10_240_000; local *HTTP::Request::Common::form_data = sub { ... my $buflength = length $buf; my $n = read($fh, $buf, $bufsize, $buflength); if ($n) { $buflength += $n; unshift(@parts, ["", $fh]); } ... };` [download] If this change alone brings "enough" speedup, it might be worth to submit a patch back upstream that makes the POST buffer size configurable.	[reply] [d/l] [select]