jve has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

After searching examples/solutions on internet, I decided to post my issue here to understand and try to solve my issue.
I am writting a perl script to upload big binary files (> 30 mo) to a file server using a Web Service.
I must POST my file(s) in a multi-part format.
To do so, I am using the LWP::UserAgent, HTTP::Request and Http::Response perl libraries. I am using a callback to read the file by chunk.
To write my 'upload' script, I took help from http://www.perlmonks.org/?node_id=761560 and https://labs.consol.de/blog/misc/fileupload-with-perl-decorated-with-a-progressbar/

Here my script section that processes the POST :

sub upload { my ($url, $binary) = @_; local $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1; my $ua = LWP::UserAgent->new( timeout => 10*60 ); my $request = POST ${url}, 'Content_Type' => 'form-data', 'Content' => { 'file_to_upload' => ["$binary"] }; my $reader = &create_content_reader($request->content(), $request->header('Content_Leng +th')); $request->content($reader); my $response = $ua->request($request); print STDOUT $response->status_line, "\n"; if ($response->is_success) { return JSON::decode_json($response->decoded_content); } return undef; }
sub create_content_reader { my ($gen, $len) = @_; my $size = 0; my $next_update = 0; return sub { my $chunk = &$gen(); $size += length($chunk) if $chunk; return $chunk; } }

My issue is when the file is less than 30 Mo, my script is running like a charm but, > 30Mo, I have got a "404 Not Found" error message as response.
Why ?

Replies are listed 'Best First'.
Re: Why I can't POST big file in a multi-part format using LWP::UserAgent ?
by zentara (Cardinal) on Mar 31, 2014 at 11:31 UTC
    When using the old CGI module, there is an upload filesize limit setting. Maybe there is a default limit built-in in your Web Service? Can you post a big file by another means, say with a browser?

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
      Thanks for the help, I fixed it. I forgot to increase the limit size on the IIS server.