overrider has asked for the wisdom of the Perl Monks concerning the following question:

Hello there, I have a Program that shall upload Files. It actually works very well with smaller Movies, but when tried for example on a 300MB File it will just crash with an Error like 
Out of memory during "large" request for 536875008 bytes, total sbrk()
+ is 7065600 bytes at /usr/local/lib/perl5/site_perl/5.8.9/HTTP/Reques
+t/Common.pm line 160.

[download]
What id like to do now is to build a callback function, so that when I POST the file, it will basically read the File chunk by chunk. That way even if i upload a 1GB Movie, Memory used stays pretty much the same. I tried to implement it like this 
my $filename = "largefile.avi";
open(FH,"<$filename");
my $upload_cb = sub {
        if ( sysread FH, my $buffer, 4096 ){
                return $buffer; 
        } else {
                close FH;
                return ''; 
        }
}
# later in the Code, when uploading the File, calling it like this
$vid_fields{field_uploadfile}             =  [ $upload_cb ] ;
$vid_fields{field_private_share_entities} = "";
$resp = upload_step_two($action_url, \%vid_fields);

[download]
When running the Program, i get a 
Can't open file CODE(0x821d6d4): No such file or directory at /usr/loc
+al/lib/perl5/site_perl/5.8.9/LWP/UserAgent.pm line 401

[download]
Can someone give me a simple Example of POSTing a large File when using LWP::Useragent? Thanks a lot

Replies are listed 'Best First'.
Re: POST'ing a large File with LWP::UserAgent
by Anonymous Monk on May 03, 2009 at 06:21 UTC
    That is not a complete program.
[reply]
      #!/usr/bin/perl --

use strict;
use warnings;

use LWP  5.825;

my $url = "http://foo/upload";
my $file = __FILE__;#"foo/bar";

my $ua = LWP::UserAgent->new;
$HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1;

use HTTP::Request::Common;
my $req = POST(
    $url,
    Content_Type => 'multipart/form-data',
    Content      => [ file => [$file] ],
);

# set up callback
{
    my $gen = $req->content();
            die unless ref($gen) eq "CODE";
    my $i = 0;
    $req->content(
        sub {
            my $chunk = &$gen(); # get chunk of data
            warn $i++;
            return $chunk;       # send it to $url
        }
    );

#use Data::Dumper;die print Data::Dumper->new([($ua,$req, $gen)])->Ind
+ent(1)->Deparse(1)->Dump;
}

my $res = $ua->request($req);   #do it 
print $res->status_line;

      [download]
[reply]
[d/l]

        Thanks for the feed back. Hi can you explain what the below code actually do

        # set up callback
{
    my $gen = $req->content();
            die unless ref($gen) eq "CODE";
    my $i = 0;
    $req->content(
        sub {
            my $chunk = &$gen(); # get chunk of data
            warn $i++;
            return $chunk;       # send it to $url
        }
    );

        [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom