I've got a program here, which submits 'n' megabytes of data into a cgi, via POST, using the UserAgent mechanisms. The core memory usage of perl appears to be about eight times greater than that directly attributable to code outside of the UserAgent module, which seems to represent an excessive degree of string copying within the module.
Has anybody any ideas about why this much inefficiency appears to exist inside of the function call? Is this something about should be looked into, or am I missing something obvious here?
#!/usr/bin/perl # Includes. use HTTP::Request::Common; use LWP::UserAgent; use HTTP::Cookies; # Changable Parameters $size=32; $WebHost='1.2.3.4'; $FirstStage="http://$WebHost/wibble/flip/pants.cgi"; $cookie_jar=HTTP::Cookies->new( file => "./cookies"); # Code print STDERR "Creating user agent..."; $ua=LWP::UserAgent->new; print STDERR "done\n"; print STDERR "Creating cookie jar..."; $ua->cookie_jar($cookie_jar); print STDERR "done\n"; %myhash = [ username => "" ]; print STDERR "making string...\n"; open(zero, "/dev/zero") || die "/dev/zero failed to open: $!"; sysread(zero, $myhash{username}, $size*1024*1024); # using sysread to remove i/o buffers from our problem space. Solaris + will honour our request in full. YMMV. close(zero); print STDERR "username now "; print length($myhash{username}), " bytes\n"; print STDERR "About to post form data..."; $response=$ua->request(POST $FirstStage, \%myhash); print STDERR "done\n";
With size=32, I get a 228M (RSS 193M) core image.
With size=64, I get a 452M (RSS 333M) core image.
Any comments or ideas would be much appreciated.
Cheers, Mike.
In reply to Excessive UserAgent/Request memory usage. by fysh
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |