in reply to Re: LWP::UserAgent & memory problems
in thread LWP::UserAgent & memory problems

It doesn't happen for me, which version of LWP are you using?

$ perl -MDevel::VersionDump -MLWP -e 1
Perl version: v5.14.1 on MSWin32
Carp               -    1.26
Config             - Unknown
Devel::VersionDump -    0.02
Exporter           -    5.66
Exporter::Heavy    -    5.66
Fcntl              -    1.11
HTTP::Date         -    6.02
HTTP::Headers      -    6.00
HTTP::Message      -    6.03
HTTP::Request      -    6.00
HTTP::Response     -    6.03
HTTP::Status       -    6.03
LWP                -    6.04
LWP::MemberMixin   - Unknown
LWP::Protocol      -    6.00
LWP::UserAgent     -    6.04
Storable           -    2.30
Time::Local        -  1.2300
URI                -    1.60
URI::Escape        -    3.31
XSLoader           -    0.15
constant           -    1.21
overload           -    1.13
strict             -    1.04
vars               -    1.02
warnings           -    1.12
warnings::register -    1.02

Replies are listed 'Best First'.
Re^3: LWP::UserAgent & memory problems
by Uree (Acolyte) on Oct 24, 2012 at 08:34 UTC
    l$ perl -MDevel::VersionDump -MLWP -e 1 Perl version: v5.14.2 on linux (BREWED) Carp - 1.26 Config - Unknown Devel::VersionDump - 0.02 Exporter - 5.67 Exporter::Heavy - 5.67 Fcntl - 1.11 HTTP::Date - 6.02 HTTP::Headers - 6.05 HTTP::Message - 6.06 HTTP::Request - 6.00 HTTP::Response - 6.04 HTTP::Status - 6.03 LWP - 6.04 LWP::MemberMixin - Unknown LWP::Protocol - 6.00 LWP::UserAgent - 6.04 Storable - 2.39 Time::Local - 1.2300 URI - 1.60 URI::Escape - 3.31 XSLoader - 0.16 constant - 1.21 overload - 1.13 strict - 1.04 vars - 1.02 warnings - 1.12 warnings::register - 1.02

    So, you can actually execute that piece of code to dl a large file (~100MB+) and have it copied directly to a file, without using a portion of memory directly proportional to file's size?
    If that is the case, then I most definitely must be missing something. Can you spot something wrong with my code?

    PS: Didnt know 'Devel::VersionDump'; thanks for the input.

      So, you can actually execute that piece of code to dl a large file (~100MB+) and have it copied directly to a file, without using a portion of memory directly proportional to file's size?

      I did 38M/135M and , the memory never grew past 10M. For server (never past 12M) I used

       plackup -l localhost:80 -e " use Plack::App::Directory; my $app = Plack::App::Directory->new({ root => q[.] })->to_app; "

      Can you spot something wrong with my code?

      Nope, I don't see any mistakes but on linux (BREWED) makes me think maybe you're using a fudgy malloc? Kinda unlikely but I've seen folks complain about it :) I'm on win32 so you probably can't compare my Config:)

      $ perl "-V:.{0,}alloc.{0,}" d_attribute_malloc='undef'; d_malloc_good_size='undef'; d_malloc_size='undef'; d_mymalloc='undef'; i_malloc='define'; i_mallocmalloc='undef'; mallocobj='malloc.o'; mallocsrc='malloc.c'; malloctype='void *'; usemallocwrap='define'; usemymalloc='n';