Re: Sending large file contents in SOAP call using SOAP::Lite

Is this the error in the SOAP::Lite module or I am using it the wrong way.

Yeah, 1.5GB seems like there might be a problem ... but not where, I doubt its within SOAP::Lite

perl -Mojo -e " b( shift )->slurp->b64_encode->say; int <>; " 49_678_352_bytesfile > 65M-b64ed

This takes about ~130mb on my machine, and around 20mb of that is perl -Mojo itself alone

I can't imagine SOAP::Lite tacking eleven times that amount, maybe one or three temporarily, but not eleven

Without mojo, with File::Slurp+SOAP::Lite, to slurp this 65M b64ed file takes about 74mb, 1500mb/74mb = 20 so still nowhere in the ballpark

I don't feel like looking into it past this; You can try

    $soap->serializer()->envelope(
        method => 'DoThis', 
        $param
    ),
[download]

to see how much memory it takes to compose that request you're using -- shouldn't be 1.5 GB

I imagine it is the response that you're getting (your program is getting) to that request is an xml bomb ... or something else entirely

Also, what is a better way to make a SOAP call with large data

I imagine maybe https://metacpan.org/module/SOAP::Lite#ATTACHMENTS,

or even XML::Compile::XOP

but if your server doesn't accept that :)

Good luck

Comment on Re: Sending large file contents in SOAP call using SOAP::Lite Download Code

Replies are listed 'Best First'.
Re^2: Sending large file contents in SOAP call using SOAP::Lite by Anonymous Monk on Oct 18, 2013 at 17:34 UTC
I tried serializer code you mentioned on a 34mb file and it almost used up 500MB while it is serializing and I printed the length of the resultant serialized text 47mb. So, the issue is not the size of the SOAP message it is sending or receiving, it is that serialization itself using up too much memory. Do you or anyone see the same issue?	[reply]
Re^3: Sending large file contents in SOAP call using SOAP::Lite by Anonymous Monk on Oct 19, 2013 at 03:59 UTC
... almost used up 500MB ... Do you or anyone see the same issue? Well 500mb/72mb string is right about 7 times Its not unheard of , that during serialization, a string gets copied seven times So I dont think this is out of the ordinary Combine that with the caching of the request/response objects (by LWP), now 1.5GB is beginning to seem a bit more expectable :) but I don't have interest to investigate further -- did I mention I hate SOAP? I do. XML::Compile::SOAP its based on XML::LibXML which is more memory efficient with its string handling, so I would look into that -- it handles the SOAP spec better than SOAP::Lite The other thing I might try is to dump the raw request to a file in similar fashion as I showed above, and then use raw LWP::UserAgent or WWW::Mechanize to POST the request from file, and SAVE the response to file, to avoid loading it all in memory Good luck I have copy/paste for you :) My soap tips (I hate soap), SOAP::Lite is too much work, SOAP::Simple is less work (but its simple, when stuck go to XML::Compile::SOAP, more verbose, but you want verbose with SOAP ) ... its built on XML::Compile::SOAP/http://perl.overmeer.net/xml-compile/#doc , see my treasure trove of soap examples and lost knowledge,$soap->transport->add_handler("request_send", \&pp_dump );, http://cookbook.soaplite.com/, SOAP endpoint , Re^3: SOAP::Lite login setup, Re: I do not understand how to write a SOAP server., An XML Overview Towards Understanding SOAP, Re^3: SOAP::Lite and custom envelopes, The XML FIles: Understanding XML Namespaces, How to Call a .NET-based Web Service Using the SOAP::Lite Perl Library (2002 )	[reply]