cris.tech has asked for the wisdom of the Perl Monks concerning the following question:

Got a nice WWW::Mechanize script, pulling data from a jsp website. Everything seems to work just fine until I post to a specific page... The returned server header doesn't set the $mech->response->header('Content-Length') and when I print the $mech->content(), it stops at 8192 characters.

I've tried setting $OPTS{MaxLineLength}, but that doesn't seem to help... Any suggestions?

Thx

Replies are listed 'Best First'.
Re: WWW::Mechanize and content length
by ww (Archbishop) on May 31, 2013 at 17:39 UTC
    Improve (update) your question with the smallest snippet of code that demonstrates the problem; errors and/or warnings, if any; and the traffic to & from the one specific page where the failure occurs ( I gather that's what you're saying. If I err, please clarify).

    If you didn't program your executable by toggling in binary, it wasn't really programming!

Re: WWW::Mechanize and content length
by karlgoethebier (Abbot) on May 31, 2013 at 18:16 UTC
    «Got a nice WWW::Mechanize script...»

    Ok, yet another answer containing NULL code - not good, but: i don't know anything about WWW::Mechanize.

    And i don't know anything about that «nice script».

    Perhaps it would be better you post that «nice script»...?

    Regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

Re: WWW::Mechanize and content length
by cris.tech (Initiate) on May 31, 2013 at 18:58 UTC
    here's some code
    #!/usr/bin/perl use CSV; use WWW::Mechanize; my %OPTS = @LWP::Protocol::http::EXTRA_SOCK_OPTS; $OPTS{MaxLineLength} = 16384; # Or however large is needed... @LWP::Protocol::http::EXTRA_SOCK_OPTS = %OPTS; my $infile = $ARGV[0]; if (! -r $infile) { die "Can't read input $infile\n"; } # Start Mech and get it ready my $startPage = "https://xxxx.org/account/logon.jsp"; my $mech = WWW::Mechanize->new( cookie_jar => {},activate => 0,autoclo +se => 1,autodie =>0 ); $mech->get( $startPage ); $mech->post("https://xxxx.org//onerequest.jsp" , 'whatToDo' => "add", 'whereFrom' => "requestframe.jsp", 'whereTo' => "onerequest.jsp", ); sleep(2); print "Content length: " . $mech->response->header( 'Content-Length' ) + . "\n"; $content = $mech->content();
    At the last line, content length is blank and the $content holds the first 8192 bytes of the webpage. I can manually do the same steps within Chrome and dump the page and verify that $mech->content() only contains the first 8192 bytes of a 25k page. One thing to note, the <head> portion of the webpage contains a lot of javascript code.