amasidlover has asked for the wisdom of the Perl Monks concerning the following question:

I'm running Perl 5.20.2 on a Gentoo virtual machine. We've had some code in place that uses a mod_perl handler to send complex hashref based structures between two servers using nfreeze / thaw; the code needed to be high performance therefore we used nfreeze/thaw rather than XML or similar. This has been working correctly on a variety of architectures since July 2014.

The mod_perl server does:

print nfreeze( $message_hash )

The client connects to the server use LWP::UserAgent and does:

my $message = thaw( $response->decoded_content() );

The message from client to server is similarly encoded with nfreeze / thaw in a cgi parameter in multi-part-form-data.

This has all worked perfectly up until I upgraded to Apache 2.4 / mod_perl 2.0.10 - I've even downgraded just those two packages to mod_perl 2.0.8 / Apache 2.2.31 and it starts working again. The message from the client gets to the server correctly and the server logs and processes the message correctly, but it then fails with an 'Out of memory!' error from the client when the response is received (due to this bug in Storable.pm) - however, regardless of that bug, something is corrupting the data in transit...

I've picked apart what's going on and found there are an additional two bytes being inserted somehow. I've recorded the data in a /tmp/ file before the print statement in the mod_perl handler and I've used perl -d to inspect what's received:

  DB<3> say "0x$_" for unpack "(H2)*", $response->content()
0x05
0x0a
0x03
0x00
0x00
0x00
0x01
0x08
0xc2
0x81
0x00
0x00
0x00
0x02
0x6f
0x6b

However, the server sent:

0x05
0x0a
0x03
0x00
0x00
0x00
0x01
0x08
0x00
0x00
0x00
0x02
0x6f
0x6b

So somehow 2 bytes have appeared in the middle of the message 0x08 and 0xc2.

So, what's going on here - is this a mod_perl bug, an Apache bug or correct behaviour and I need to work around it?

  • Comment on Strange issue with mod_perl 2.0.10 / Apache 2.4 corrupting nfreeze data

Replies are listed 'Best First'.
Re: Strange issue with mod_perl 2.0.10 / Apache 2.4 corrupting nfreeze data
by Corion (Patriarch) on Mar 28, 2016 at 11:43 UTC

    Maybe (maybe) switching to ->decoded_content works, or maybe you need to add appropriate Content-Type headers (like x-application/binary) so that you receive the content as unmolested octets. C2 81 interpreted as UTF 8 would denote an unsupported/invalid Unicode character. Maybe something encodes some control character to that.

[reply]
[d/l]
[select]

      I should have mentioned; I've tried both decoded_content and content and they both return the same.

      It would seem very strange for something to be embedding control characters mid response, but I haven't ruled it out.

      I'll see what other suggestions come up (as well as your Content-Type headers) and then roll forwards again as the roll forwards requires config changes etc. as well so takes some time.

[reply]
        I've tried adding Content-Type => 'x-application/binary' and it made no difference. I also wondered if a content-length header would help, but it made no difference either. I think I'll report a mod_perl bug and go from there.
[reply]
Re: Strange issue with mod_perl 2.0.10 / Apache 2.4 corrupting nfreeze data (json)
by tye (Sage) on Mar 28, 2016 at 16:17 UTC
    the code needed to be high performance therefore we used nfreeze/thaw rather than XML or similar

    FYI, in my tests, JSON::XS is faster not slower.

    - tye        

[reply]
      That's useful to know; I'll do some testing and see if that's the case for our usage.
[reply]

Back to Seekers of Perl Wisdom