in reply to Re^2: Confusing UTF-8 bug in CGI-script
in thread Confusing UTF-8 bug in CGI-script

   # Wrong, and the cause of the OP's problem. See my reply to the OP.
   binmode(STDIN, ':encoding(UTF-8)');

[download]

That's what I would've thought, too, but interestingly, it doesn't do any harm in practice (I did try it), and 

   # Necessary to encode the returned HTML.
   binmode(STDOUT, ':encoding(UTF-8)');

[download]

only seems to be required with newer versions of CGI.pm (as I mentioned). Older versions apparently did the encoding themselves before printing to STDOUT (?)

Replies are listed 'Best First'.
Re^4: Confusing UTF-8 bug in CGI-script
by ikegami (Patriarch) on Feb 01, 2011 at 19:20 UTC

    it doesn't do any harm in practice

    I don't know how you can say that after saying yourself that removing it also fixes the OP's problem.

    Update: Well, you said that removing use open fixes the issue, but I doubt you're claiming that binmoding output handles leads to a decoding error, so that leaves the binmoding of the input handle.

[reply]
[d/l]
      How can you say that? You said yourself...

      If you re-read carefully what I said, you'll see that I said the script works both as is and when I remove those use statements (except for the special case I mentioned in the correction).

[reply]
[d/l]

        ok, I follow.

        Note that even if it seems to work, that doesn't make what you say correct. Depending on how the client encodes the request, the OP's code will work. That doesn't make it right.

        I suspect one of two reasons for the differences:

        • Your client encoded the request such that the initial decoding is a no-op (e.g. it %-encodes every byte with the 8th bit set). You won't be so lucky with a different client.

        • Maybe your version of decode silently does nothing (instead of dieing) when it guesses a double-decode is being attempted. If so, that will make the OP's code work for all but unlikely inputs. But that means you're relying on decode to catch your bug.

        Update: Cleaned up. Replaced first paragraph. (It was "Gotcha.", which is ambiguous.)

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom