in reply to Re: :utf8 I/O layer vs encoding(UTF8), segfault and speed
in thread :utf8 I/O layer vs encoding(UTF8), segfault and speed

ok, I get that but where do the warnings/errors come from?

What I mean is are the errors separate from encoding problems

  • Comment on Re^2: :utf8 I/O layer vs encoding(UTF8), segfault and speed

Replies are listed 'Best First'.
Re^3: :utf8 I/O layer vs encoding(UTF8), segfault and speed
by almut (Canon) on Apr 01, 2009 at 19:22 UTC
    What I mean is are the errors separate from encoding problems

    I only see two types of errors: "utf8 "\x.." does not map to Unicode", and "Malformed UTF-8 character (...details...)", both of which indicate encoding problems due to malformed input.

      I'm obviously missing something. I know the input is not correctly utf8 encoded. If :utf8 I/O layer only sets the internal utf8 flag and does not check the encoding then why am I getting encoding errors?

        Just because the :utf8 I/O layer only sets the flag doesn't mean subsequent operations were disabled from spitting out their regular warnings when encountering junk data...