I am somehow embarrassed that I didn't look into the source code myself. My excuse is that I was convinced that somebody out there already had a list showing which value SUBCHAR is under which circumstances.

In a perfect world, you shouldn't have to look into the source, everything would be explained in the documentation.

Regarding using a coderef for CHECK, I think I had got that in the meantime, but IMHO the question remains if a malformed character has an ordinal value at all (quite sure yes, but which?).

I would assume that - for converting bytes pretending to be UTF-8 to perl's internal representation - the CHECK coderef would be called with the value of the (first) offending byte. For the other way (perl to UTF-8 bytestream), I would expect to get the (first) offending perl character that can't be expressed as UTF-8 bytestream.

Let's test that:

#!/usr/bin/perl

use v5.10;
use strict;
use warnings;
use Encode 2.12 qw();

my $octets="a\xFEb"; 
#            ^-- byte 0xFE ist invalid in UTF-8, see https://en.wikipe
+dia.org/wiki/Utf-8
my $string=Encode::decode(
    'utf-8',
    $octets,
    sub {
        my $value=shift;
        return sprintf('<0x%04X>',$value);
    }
);
say $string;

$string="a\x{123456}b";
#         ^-- Unicode is defined from 0 to 0x10FFFF
$octets=Encode::encode(
    'utf-8',
    $string,
    sub {
        my $value=shift;
        return sprintf('<0x%08X>',$value);
    }
);
say $octets;

$string="a\x{00C4}b\x{263A}c"; # a A-Umlaut b Smile c
#                   ^-- not available in ISO-8859-1

$octets=Encode::encode(
    'utf-8',
    $string,
    sub {
        die "Should not happen";
    }
);

# from_to() converts bytes, not characters. To make things easier,
# I use a destination encoding where 1 byte = 1 character.
Encode::from_to(
    $octets, # in-place
    'utf-8',
    'iso-8859-1',
    sub {
        my $value=shift;
        return sprintf('<0x%04X>',$value);
    }
);

binmode STDOUT,':encoding(utf-8)'; # I use a UTF-8 terminal
say $octets; # implicit converting from ISO-8859-1 to UTF-8 due to bin
+mode above
[download]

Output:

a<0x00FE>b
a<0x00123456>b
aÄb<0x263A>c
[download]

I have asked question 2) mainly because I felt that not explaining SUBCHAR is a substantial lack of documentation and was hoping that I had overlooked something.

Yes, the documentation could be improved. If you can spend five minutes, file a bug. If you can spend an hour or two, create a patch for the POD and submit it. You know now quite well what's missing in the POD, and how it should be explained. Perhaps post a preview for discussion here.

(If you are working for a boss (and not for fun), explain him/her that this little bit of time is a kind of "usage fee" for the huge amount of well-written code you use from the perl community. That was my argument for publishing the initial Unicode patch for DBD::ODBC, and my boss was quite happy with that. We had the patch, we needed it, and by publishing it, the Unicode support became even better. And the best: I don't have to support it any more. mje has merged it into DBD::ODBC and improved it since then.)

Alexander