Thanks for the pointers, it was fun to play, but neither helps.

With Errno::AnyString I'm able to change string value of $!, but looks like it doesn't matter how I set it - to Unicode scalar or binary UTF8-encoded string - because it always fetched as binary UTF8-encoded string.

Variable::Magic doesn't actually see when $! set on errors, and even on manual set it sees (and can fake on get) only numeric value.

I'm now on perl-5.16.3 so I didn't see this in doc before, but looks like perldoc perlvar for 5.20 mention this issue: «Note that when stringified, the text is always returned as if both "use locale" and "use bytes" are in effect. This is likely to change in v5.22.»