in reply to Re^5: Seeking Perl docs about how UTF8 flag propagates
in thread Seeking Perl docs about how UTF8 flag propagates

I would be very surprised if there where normal cases where the UTF8-Flag isn't preserved when passing around. So no need to document the obvious.

It sounds like you've confused "expected" with "obvious."

"Widget can contain any ASCII character. This includes the semicolon." That second sentence is obvious—it's an easily deducible consequence of the first—so it need not (and should not) be stated.

Conversely, you can expect things to happen a certain way, but software can sometimes defy your expectation—as hv's remark below about substr demonstrates. That doesn't mean the software is misbehaving; it just means your expectation didn't align with it. In contrast, if an ASCII-holding object fails to be able to contain a semicolon, that's beyond unexpected—that's a bug.

And frankly, I don't think there is a clear expectation about whether things like substr and split, which create brand new strings out of pieces of existing strings, should blindly copy the UTF8 flag of their input.

Ideally, "expected" behavior would still be documented. (That statement strays into tautologyland: documenting things is how users know to expect them.) If the behavior is expected but still intentionally undefined, that fact ought to be documented too, so that coders know not to rely on it. The case at hand is neither, which to me suggests a shortfall in the documentation. A sentence or two in perlunicode about what's not guaranteed regarding the UTF8 flag would solve this, and let coders ensure their code isn't making unwarranted assumptions.