in reply to Re: Interactions of \Q, \U, \L and \E
in thread Interactions of \Q, \U, \L and \E

Yes, they are handled in the tokeniser. It was while reading the tokeniser code that I became confused/curious about how they operate and why.

In the case of qq(\Q\U,a) this is equivalent (converted to) quotemeta(uc(',a')), as can be seen from the following:

[ian@alula perl]$ /usr/local/perl-igannotate/bin/perl -DpT -e 'print q
+q(\Q\U,x),"\n"'
Starting parse
Entering state 0
Reducing stack by rule 5 (line 164), -> progstart
Entering state 2
Reducing stack by rule 8 (line 185), -> lineseq
Entering state 4
Reading a token: ### 0:LEX_NORMAL/XSTATE "\n;"
### <== LSTOP(ival=op_print)

Next token is token LSTOP (0xde)
Reducing stack by rule 51 (line 474), -> label
Entering state 15
Next token is token LSTOP (0xde)
Shifting token LSTOP, Entering state 62
Reading a token: ### 1:LEX_NORMAL/XREF " qq(\\Q\\U,x),\"\\n\"\n"
### <== FUNC(ival=op_stringify)

Next token is token FUNC (0x43)
Shifting token FUNC, Entering state 60
Reading a token: ### 1:LEX_INTERPPUSH/XTERM ",\"\\n\"\n"
### <== '('

Next token is token '(' (0x43)
Shifting token '(', Entering state 137
Reading a token: ### 1:LEX_INTERPCONCAT/XTERM "\\Q\\U,x"
### 1:LEX_INTERPCASEMOD/XTERM "\\Q\\U,x"
### Saw case modifier
#### pushed 1 PL_lex_casestack: Q, PL_lex_casemods: 1
### forced token:
### <== '('

### forced token:
### <== FUNC(ival=op_quotemeta)

### 1:LEX_KNOWNEXT/XTERM "\\U,x"
### <== FUNC(ival=op_quotemeta)

Next token is token FUNC (0x7d)
Shifting token FUNC, Entering state 60
Reading a token: ### 1:LEX_KNOWNEXT/XTERM "\\U,x"
### <== '('

Next token is token '(' (0x0)
Shifting token '(', Entering state 137
Reading a token: ### 1:LEX_INTERPCONCAT/XTERM "\\U,x"
### 1:LEX_INTERPCASEMOD/XTERM "\\U,x"
### Saw case modifier
#### pushed 1 PL_lex_casestack: QU, PL_lex_casemods: 2
### forced token:
### <== '('

### forced token:
### <== FUNC(ival=op_uc)

### 1:LEX_KNOWNEXT/XTERM ",x"
### <== FUNC(ival=op_uc)

Next token is token FUNC (0x7b)
Shifting token FUNC, Entering state 60
Reading a token: ### 1:LEX_KNOWNEXT/XTERM ",x"
### <== '('

Next token is token '(' (0x0)
Shifting token '(', Entering state 137
Reading a token: ### 1:LEX_INTERPCONCAT/XTERM ",x"
### forced token:
### <== THING(opval=op_const) PV(",x"\0)

### 1:LEX_KNOWNEXT/XTERM ""
### <== THING(opval=op_const) PV(",x"\0)

Next token is token THING (0xa1a3348)
Shifting token THING, Entering state 42
Reducing stack by rule 164 (line 1137), THING -> term
Entering state 89
Reading a token: ### 1:LEX_INTERPSTART/XTERM ""
### 1:LEX_INTERPCASEMOD/XTERM ""
#### popped 1 PL_lex_casestack: Q, PL_lex_casemods: 1
### <== ')'

### <== ')'

Next token is token ')' (0xa1a3348)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ')' (0xa1a3348)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 232
Next token is token ')' (0xa1a3348)
Reducing stack by rule 198 (line 1287), expr -> listexprcom
Entering state 233
Next token is token ')' (0xa1a3348)
Shifting token ')', Entering state 319
Reducing stack by rule 95 (line 752), FUNC '(' listexprcom ')' -> list
+op
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 89
Reading a token: ### 1:LEX_INTERPCASEMOD/XTERM ""
#### popped 1 PL_lex_casestack: , PL_lex_casemods: 0
### <== ')'

Next token is token ')' (0xa1a3348)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ')' (0xa1a3348)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 232
Next token is token ')' (0xa1a3348)
Reducing stack by rule 198 (line 1287), expr -> listexprcom
Entering state 233
Next token is token ')' (0xa1a3348)
Shifting token ')', Entering state 319
Reducing stack by rule 95 (line 752), FUNC '(' listexprcom ')' -> list
+op
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 89
Reading a token: ### 1:LEX_INTERPCASEMOD/XTERM ""
### 1:LEX_INTERPCONCAT/XTERM ""
### <== ')'

Next token is token ')' (0xa1a3348)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ')' (0xa1a3348)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 232
Next token is token ')' (0xa1a3348)
Reducing stack by rule 198 (line 1287), expr -> listexprcom
Entering state 233
Next token is token ')' (0xa1a3348)
Shifting token ')', Entering state 319
Reducing stack by rule 95 (line 752), FUNC '(' listexprcom ')' -> list
+op
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 89
Reading a token: ### 1:LEX_NORMAL/XOPERATOR ",\"\\n\"\n"
### <== ','

Next token is token ',' (0xa1a3348)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 141
Next token is token ',' (0xa1a3348)
Shifting token ',', Entering state 180
Reading a token: ### 1:LEX_NORMAL/XTERM "\"\\n\"\n"
### Saw string before "\n"
### <== FUNC(ival=op_stringify)

Next token is token FUNC (0x43)
Shifting token FUNC, Entering state 60
Reading a token: ### 1:LEX_INTERPPUSH/XTERM "\n"
### <== '('

Next token is token '(' (0x43)
Shifting token '(', Entering state 137
Reading a token: ### 1:LEX_INTERPCONCAT/XTERM "\\n"
### forced token:
### <== THING(opval=op_const) PV("\n"\0)

### 1:LEX_KNOWNEXT/XTERM ""
### <== THING(opval=op_const) PV("\n"\0)

Next token is token THING (0xa1a3bb0)
Shifting token THING, Entering state 42
Reducing stack by rule 164 (line 1137), THING -> term
Entering state 89
Reading a token: ### 1:LEX_INTERPSTART/XTERM ""
### <== ')'

Next token is token ')' (0xa1a3bb0)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ')' (0xa1a3bb0)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 232
Next token is token ')' (0xa1a3bb0)
Reducing stack by rule 198 (line 1287), expr -> listexprcom
Entering state 233
Next token is token ')' (0xa1a3bb0)
Shifting token ')', Entering state 319
Reducing stack by rule 95 (line 752), FUNC '(' listexprcom ')' -> list
+op
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 258
Reading a token: ### 1:LEX_NORMAL/XOPERATOR "\n"
### <== ';'

Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 86 (line 694), argexpr ',' term -> argexpr
Entering state 141
Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 196 (line 1281), argexpr -> listexpr
Entering state 142
Reducing stack by rule 94 (line 748), LSTOP listexpr -> listop
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 89
Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 81
Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 18 (line 249), expr -> sideff
Entering state 78
Next token is token ';' (0xa1a3bb0)
Shifting token ';', Entering state 170
Reducing stack by rule 16 (line 228), label sideff ';' -> line
Entering state 12
Reducing stack by rule 10 (line 193), lineseq line -> lineseq
Entering state 4
Reading a token: ### 1:LEX_NORMAL/XSTATE ""
### Tokener got EOF
### <== EOF

Now at end of input.
Reducing stack by rule 1 (line 140), progstart lineseq -> prog
Entering state 1
Now at end of input.

EXECUTING...

\,X

[download]

Having further reviewed the tokeniser code, I am reasonably certain how these escapes are handled but still wondering if there is documentation other than that in perlop. I would be particularly interested to understand the cases/situations in which the inconsistencies are necessary or beneficial. I am guessing there are such cases as it would have been significantly simpler to implement, document and understand without the inconsistencies.

In Section Seekers of Perl Wisdom