Yes, they are handled in the tokeniser. It was while reading the tokeniser code that I became confused/curious about how they operate and why.

In the case of qq(\Q\U,a) this is equivalent (converted to) quotemeta(uc(',a')), as can be seen from the following:

[ian@alula perl]$ /usr/local/perl-igannotate/bin/perl -DpT -e 'print q
+q(\Q\U,x),"\n"'
Starting parse
Entering state 0
Reducing stack by rule 5 (line 164), -> progstart
Entering state 2
Reducing stack by rule 8 (line 185), -> lineseq
Entering state 4
Reading a token: ### 0:LEX_NORMAL/XSTATE "\n;"
### <== LSTOP(ival=op_print)

Next token is token LSTOP (0xde)
Reducing stack by rule 51 (line 474), -> label
Entering state 15
Next token is token LSTOP (0xde)
Shifting token LSTOP, Entering state 62
Reading a token: ### 1:LEX_NORMAL/XREF " qq(\\Q\\U,x),\"\\n\"\n"
### <== FUNC(ival=op_stringify)

Next token is token FUNC (0x43)
Shifting token FUNC, Entering state 60
Reading a token: ### 1:LEX_INTERPPUSH/XTERM ",\"\\n\"\n"
### <== '('

Next token is token '(' (0x43)
Shifting token '(', Entering state 137
Reading a token: ### 1:LEX_INTERPCONCAT/XTERM "\\Q\\U,x"
### 1:LEX_INTERPCASEMOD/XTERM "\\Q\\U,x"
### Saw case modifier
#### pushed 1 PL_lex_casestack: Q, PL_lex_casemods: 1
### forced token:
### <== '('

### forced token:
### <== FUNC(ival=op_quotemeta)

### 1:LEX_KNOWNEXT/XTERM "\\U,x"
### <== FUNC(ival=op_quotemeta)

Next token is token FUNC (0x7d)
Shifting token FUNC, Entering state 60
Reading a token: ### 1:LEX_KNOWNEXT/XTERM "\\U,x"
### <== '('

Next token is token '(' (0x0)
Shifting token '(', Entering state 137
Reading a token: ### 1:LEX_INTERPCONCAT/XTERM "\\U,x"
### 1:LEX_INTERPCASEMOD/XTERM "\\U,x"
### Saw case modifier
#### pushed 1 PL_lex_casestack: QU, PL_lex_casemods: 2
### forced token:
### <== '('

### forced token:
### <== FUNC(ival=op_uc)

### 1:LEX_KNOWNEXT/XTERM ",x"
### <== FUNC(ival=op_uc)

Next token is token FUNC (0x7b)
Shifting token FUNC, Entering state 60
Reading a token: ### 1:LEX_KNOWNEXT/XTERM ",x"
### <== '('

Next token is token '(' (0x0)
Shifting token '(', Entering state 137
Reading a token: ### 1:LEX_INTERPCONCAT/XTERM ",x"
### forced token:
### <== THING(opval=op_const) PV(",x"\0)

### 1:LEX_KNOWNEXT/XTERM ""
### <== THING(opval=op_const) PV(",x"\0)

Next token is token THING (0xa1a3348)
Shifting token THING, Entering state 42
Reducing stack by rule 164 (line 1137), THING -> term
Entering state 89
Reading a token: ### 1:LEX_INTERPSTART/XTERM ""
### 1:LEX_INTERPCASEMOD/XTERM ""
#### popped 1 PL_lex_casestack: Q, PL_lex_casemods: 1
### <== ')'

### <== ')'

Next token is token ')' (0xa1a3348)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ')' (0xa1a3348)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 232
Next token is token ')' (0xa1a3348)
Reducing stack by rule 198 (line 1287), expr -> listexprcom
Entering state 233
Next token is token ')' (0xa1a3348)
Shifting token ')', Entering state 319
Reducing stack by rule 95 (line 752), FUNC '(' listexprcom ')' -> list
+op
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 89
Reading a token: ### 1:LEX_INTERPCASEMOD/XTERM ""
#### popped 1 PL_lex_casestack: , PL_lex_casemods: 0
### <== ')'

Next token is token ')' (0xa1a3348)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ')' (0xa1a3348)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 232
Next token is token ')' (0xa1a3348)
Reducing stack by rule 198 (line 1287), expr -> listexprcom
Entering state 233
Next token is token ')' (0xa1a3348)
Shifting token ')', Entering state 319
Reducing stack by rule 95 (line 752), FUNC '(' listexprcom ')' -> list
+op
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 89
Reading a token: ### 1:LEX_INTERPCASEMOD/XTERM ""
### 1:LEX_INTERPCONCAT/XTERM ""
### <== ')'

Next token is token ')' (0xa1a3348)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ')' (0xa1a3348)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 232
Next token is token ')' (0xa1a3348)
Reducing stack by rule 198 (line 1287), expr -> listexprcom
Entering state 233
Next token is token ')' (0xa1a3348)
Shifting token ')', Entering state 319
Reducing stack by rule 95 (line 752), FUNC '(' listexprcom ')' -> list
+op
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 89
Reading a token: ### 1:LEX_NORMAL/XOPERATOR ",\"\\n\"\n"
### <== ','

Next token is token ',' (0xa1a3348)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 141
Next token is token ',' (0xa1a3348)
Shifting token ',', Entering state 180
Reading a token: ### 1:LEX_NORMAL/XTERM "\"\\n\"\n"
### Saw string before "\n"
### <== FUNC(ival=op_stringify)

Next token is token FUNC (0x43)
Shifting token FUNC, Entering state 60
Reading a token: ### 1:LEX_INTERPPUSH/XTERM "\n"
### <== '('

Next token is token '(' (0x43)
Shifting token '(', Entering state 137
Reading a token: ### 1:LEX_INTERPCONCAT/XTERM "\\n"
### forced token:
### <== THING(opval=op_const) PV("\n"\0)

### 1:LEX_KNOWNEXT/XTERM ""
### <== THING(opval=op_const) PV("\n"\0)

Next token is token THING (0xa1a3bb0)
Shifting token THING, Entering state 42
Reducing stack by rule 164 (line 1137), THING -> term
Entering state 89
Reading a token: ### 1:LEX_INTERPSTART/XTERM ""
### <== ')'

Next token is token ')' (0xa1a3bb0)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ')' (0xa1a3bb0)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 232
Next token is token ')' (0xa1a3bb0)
Reducing stack by rule 198 (line 1287), expr -> listexprcom
Entering state 233
Next token is token ')' (0xa1a3bb0)
Shifting token ')', Entering state 319
Reducing stack by rule 95 (line 752), FUNC '(' listexprcom ')' -> list
+op
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 258
Reading a token: ### 1:LEX_NORMAL/XOPERATOR "\n"
### <== ';'

Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 86 (line 694), argexpr ',' term -> argexpr
Entering state 141
Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 196 (line 1281), argexpr -> listexpr
Entering state 142
Reducing stack by rule 94 (line 748), LSTOP listexpr -> listop
Entering state 83
Reducing stack by rule 186 (line 1236), listop -> term
Entering state 89
Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 87 (line 703), term -> argexpr
Entering state 82
Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 84 (line 680), argexpr -> expr
Entering state 81
Next token is token ';' (0xa1a3bb0)
Reducing stack by rule 18 (line 249), expr -> sideff
Entering state 78
Next token is token ';' (0xa1a3bb0)
Shifting token ';', Entering state 170
Reducing stack by rule 16 (line 228), label sideff ';' -> line
Entering state 12
Reducing stack by rule 10 (line 193), lineseq line -> lineseq
Entering state 4
Reading a token: ### 1:LEX_NORMAL/XSTATE ""
### Tokener got EOF
### <== EOF

Now at end of input.
Reducing stack by rule 1 (line 140), progstart lineseq -> prog
Entering state 1
Now at end of input.

EXECUTING...

\,X

[download]

Having further reviewed the tokeniser code, I am reasonably certain how these escapes are handled but still wondering if there is documentation other than that in perlop. I would be particularly interested to understand the cases/situations in which the inconsistencies are necessary or beneficial. I am guessing there are such cases as it would have been significantly simpler to implement, document and understand without the inconsistencies.

In reply to Re^2: Interactions of \Q, \U, \L and \E by ig
in thread Interactions of \Q, \U, \L and \E by ig

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.