szabgab has asked for the wisdom of the Perl Monks concerning the following question:

I am building a new parser for CPAN::Forum with Parse::RecDescent that I have just started to play with.Here is a problem I encountered: I'd like to convert 
<xxx> $x < $y </xxx>

[download]
to 
<div class="code"> $x &lt; $y </div>

[download]

I came up with the following grammar: 

my $grammar = q {
    entry: chunk(s) eodata  { $item[1] }
    chunk: text | code
    text: m{[\w ]+} { qq(<div class="text">$item[1]</div>); }
    code: opencode codetext closecode {$item[2] }
    opencode:  m{<xxx>}
    closecode: m{</xxx>}
    codetext: m{[\w <\$]+(?=</xxx>)} { qq(<div class="code">) . CGI::e
+scapeHTML($item[1]) . qq(</div>); }
    eodata:   m{^\Z}
};

[download]
I wonder if I could solve this problem without that lookahead in the codetext. I mean I already said in the closecode what should be the end of the code. Or should I drop altogether the closecode tag ?

BTW it is not really xxx there but I could not keep the real word while posting on PerlMonks. Extra XP for those who can guess what do I have instead of xxx ;-)

Update:

In the end the text parts will be able to contain some HTML markup and some special tags while the code parts will contain anything that can resemble some software source code. See the discussion about Web forum markup language and the Monastery

The exact nature of the special tags is not yet clear to me, though I am thinking about <<special_tag>>

Replies are listed 'Best First'.
Re: Parse::RecDescent with lookahead or without ?
by ikegami (Patriarch) on Jan 22, 2005 at 17:43 UTC

    Does your format ressemble XML enough to use an XML parser? That would be best. The remainder of the post assumes you can't use an XML parser.

    If I follow correctly, you're using the lookahead so that the < of <div> doesn't get absorbed by the regexp. You can avoid that by breaking codetext into smaller tokens instead of treating it as a single token.

        entry        : chunk(s?) EOF { join('', @{$item[1]}) }

    chunk        : text { $item[1] }
                 | code { $item[1] }

    text         : TEXT                      { join('', '<div class="t
+ext">', $item[1],                  '</div>') }
    code         : CODE_OP CODE_TEXT CODE_CL { join('', '<div class="c
+ode">', CGI::escapeHTML($item[2]), '</div>') }

    # Tokens
    EOF          : m{^\Z}
    TEXT         : m{[\w ]+}                             { $item[1] }
    CODE_OP      : m{<xxx>}                              { $item[1] }
    CODE_CL      : m{</xxx>}                             { $item[1] }
    CODE_CHARS   : m{[\w $]+}                            { $item[1] }
    CODE_SPECIAL : m{<\w+>} { $item[1] ne '<xxx>' }      { $item[1] }

    # Pseudo-token
    CODE_TEXT    : CODE_TEXT_(s?)                        { join('', @{
+$item[1]}) }
    CODE_TEXT_   : CODE_CHARS                            { $item[1] }
                 | CODE_SPECIAL                          { $item[1] }

    [download]

    UNTESTED.

[reply]
[d/l]
[select]
      I might be able to turn it to XML but I'll need to munge the input fot that. I am not sure I want to start going down that slope. See my updates above regarding what the input should look like.
[reply]
Re: Parse::RecDescent with lookahead or without ?
by dimar (Curate) on Jan 22, 2005 at 17:35 UTC

    This does not answer your primary inquiry, however extra XP to you if you can figure out why there is no xxx in this response (besides the one in this very sentence ;-). 

    my  $grammar = q^
    opencode:  m{<code>}
    closecode: m{</code>}
^;
[reply]
      <code>I have no idea :)</code>
[reply]

Back to Seekers of Perl Wisdom