Using Parse::RecDescent to parse Perl-ish strings without resorting to string eval

polypompholyx has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

I'm in the middle of updating a module that grades Excel spreadsheets by comparing the contents of cells to a model text file parsed by Parse::RecDescent. Strings are allowed in the comparisons, but in the previous version of the module, they were hacked in by simply returning a manually unescaped version of the matched text, e.g.:

quoted_string: '"' m{(([\\]"|[^"])*)} '"' 
{
  $item[2] =~ s{\\"}{"}g;   # Unescape quotes
  $item[2] =~ s{\\\\}{\\}g; # Unescape backslashes
  $item[2];
}
[download]

I want to use the <perl_quotelike> production (a wrapper around Text::Balanced), for greater flexibility with quoted strings and regexes. The problem is that <perl_quotelike> extracts the Perl-ish string/regex, but the only way I can think of to interpret the string/regex correctly (which could contain quotes, backslashes, Unicode hexes, regex modifers, etc.) is to eval the relevant bits, e.g.:

quoted_string: <perl_quotelike>
{
  my ( $name, $ldelim, $text, $rdelim ) = @{ $item[1] };
  if ( $name eq 'qq' ) {
    $text = eval 'qq' . $ldelim . $text . $rdelim;
  }
  # etc...
}
[download]

Which is nasty, as the model file text could then contain:

A1  mean(B1:B10)           &&
A2  "Something innocuous"  &&
A3  C1/C2                  &&
A4  qq(Oh dear @{[ system 'rm -rf *' ]})
[download]

Am I missing something, or am I trapped between either implementing my own interpolator/unescaper (which certainly won't be able to replicate all the useful features of perl quoting and regex modifiers), or using string eval (and hoping that no-one does anything nasty)?

Comment on Using Parse::RecDescent to parse Perl-ish strings without resorting to string eval Select or Download Code

Replies are listed 'Best First'.
Re: Using Parse::RecDescent to parse Perl-ish strings without resorting to string eval by ikegami (Patriarch) on Feb 29, 2008 at 17:10 UTC
Don't. It might be a lot of work to convert from Perl, but it's even more work to parse Perl. If you insist, you probably should start by taking a look at String::Interpolate (uses a Safe compartment to confine the interpolation to specified `VAR => $value` pairings) and PPI (a Perl parser). `quoted_string: '"' m{(([\\]"\|[^"]))} '"'`* I hope your never have spaces after the first "`"`" in the text you are parsing, or did you change `<skip>`? It's best to avoid separating a token into multiple items. It's rarely necessary or even easier.	[reply] [d/l] [select]
Re^2: Using Parse::RecDescent to parse Perl-ish strings without resorting to string eval by polypompholyx (Chaplain) on Feb 29, 2008 at 20:08 UTC
Yes, in the original code, there's some fiddling with `<skip>` (since whitespace has syntactic significance in the grammar), and unescaping of Unicode hex character escapes. I left them out for clarity. I can probably live with handrolled code that parses its own definition of a double quoted string, since most of the interpolation is unnecessary, but I'm not sure of any way out with regexes: I'd like to be able to do something like: `$text = qr/$text/$modifiers` [download] but `$modifiers` doesn't interpolate, and I can't think of any easy way to simulate this without having a very large switch statement.	[reply] [d/l] [select]
Re^3: Using Parse::RecDescent to parse Perl-ish strings without resorting to string eval by ikegami (Patriarch) on Feb 29, 2008 at 20:20 UTC
`my $mods_p = ''; my $mods_m = ''; ($opts{$_} ? $mods_p : $mods_m) .= $_ for qw( x i s m ); $re = qr/(?$mods_p-$mods_m:$re)/;` [download]	[reply] [d/l]
Re^4: Using Parse::RecDescent to parse Perl-ish strings without resorting to string eval by polypompholyx (Chaplain) on Feb 29, 2008 at 20:25 UTC
Re: Using Parse::RecDescent to parse Perl-ish strings without resorting to string eval by locked_user sundialsvc4 (Abbot) on Feb 29, 2008 at 17:06 UTC
Is it feasible to output the Excel data in a known structured format, such as XML? That would open up the world of `XML::XPath` where you could query the XML-formatted data ... in effect letting something else (namely, XPath) do the dirty-work of pulling out the data you need. The “model file” would not need to be the same format. If it consists of a series of lines containing “cell-address, and what should be in it,”then your code could work by constructing an XPath-string, letting XPath dredge the file for that piece of data, compare it, and move on. “The model file” is the part that you can easily control; the student's homework is the Great Unknown. But if the complexity were hidden from view (XPath is very good at what it does, and XPath expressions are very powerful), the complexity of your program would be reduced by at-least half. I'd call that “a win.”
Re^2: Using Parse::RecDescent to parse Perl-ish strings without resorting to string eval by polypompholyx (Chaplain) on Feb 29, 2008 at 20:21 UTC
Pulling the data out of Excel isn't too difficult (`Win32::OLE`), it's the parsing of the model file that's the problem. The model file, in its simplest form, is exactly what you say: a list of cells and what should be in them, in terms of strings, calculations, etc. I leverage Perl's regexes to grade things like graph axis titles - as you guessed, there's a hideous number of ways for them to write µmol min⁻¹ mg⁻¹, which makes me ache for the `/x` modifier. Using `<perl_quotelike>` seemed like an easy way out for both strings and regexes, but it looks increasingly like it'd be better to hand-roll something more limited.	[reply] [d/l] [select]