Best Macro Syntax for Perl?

Jeffrey Kegler has asked for the wisdom of the Perl Monks concerning the following question:

I'm writing a parser which allows Perl callbacks, and I want to make some macros available to the programmer writing the callbacks. An example probably explains most easily. For a grammar rule to do addition

Sum ::= Addend + Addend
[download]

the user might want to supply a callback like

sub { <first Addend> + <second Addend> }
[download]

In this example, the things in angle brackets are the macros. The idea is that the parser replaces these before the callbacks are compiled.

My question is what's the best syntax to use for macros? I want macros that are easy to use, but on the other hand don't get in the way of Perl constructs, and on the third hand (?) aren't impossible to parse out. Obviously there are tradeoffs. The angle brackets of the above example are, to my mind, not the right choice. They too easily are mistaken for other Perl syntax, both by the programmer and the parser. Note that my primary question is not how to program the macro substitution, but what syntax to use for the macros?

I'm sure other modules have tackled this same issue. Pointers to modules which monks feel did macros right would be very helpful.

thanks, jeffrey kegler

Comment on Best Macro Syntax for Perl? Select or Download Code

Replies are listed 'Best First'.
Re: Best Macro Syntax for Perl? by FunkyMonk (Bishop) on Nov 12, 2007 at 00:10 UTC
Assuming you're not thinking about a source filter (which I always avoid)... What about using Template::Toolkit as the processing engine? You could let the user decide what syntax to use, defaulting to `[% ... %]`	[reply] [d/l] [select]
Re^2: Best Macro Syntax for Perl? by Jeffrey Kegler (Hermit) on Nov 12, 2007 at 01:08 UTC
The more I think about it, the better I like this idea (letting the user specify his own choice of macro syntax, but defaulting to `[% ... %]`).	[reply] [d/l]
Re: Best Macro Syntax for Perl? by Joost (Canon) on Nov 11, 2007 at 23:40 UTC
Well, you could go for JSP/ASP style constructs like `<% %>` and `<%= %>` which at least are fairly unambiguous but a bit clunky. Without using special unicode characters it's hard to see how you could reduce the construct to a single (pair of) character(s) - update: if you're using Perl, that is. In Lisp, it could probably be done a lot easier. There are a few modules on CPAN that already parse this kind of syntax. Or take a look at what other template modules on CPAN use. Update2: depending on your exact needs, you could also get away with things like `sub { $1 + $2 }` but that more or less implies you're not parsing whatever is in $1 and $2 at compile time. "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply] [d/l] [select]
Re: Best Macro Syntax for Perl? by TGI (Parson) on Nov 12, 2007 at 17:01 UTC
What about JavaScript? There's also a pure Perl JavaScript::Engine if you don't want to use Spidermonkey. TGI says moo	[reply]
Re: Best Macro Syntax for Perl? by pemungkah (Priest) on Nov 13, 2007 at 00:34 UTC
A couple random observations, thrown out sort of at random. I've implemented macro substitution code in Perl; the real core of it was this, stolen shamelessly from Mastering Regular Expresssions: `# Define base variable detector: any (possibly-nested) angle-bracketed + string. # Patterns to extract <variables> or >variables< from a string. my $out_angled; $out_angled = qr/ < ( [^<>] \| (??{$out_angled}) )* > /x; # open angle-bracket then ... # non-angle chars ... # or ... # another angle-bracketed item ... # if there are any .. +. # and a close angle +-bracket` [download] This detects arbitrarily-nested angle-bracket expressions. You would want to, I think, switch the macro brackets to a different string - angle-brackets were a requirement in my application. Fortunately the strings I was substituting in didn't need angle brackets for anything else. If you want to look at the complete class that implements macro expansion, take a look at App::SimpleScan::Substitution, which is the core of it. This routine as supplied is probaly horrendous overkill, because it was concerned with doing combinatorial substitutions (i,e, var1 has 3 values, var2 has 2, and var3 has 7, so a string containing all 3 variables results in 327 = 42 different expansions). You might consider gutting it to only do single substitutions, or simply only allow your variable dictionaries to have one value per variable. The code is pretty complex, because it allows for expansions that create strings that need to be re-expanded, because the values are actually further macro expansions. This lets you define really complex relationships "down the tree", and simply substitute one variable in at the top to expand all the possibilities. Like I said, probably total overkill for your application, but maybe the variable substitution and detection stuff will help. Probably the most difficult part of the code was remembering what had already been substituted so that recurrences of the variable (from expansion of other constructs) were substituted with the same value. The pattern match at the top will help a lot in discovering the macro variables. You should be able to rewire it for multi-character delimiters like this: `$templaty = qr/ \[% # open delim ( (?!%\]) # not-a-closer lookahead . # anything else )* # as many as there are %\] # close delim /x; @vars = ($string =~ $templaty);` [download] Take a look at pages 261 to 281 of Mastering Regular Expressions. Those 20 pages were worth the entire purchase cost of the book.	[reply] [d/l] [select]