After reading Brother hhdave's meditation about Lisp to Perl compilation I started to ponder the wonders of macros and what kinds of source transformations we could do with perl's source filters. The concensus seems to be that source filters are better than the simple text substitutions of CPP, but are lesser than Lisp macros, since Lisp macros transform the code after the parsing stage. So I wanted to see just what could be accomplished with the resources at hand. I figured it would be best not to try to roll my own parser, especially since perl's already got a pretty darn good perl parser. Since I didn't want to delve too far into perl's guts or mess around with op-codes, I decided to take a look at B::Deparse. B::Deparse is code generator which takes a compiled perl program's internal representation and converts it back to perl code. I figured I'd take my macro, compile it, B::Deparse it, and use this representation at a template for matching against a deparsed version of the source code we want to macroify. My first filter employing this concept tries to unroll 3-part for(;;) loops with fewer than 10 iterations. (Warning! This is pre-alpha quality proof-of-concept code.)
package Loop_unroll; use strict; use Filter::Simple; FILTER { my $unroll_limit = 10; #max unrolling depth my ($var1, $cons2, $var3, $cons4, $var5, $body6); #This is the macro replacement pattern we're looking for... my $macro = 'for ($var1=$cons2;$var3<$cons4;$var5++) {$body6}'; #run the macro snippet through B::Deparse (better way to grab outp +ut?) my $compiled_macro=`perl -MO=Deparse,-x9 -e'$macro'`; #screws with + ', FIX ME. #compile the main script my $source = `perl -MO=Deparse,-x9 -e'$_'` if $_; #same proble +m, Bad dog. #These are crude regex to try to match perl variables and integers $var1 = $var3 = $var5 = '(\$[a-zA-Z][a-zA-Z0-9]*)'; $cons2 = $cons4 = '(\d+)'; $body6 = '([^}]+)'; #barf, should be something nicer, no nested bl +ocks #deswizzle variable order (mangled by compile) #variables should be guarenteed to occur only once because of side +-effects my $i; my $reorder; while ($compiled_macro=~/(var1|cons2|var3|cons4|var5|body6)/g) { $reorder.="\$$1 = \$".++$i.";"; # map vars to capturing parens } $compiled_macro=~s/([|(){}\^*+?.\[\]])/\\\\$1/g; #de-meta stuff, n +ot $ (ick) my $regexed_macro; eval '$regexed_macro=qq/'."$compiled_macro".'/'; #interpolate patt +ern match junk $source =~ s{$regexed_macro} { eval $reorder; # better method? ($var1 eq $var3 and $var1 eq $var5 and $cons4 - $cons2 < $unroll_limit) ? "$var1 = $cons2;".("{$body6;} $var1++;") x ($cons4-$cons2) : $&; #Lazy like a fox. }eg; $_ = $source; #print STDERR "$source\n"; }; 1;
And here's a program you can run it against...
#!/usr/bin/perl -w use strict; use Loop_unroll; my $x; for($x=0;$x<8;$x++) { my $f=fact($x); print "factorial($x)=$f\n"; } sub fact { return(1) if ($_[0] <= 0); return $_[0]*fact($_[0]-1); }
My second filter attempts to undo one level of recursion in simple tail-recursive subroutines.
package Recur_unroll; use strict; use Filter::Simple; FILTER { my ($name1, $body2, $name3, $param4, $body5); #This is the macro replacement pattern we're looking for... my $macro = 'sub name1 {$body2; return name3($param4); $body5}'; #run the macro snippet through B::Deparse (better way to grab outp +ut?) my $compiled_macro=`perl -MO=Deparse,-x9 -e'$macro'`; #screws with + ', FIXME. #compile the main script my $source = `perl -MO=Deparse,-x9 -e'$_'` if $_; #same proble +m, Bad dog. #These are crude regex to try to match perlisms $name1 = '\s+([a-zA-Z][a-zA-Z0-9]*)\s+'; $body2 = '(.*?'; $name3 = '[^;]*?(\1'; $param4 = '([^)]*?)'; #no parens allowed inside sub parameter lis +t $body5 = ').*?)'; #deswizzle variable order (possibly mangled by compile) #variables should be guarenteed to occur only once because of side +-effects my $i; my $reorder; while ($compiled_macro=~/(name1|body2|name3|param4|body5)/g) { $reorder.="\$$1 = \$".++$i.";"; # map vars to capturing parens } $compiled_macro=~ s/name/\$name/g; #makes subroutine names var for + interpolation $compiled_macro=~ s/(body\d);/$1/g; #get rid of dummy placeholder +semi's $compiled_macro =~ s/([|(){}\^*+?.\[\]])/\\\\$1/g; #de-meta stuff, + not $ (ick) my $regexed_macro; eval '$regexed_macro=qq/'."$compiled_macro".'/'; #interpolate patt +ern match junk $source =~ s{$regexed_macro} { eval $reorder; $name3=quotemeta($name3); $body2=~s/$name3/eval{local \@_=$param4;$body2 $bo +dy5};/; "sub $name1 { $body2 $body5 }"; }esgx; $_ = $source; #print STDERR "$source\n"; }; 1;
That's not too terribly many lines of code, although its power is limited by the need for your macros to be compilable. Then there is the fact that there is still a certain amount of tinkering necessary to get each case to work correctly. And, of course, the macros I chose to implement are probably of dubious value. Can this approach be generalized/extended? Is it unbearably fragile? If perl5 had perl6-like macros available today, what would you try to accomplish with them? I'll leave those questions as an excersize to the reader :-)

Greg Buchholz


In reply to Playing with (macro/source-filter) fire by sleepingsquirrel

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.