Reciently I've read a couple postings and articles about Perl optree manipulations. Now I happen across this thing by Paul Graham:
If you understand how compilers work,what ’s really going on is not so much that Lisp has a strange syntax as that Lisp has no syntax.You write programs in the parse trees that get generated within the compiler when other languages are parsed.But these parse trees are fully accessible to your programs.You can write programs that manipulate them.In Lisp,these programs are called macros.They are programs that write programs.
We're talking about manipulating the parse tree after the front-end "syntax" of the language is all gone. Optree manipulation is close to that. In my own musings in the past, I thought about having an explicit intermediate tree form that can be manipulated in this manner, without having to actually write in it in the first place.

Now Perl 6 will have extensible grammers. To change the grammer, you will have to know the underlying parse tree structure so you can write new productions that are meaningful to the next stage.

This means that the parse tree stage as an abstract representation of the program will be documented and stable, sort of like the interface designed for derived classes to use.

Already Perl interleaves compile and run phases, and this is how pragmatic imports work. Actually, all imports are "pragmas", in that they work by manipulating the caller's symbol table before compilation continues forward.

So, I wonder what it would take to have a Perl feature similar to what LISP calls Macros, and I fully-qualify as a parse-tree macro?

I think the crux of the matter is to make it implictly a BEGIN block, just like use is today. A function that was implicitly called as immediatly after compiling the call to it could contain code to manipulate the compiler's state such as the flags and symbol table (as import does today) and the parse tree in progress, using the hypothisized abilities.

Knowing this, I wonder if the component features can be tweeked in the design phase to better support this emergence.

Feel free to pass this along to anyone interested.

—John

Replies are listed 'Best First'.
Re: Parse-Tree Macros : from LISP to Perl 6?
by Elian (Parson) on Nov 26, 2002 at 22:22 UTC
    The reason you can do this so easily in lisp is because you're actually writing the parse tree by hand--lisp has no syntax to speak of.

    The reason you have to do this in lisp is because lisp has no syntax to speak of, and if you actually want any you need to roll it yourself. (Not, mind, that you can actually get syntax with macros, since a macro-ized lisp program is still a lisp program)

    You'll have access to the AST the perl parser generates, and can manipulate it as you see fit before it goes to the compiler, though you'll find, I think, that you'll want (and have) to do this far less often in perl than you might in lisp.

    Also be aware that this will be one of the features you'll most want to kill someone for using, if someone actually uses it. With great power comes great fuck-ups. (This is true in the Lisp community as well, which should be very, very worrisome. There are very few non-uber-hackers in the lisp world, as that sort of folk use other, more useful and less painful languages. And if the scary-smart people mess it up, well...)

    If past history is anything to go by, the first word you should utter when you hear someone say "I can mess with the parse tree!" is... Don't!

[reply]

      I can mess with the opcode tree ... I did eventually get farther with that code but never quite got it working. I blame it on my own incompetance at optree hacking. Seriously... B::Generate is quite the awesome module. There are other more mundane uses for this but it's still quite fun for just this one anyway.

      Update: Oh duh. Ok well I did this in perl 5 which is offtopic from John's perl6 question but I rather expect (for no particular reason) that such hackery will become even easier for perl6

      __SIG__
use B;
printf "You are here %08x\n", unpack "L!", unpack "P4", pack
  "L!", B::svref_2object(sub{})->OUTSIDE;

      [download]
[reply]
[d/l]

        I wouldn't be too sure about that. I doubt that the ability to hack the optree will be a central goal in the desing of the perl6 compiler. However, the compiler isn't anywhere near completed yet, so its probably too early to jump to any sort of conclusion about it.

        However, it should be possible to get at the intermediate compilition step, although it won't be as easy to manipulate.

[reply]

Back to Meditations