It sure did help my project. I see your point about programming language conversion. I'd be suspicious of it too. It's generally a pretty hard thing to do due to fundamental differences in programming languages. I'd hate, for example, to try and convert perl or lisp into VB - lack of lexical closures would just be too crippling.

I didn't think at one time that it would be possible to do justice to a lisp->perl conversion either.

As I've said on my website, the main use I've put this to (apart from compiling itself) is in converting XML from one form to another, whilst validating it at the same time. I wrote the previous version of this program in perl and it worked pretty well. The code was reasonably nice. I always had a feeling, though, that there must be a much easier way of doing it. What I almost wanted was a special purpose 'conversion language', but with the power of a general purpose language. (XSLT would not nearly have done).

I don't see lisp as a non general purpose language, though I realise it was originally intended/used for AI. I think it makes an excellent general language (though perl has its advantages - otherwise I wouldn't use it). I don't think that <it>it</it> was the wrong choice of language. Perl was. It wouldn't have been unreasonable to implement the whole thing in lisp. The lisp code base for this translator is MUCH cleaner, more complete and nicer to work with than the perl one, despite the general strangeness of translating it to perl. Its also faster!

There is one feature of lisp (this one included) which made it worth doing this project in lisp, and that is macros. Macros are _enourmously_ powerful. If you haven't ever used them I'd strongly recommend you look into them. Going from a language w/o macros to one with is like going from a line number version of BASIC to perl (almost/sort of). Paul Graham has written a lot about this (www.paulgraham.com).

Of course, perl 6 is going to implement macros, which will be very cool.

I have not yet had the time to investigate source filters to my liking (its on my list), but it seems like you have the full power of Perl available to you at compile time to munge the code in any way you see fit. Could some one give me the 50,000 ft overview of why Lisp macros are easier/stronger/faster/better? Is it a matter of syntatic sugar, or is there a deeper principle involved. (OK, my initial guess is that you'd have to include a full perl parser in your source filter to have a truly flexible macro system.)

Lisp macros act on the parsed version of your code--they go after the parser. Source filters, on the other hand, work on the raw text of your code--they go before the parser.

For a language like Lisp, there's argably little difference (Lisp is dead-simple to parse) though it's definitely less of a hassle to work on the parsed tree. For a language like perl, macros would be a very nice thing since parsing perl correctly is a profoundly non-trivial exercise.

Macros and source filters both have their place. It's just a different place...

Could some one give me the 50,000 ft overview of why Lisp macros are easier/stronger/faster/better?

You might find Macros, LFSPs and LFMs and Pretending to be lisp: macros of interest.

Especially from a language that has specially targetted application area, to a general language like Perl. The need or motivation of such coversion, most likely, indicates that the choice of Lisp was wrong at the beginning.

I know many Lisp developers (myself included) who would disagree that it's aimed at a particular application area. It's like saying Perl is only useful for CGI scripts. While Lisp was certainly used a lot for AI work, it wasn't targeted at the AI field - it just happened to be the language best suited to it at the time.