It's an even bigger win if you have components that rarely change. If you only have to compile a component once, it'll run at full Perl speed each time through.

If you have a working parser right now, it's not too difficult to generate Perl code. You'll certainly learn a lot. Feel free to download either package and dig through to see how it works.

Wow, a reply in a very short ammount of time.. :)

Well. I don't quite understand
what do you mean "turn a template into real Perl code"? I already have perl code. maybe I was unclear (I am sometimes..)

Well, sort of everything changes...

it parsers generates HTML, not Perl code. and by 'compile' I ment it puts the same head and foot file on every page.

I use the <@TAGS@> for all the colors.. When called like: manifold.cgi?mode=b it has basic, black and white and blue colors. and so on. So, my site is thus skinned. Perhaps I should post the code..

Turning a template into Perl code means turning a template inside out: typical templating systems use a mixture of HTML and Perl (or Perl and "something else", which isn't necessarily HTML), where the Perl is "inside" the HTML. Like this, sort of:

    <b>Name: </b><% $name %><br>
[download]

The HTML is the outer layer, and the Perl is contained within special sections marked off by some delimiters. In this case, <% and %>.

Most templating systems will turn something like the above into this:

    $_out->("<b>Name: </b>");
    $_out->( $name );
    $_out->("<br>");
[download]

Imagine $_out is a subroutine reference that is basically just

    my $_out = sub { print @_ }
[download]

So essentially, "turning a template into Perl code" has created a piece of Perl code with a bunch of print statements, which is what you might have written if you hadn't used templates at all. This is what I mean by turning the template inside out: you have taken a template that was HTML with embedded Perl, and have turned it into a Perl script with embedded HTML.

The advantage to Perl code is that Perl code is fast. :) Ie. if you have a block of Perl code, you can execute it directly; you don't have to parse it as a template. So you can store it in this intermediate Perl script form, and then just run it again without reparsing the original template.

This is a relatively standard way of caching templates for many Perl templating engines. And obviously, this is just a small example, but it shows the basics of how it works.

Oh dear. In my question, I said a crazy perl and I ment to say a crazy person..

Which web server are you using? If you're using Apache or Zeus, then I must recommend using FastCGI. Basically, if your program looked something like this:

my $cgi = new CGI;

$my_code->{$cgi->param("action"))->($cgi);
[download]

You'd change it to:

my $req = FCGI::Request(...);

while ($req->Accept()) {
    my $cgi = new CGI;

    $my_code->{$cgi->param("action"))->($cgi);
}
[download]

Then your perl program is just sitting in a little loop calling the code to generate a page when the web server connects to it. You should be able to get of the order of hundreds of dynamic page impressions per second. I'm getting 50 a second with a project I'm working on on my system, which includes some heavy database access.

For your sanity, you might consider switching over to one of the standard templating tools like Apache::SSI or Template Toolkit. You can read my recent article on perl.com for some background on these. You should also think about breaking up this 400 lines monster into modules. It will make maintenance easier.

I think using ePerl would be a good idea.

Why? Well, look at this: Manifold.txt
Ugly, and very long. I have a bet with myself that I can make it shorter, with ePerl. :)

If it's still slow after that, you should do some profiling on it and find the slow part.

And what exactly do you do when you realise the slow part is the system paging, from having to keep a seperate perl process for each and every web server child that most of the time is blocked on network I/O?

You have at that point many options.
1. Buy more RAM.
2. Reconfigure Apache to have fewer processes.
3. Identify parts of your system that take up the most memory. Then you can:
   1. Rewrite them to be less memory intensive.
   2. Turn them back to CGI.
   3. Use a dedicated serving process (eg FastCGI) for those components.
   4. Use shared memory segments.
4. Reduce the number of requests that a given child will handle before restarting.
5. Move to an OS with on demand paging (rather than per process) and give it lots of temp space. (This mainly applies if you are not already using some form of *nix.)
6. Buy faster hard drives.
There is no universal solution to poor performance. However the vast majority of the time mod_perl is a massive improvement over straight CGI. Time and money spent optimizing CGIs to run faster is typically penny wise and pound foolish. Hardware is cheap. With the spare hardware from the dotbombs, it is really cheap. Unless you are doing something ridiculous, the odds are very good that hardware is the best solution at hand.

Maybe eliminate the network I/O for your heavy mod_perl procs using a proxy. Perlmonth has a great article on how to properly set this up.

-Blake

I figured if I put most of the neat functions in a module, than I could use them if I ever want to use some other templating toolkit.