Over the last weekend, I finally found some time to finish my conversion from HTML::Template to Template Toolkit, as was suggested by more than a few people back in September when I asked about generating dynamically-static documents. It took that long partly because this is a volunteer position, and work has been particularly crazy, and partly because it took that long for me to figure out the Template Toolkit way of doing things.

So I thought I'd share my perspective on this conversion process, what I think I gained, and what I think I lost. My project wasn't really involved, thankfully, although I did find that DBD::CSV had a new quirk since I last created these documents via HTML::Template.

I started with some HTML templates, CGI::Application, and a home-grown framework that ran a CGI application through each runmode, saving each to a separate HTML file. It was the loss of this last piece in switching computers that caused my query 6 months ago - to see if there was a better way. For the most part, Template Toolkit's ttree was that better way.

One of the first problems I faced, which stalled me for nearly 6 months, was my navigation bar. It is a mostly-common menu system that all my pages share. Yet, it is just a tiny bit different on each page. For starters, depending on which page is being shown, one of the entries in the navigation bar would have a class of "active" so that, via CSS, it would be shown differently (e.g., bold and a different background colour). Also, if a certain page was shown, it would "open" the navigation. e.g., if the current page was "Topic 3", and that topic had 2 sub pages, you would see "Topic 3a" and "Topic 3b" in the navigation bar. But if the current page was "Topic 2", you wouldn't see either 3a or 3b. Of course, if the current page is 3a or 3b, then you would also see both. Figuring out how to do this in TT2 stumped me. I had something like this:

[% SET navbar = [ { title = 'Home', url = 'index.html', }, { title = 'Topic 1', url = 'topic1.html', sub_pages = [ { title = 'Topic 1a', url = 'topic1a.html', }, ] }, ] %]
But I couldn't figure out how to modify that to add extra items based on the current page, such as signifying which page was current, or opening and closing sub pages (not all entries would open and close - some would always remain opened). As mentioned, I gave up. I moved the data structure into a Template::Plugin-derived object, and when I would USE it, I would pass in the page name, and let pure-perl massage the object into what I needed based on the current page.

This is not significantly different than what I did with HTML::Template, however. I had an object encapsulating this whole concept. I didn't fully like it there, still don't, but that's not the fault of HTML::Template or Template toolkit. I just don't like seperating information between two places. (If someone tells me how, I may go back to having this in a template because at least it would be physically closer to the templates it is referring to.)

Once I started down this road, the rest became much easier. I created another plugin which had all my DBD-reading and massaging of data. This was pretty much a rip of the code I had for CGI::Application, and plopped right into the plugin. All I did was remove the calls to HTML::Template, and just return the data I was previously sending to HTML::Template via the param flag. However, one small change was that in HTML::Template-land, I found that if I wanted some dissimilar data, I had to generate all of that and put it in a single hash. In TT2-land, it seemed easier to seperate out each into their own variables or methods. So I ended up with a lot more functions in the TT plugin than I had before. At that point, I added Memoize to the mix, and just memoize'd any functions that were called multiple times (since the output was always the same for everything but the navigation bar). If it took any more effort than that, I'd call it premature optimisation. As it was, it didn't cost me any effort, so I figured the trade-off was worth it even for what may be no gain.

Converting the perl code was probably more work than converting the templates, however. The templates were mostly easy - although I had some nested loops that messed me up where I didn't have the [% END %]'s nested properly:

<div class="navbar-level1"> [% FOREACH n1 = NavBar.navbar %] [% PROCESS navbarentry nav = n1 %] [% IF n1.sub_level %] <div class="navbar-level2"> [% FOREACH n2 = n1.sub_level %] [% PROCESS navbarentry nav = n2 %] ... [% END %] </div> [% END %] [% END %] </div>
This is the correct way. The wrong way had the /div tags in the wrong place I can't quite recall how I had it the wrong way. I had to indent everything like this to make it possible to figure out. When I did this for the HTML::Template, I didn't have the convenience of that PROCESS command, but the closing of the loops and ifs being different did make it a bit easier to follow. Also, if someone knows how to make this recursive such that I don't need to keep repeating myself like this, it would be appreciated - I'm pretty sure it's not possible in HTML::Template, but am not sure about TT.

I found a simple regular expression helped with the majority of the template conversion: s/\&lt;tmpl_var name="([^"]+)"\&gt;/[% $1 %]/g. That's not quite generic because H::T allows you to omit the quotes, or even the name= identifier. However, I never did that, so this worked for me. Converting the tmpl_if's and tmpl_loop's, however, I did completely by hand. Partly because there were fewer of them, partly because I wanted to ensure I had everything lined up properly.

During this, I found that TT didn't like constructs such as:

<a[% IF url %] href="[% url %]"[% END %][% IF active %] class="activ +e"[% END %]>
(There were actually a lot more than that on the line, but I'm leaving it at that to keep from having a too-long line.) The problem seemed to be having more than one "END" on a single line. H::T didn't have a problem with that, which is good, because anything outside the <tmpl_*>'s was literal. If I didn't want an end-of-line, I had to keep everything on one line. With TT2, I could use the "-" modifier on the opening or closing tags, but what I found is that it's much harder to visualise this way. I still have extra blank lines I haven't figured out how to rid myself of, which wasn't an issue with H::T.

Testing a single template via tpage was only moderately painful, but it was definitely a far cry from doing the same thing before. So that was a definite plus. Later on, I switched to using ttree, and my only real complaint thus far is the fact that the .ttreerc defaults to one's home directory unless overridden with an environment variable, instead of a command line parameter. Relatively minor, but I still wrapped that in a shell script so I wouldn't be able to forget. I figure that if I do this for this project, I may do it again for another project, and I don't want to create a new user for each project.

The real benefit, of course, is that I will have an easier time sending this whole thing on to another group who wants to steal our look, feel, and production methods. Whereas before I had to tell them that they needed perl experience and some home-grown stuff to run, now they have a chance to know ttree without me explaining it to them. In other words, if I lose it again, it'll be easier to rebuild ;-)

Key points:

Conversion of templates from H::T to TT2
Easy. The concepts are the same, so the significant difference is really the syntax. I'm sure that this could mostly be automated if desired.
Syntax
TT2 has some quirks in END handling, and the fact that the end of IF statements is the same as the end of FOREACH statements as everything else does make it a bit harder to read. I realise that perl has the same ending as well (the closing brace), but in perl, you're only reading perl. In TT2, you're reading both template markup and the underlying language, whether that is HTML, shell scripts, XML, or whatever you're actually attempting to produce. And that underlying language may be sensitive to the whitespace (such as shell script) which makes it really important that you can easily eliminate those.
Ability to generate static documents from dynamic sources
ttree is the killer app for this requirement. It does a great job at this. The only problem I had was that I had multiple html outputs from the same H::T template input. So I put the common part into a single template, and then created a bunch of templates that called it. A bit extra work over doing it all in runmodes in C::A, but not a lot.
Seperation of code from markup (MVC)
I found that I had way more logic in my markup with TT. Things like PROCESS and BLOCK and the like make things easy in some ways, but a bit convoluted in others. USEing plugins just feels too much like code to me. That said, this is precisely what enables us to use ttree to convert an entire tree over. I can't have it both ways. I do feel, however, that H::T would be easier to teach to my wife, who used to know a bit of HTML, than TT2 would be. And that can be an important measurement as well.
It's actually that last point which is why I'm unlikely to convert my workgroup's web site over to TT. I'm handing it off to someone else right away as I'm changing groups. And she doesn't know perl, but knows a bit about HTML. Getting her to learn perl is one step - teaching her TT2 at the same time seems to be a bit of a stretch.

All in all, I'm mostly happy with the conversion. It seems a bit easier to handle overall by removing CGI::Application from the mix, and gave me an excuse to get my hands wet with a technology that two-thirds of the monks seem to rave about ;-)

Update: I remembered what my nested problem was, and fixed a typo.

Update2: I'm trying to replicate my multiple-ENDs problem and failing miserably at it. I must have been doing something wrong. Although my conclusion must have been wrong, I will state that my observations were correct: I had a line like that which wouldn't come out, when I split it up it started to work. Since then, I've fixed a number of other seemingly minor issues with the output in getting it to do what I wanted, and now I put it all back on a single line, and it works. I don't know what it was that made it not work. Now I believe, however, that if TT had different end-tags for IF, FOREACH, etc., that it would have detected my problem and told me I was being an idiot for incorrect nesting or something. ;-)

Replies are listed 'Best First'.
Re: Thoughts on converting from HTML::Template to Template Toolkit
by brian_d_foy (Abbot) on Mar 22, 2006 at 17:04 UTC

    If you don't want TT to add whitespace to your output, you can also use the PRE_CHOMP or POST_CHOMP directives when you make your Template object.

    My Template (2.802.14) doesn't seem to mind multiple [% END %] directives on the same line. Perhaps something else was going on in your code?

    #!/usr/bin/perl use Template; my $tt = Template->new; my $template = <<"HERE"; <a[% IF url %] href="[% url %]"[% END %][% IF active %] class="active" +[% END %]> HERE $tt->process( \$template, { url => 'http://www.example.com', active => 1 } ); $tt->process( \$template, { url => 'http://www.example.com', active => 0 } );

    As for ttree, you can specify the config file with the -f switch. Are you playing with an older TT, by any chance?

    --
    brian d foy <brian@stonehenge.com>
    Subscribe to The Perl Review

      Where did you get a 2.80 version of Template? Even their website says that the latest is 2.14a. There very well could be something going on, but when I split everything on to seperate lines, things got better - the loop actually printed something out. Also, using ttree, I'm not creating any Template object, although when I was having that problem, I was still using tpage to run my first template through the translator to ensure things were working.

      And config file - when perusing the ttree documentation, I didn't realise that the config file and the rc file were the same but different. I didn't think there would be two config files for the same executable.

        Maybe I got the distro version wrong. That's the version inside Template.pm, but the dist version is 2.14. Still, I don't have a problem with multiple [% END %]s on the same line. Perhaps you can post a script which illustrates the problem.

        You can specify Template directives such as PRE_CHOMP and POST_CHOMP in your ttree config file. The ttree man page explains it, or you can do ttree -h to see all of the options. You may not think you are creating a Template object, but you are; ttree is just doing it for you.

        --
        brian d foy <brian@stonehenge.com>
        Subscribe to The Perl Review
Re: Thoughts on converting from HTML::Template to Template Toolkit
by merlyn (Sage) on Mar 23, 2006 at 00:00 UTC
    During this, I found that TT didn't like constructs such as:
    <a[% IF url %] href="[% url %]"[% END %][% IF active %] class="activ +e"[% END %]
    I'm a pretty heavy user of TT, and I've never seen this. Can you reduce this to a test case that I can run? If so, I'd like to submit it to get it fixed.

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

Re: Thoughts on converting from HTML::Template to Template Toolkit
by idsfa (Vicar) on Mar 22, 2006 at 23:29 UTC

    You are not thinking about your issue with the navigation bar correctly. You do not need to add certain sub-items depending upon the current page. You need to hide certain sub-items. Plain HTML/CSS example:

    <style type="text/css"> ul li ul {display:none;} #active ul {display:block;} </style> <ul> <li id="active">Section 1 <ul> <li>Subsection 1.1</li> <li>Subsection 1.2</li> </ul> </li> <li>Section 2 <ul> <li>Subsection 2.1</li> <li>Subsection 2.2</li> </ul> </li> </ul>

    Renders as:

    • Section 1
      • Subsection 1.1
      • Subsection 1.2
    • Section 2

    Even though this contains both submenus, only the first one gets displayed. Without knowing exactly what the generated HTML of you navbar looks like, I can only point to the general technique ...


    The intelligent reader will judge for himself. Without examining the facts fully and fairly, there is no way of knowing whether vox populi is really vox dei, or merely vox asinorum. — Cyrus H. Gordon
Re: Thoughts on converting from HTML::Template to Template Toolkit
by holli (Abbot) on Mar 22, 2006 at 19:28 UTC
    I still have extra blank lines I haven't figured out how to rid myself of, which wasn't an issue with H::T.
    If you can effort the extra memory you can use a [%FILTER%] with a simple regex to get rid of the blank lines. But, if your not outputting text/plain who cares about newlines in html, xml and the like?
    USEing plugins just feels too much like code to me.
    Plugins are one of the strengths of TT. The ability to move the DB logic (DBI connections, querys, etc) into the template, is a feature I don't want to miss anymore. With plugins you can pull in the power of every perl module into your template.


    holli, /regexed monk/
      But, if your not outputting text/plain who cares about newlines in html, xml and the like?

      Well, as an example, if the blank line was inside the quotes of an attribute, such as:

      <a href="something.html ">
      That would be bad. However, while this particular case I was converting HTML::Template to TT, where that is mostly not an issue, I was also thinking at the same time of using TT at work where we are generating shell scripts. Extra end-of-line characters can be catastrpohic there.

      With plugins you can pull in the power of every perl module into your template.

      No doubt you could. Moving the logic in there, however, gives me some eeby-jeebies. It's great if there's a single developer working on non-translated text. It gets a bit uglier if the file, with its code, gets to go out for translation, or if you have someone who knows code great, and another person who can design the heck out of a website. (I'm definitely not in the latter category - I'd love to get my wife doing the site design instead of me... and too much logic in there will just confuse her.) I get the power. I do. But something about it just makes me feel like it's a power that's drawing me to the "dark side" of intermixing logic and layout.

Re: Thoughts on converting from HTML::Template to Template Toolkit
by bsb (Priest) on Mar 28, 2006 at 08:21 UTC
    Here's some scrappy code I used to convert HTML::Template templates to TT2 some years back, just to help the stragglers along...
    #!/usr/bin/perl -p # rough conversion of HTML::Template files to TT2 use strict; s{<(\/)?TMPL_(\S+)\s*(?:(?:NAME=)?['"]?(.*?)['"]?)?>} < #warn "Got ($1) ($2) ($3)\n"; my ($close, $tag, $name) = ($1,uc($2),$3); my $out = ''; if($tag =~ /^IF|UNLESS$/) { $out = $close ? "[% END %]" : "[% $tag $name % +]"; } elsif($tag =~ /^INCLUDE|ELSE$/) { $out = "[% $tag $name %]"; } elsif($tag eq 'LOOP') { $out = $close ? "[% END %]" : "[% FOREACH $nam +e %]"; } elsif($tag eq 'VAR') { $out = "[% $name %]"; } else { warn "Unknown tag $tag ($close,$name)\n"; } warn $out; $out; >gei;