Building html site maps

wfsp has asked for the wisdom of the Perl Monks concerning the following question:

I have an alphabetical site map divided into 26 pages.
Each page has a letter menu with links to the other pages.
Each page has a subject menu with links to bookmarks of the subjects on that page.
Each subject is followed by links (one per line) to all the articles on that subject.
The information for each article is held in a tab delimited flat file database (currently 1,300 records, 135KB, example record below). The database is loaded into a hash and traversed as shown in the pseudo code.
The resulting page looks like the pseudo page below.

It is a static web site updated up to 4 times a day. Rebuilding all the site maps every time is easier, in my opinion than trying to update them.

As it stands the body html is built tag by tag and then loaded into a template.
There are four loops each with various conditions (e.g. in the letter menu there is no link to the page you are on).
For me that is a significant amount of logic and I would like all of it to be in the script.

I have looked at HTML::Template and believe that although it could be used I'm afraid it would be very difficult to maintain.

How would other monks approach this task?

------------------------------------
pseudo code
------------------------------------

foreach letter ('a'..'z')
  make letter menu # for loop 1
  make subject menu # for loop 2
    foreach subject (subjects){ # for loop 3
      make subject header with bookmark
        foreach article (articles){ # for loop 4
          make link to article
        }
    }
  make page
}

------------------------------------
pseudo html page
------------------------------------

page a

letter menu: a b c d e f etc. (links to other pages)

subject menu: afghanistan africa asia etc. (links to subjects on this 
+page)

afghanistan
link to article one (newest first)
link to article two
link to article three
etc.

africa
link to article four
link to article five
link to article six

asia
link to article seven
link to article eight
link to article nine

subject menu (as above)

letter menu (as above)

------------------------------------
record from database
------------------------------------

u Ukraine 20050122 ../2005/01/22ukraine.html Ukraine: 'Orange Revoluti
+on' in Ukraine, 22 January 2005
(fields: letter, subject, date sort key, file, link text)
[download]

Updated: clarified pseudo code

Comment on Building html site maps Download Code

Replies are listed 'Best First'.
Re: Building html site maps by CountZero (Bishop) on Jan 30, 2005 at 14:26 UTC
As it happens I am just reading the O'Reilly "Badger" book about the Template Toolkit which devotes the whole of Chapter 11 to Managing Static Webpages. I think it will be perfectly suited for your purpose. On the O'Reilly site you can read Chapter 2 which deals with basic static web-sites with TT2. You will need some advanced techniques though (they are in Chapter 11). CountZero "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law	[reply]
Re: Building html site maps by dimar (Curate) on Jan 30, 2005 at 17:33 UTC
... and in addition to Template::Toolkit and the other fine and dandy modules out there ... you might consider the fine and dandy language known as "perl" (aka "Perl", aka "PERL"). (Who woulda imagined it!) Straight out of the box, you can write perl code that "cleans up" all that code you have inherited, and encourages all that great stylesheet conformity, and all that great MVC separation, and whips up a hot batch of any other buzzword-soup-of-the-day you may want to serve up. Here is an example that is all plain perl code (with one added subroutine) that keeps our HTML template looking more like HTML than like perl (instead of the other way around). ### begin_: init perl use strict; use warnings; ### begin_: init vars my $sOut; my $oAlph = [('A' .. 'Z')]; my $oSubj = {}; $oSubj->{A} = [qw(africa asia)]; $oSubj->{B} = [qw(belgium brazil )]; my $oArti = {}; $oArti->{africa} = [qw(afone aftwo afthree )]; $oArti->{asia} = [qw(asone astwo )]; $oArti->{belgium} = [qw(beone)]; $oArti->{brazil} = [qw(brone)]; ### begin_: OutputTemplate ### ------------------ ### ------------------ $sOut .= " <html> <head></head> <body> " .sLoop($oAlph,sub{ "<a href='#'>$_</a> \|\| " }) ."<hr /> " .sLoop([keys %{$oSubj}],sub{my $sLett=$_; "<h2>$sLett</h2> " .sLoop($oSubj->{$sLett},sub{my $sName=$_; "<h3>$sName</h3> " .sLoop($oArti->{$sName},sub{ "<h4>$_</h4> " }) }) }) . " </body> <html> " ; ### ------------------ ### ------------------ print $sOut; ### begin_: subroutines ### Simple Looping subroutine ### we use to make the perl code in our ### output template more compact sub sLoop { join"",map{$_[1]->();} (@{$_[0]}) }###end_sub ### begin_: end perl 1; __END__ [download] which gives us something akin to ... Read more... (1183 Bytes)	[reply] [d/l]
Re^2: Building html site maps by wfsp (Abbot) on Jan 31, 2005 at 18:54 UTC
scooterm Many thanks for reply. I'll need to make a hard copy of that and study it to try and see what's happening there. Certainly looks a lot neater than what I have at the moment. I think I need to have one more 'go' at H::T, but I must admit I still hanker after having it all in the script! Again, many thanks John	[reply]
Re^3: Building html site maps by punkish (Priest) on Jan 31, 2005 at 21:28 UTC
I think I need to have one more 'go' at H::T, but I must admit I still hanker after having it all in the script! It is your script, so do what you wish. However, it would be great if you share the reason for your "hankering." Maybe we will learn something. Otoh, trust me, no better yet, trust the wisdom of ages -- separating logic from display would be the best transition you will make in programming. Putting millions of print and printf statements in the script not only creates a rat's nest, it is a mess trying to fix errors, and a chore trying to change its looks after-the-fact. Other templating systems have received a lot of traction, and I know not much about them. What I like about H::T is its desire to be good at very, very few things -- kinda like the iPod. And I really appreciate Sam Tregar's (the author of H::T) almost bull-headed insistence (as it seems to me on reading his posts on the H::T list) on not introducing unnecessary logic in the H::T code. It makes for one of the most wonderful pieces of code to work with. Try it. You may never work without it again whether you are creating dynamic or static websites or even munging data in non-web apps.	[reply]
Re^4: Building html site maps by wfsp (Abbot) on Feb 04, 2005 at 19:22 UTC
Re: Building html site maps by punkish (Priest) on Jan 30, 2005 at 15:52 UTC
It is a static web site updated up to 4 times a day. Rebuilding all the site maps every time is easier, in my opinion than trying to update them. Don't quite understand the above sentence. What is the difference between "rebuilding al the site maps every time" and "trying to update them."? That aside, H::T can do the job for you happily whether you want to dynamically build your site on every hit, or you want to build static versions periodically. Just write to a file in the case of the latter. I do that with a website I've built that doesn't get updated that often. One generic template page with TMPL_LOOP within TMPL_LOOP will work happily for you. After that you can muck around with the physical looks of the page as you please. I like H::T for its sheer simplicity, and very little provision for logic on the display side. That helps enforce a clear MVC separation. With a little bit of care, you can eliminate all display-side logic. And, it is very fast. I have looked at HTML::Template and believe that although it could be used I'm afraid it would be very difficult to maintain. Any particular reason behind that thinking you want to share?	[reply]
Re^2: Building html site maps by wfsp (Abbot) on Jan 31, 2005 at 18:45 UTC
punkish Thanks for your quick reply and apologies for my late response (the day job!). "What is the difference between "rebuilding al the site maps every time" and "trying to update them."?" I should have said trying to update one page. It looks like I need to take another look at H::T (but I have a feeling I'll be back with more questions!). Again, many thanks John	[reply]