Hi, I have too look after some web sites, many of which are in languages that I don't speak (German, French and so on). I would like to build a site map for each site, listing all the HTML files in a recursive directory manner.

Cut and paste is error prone and slow, plus the sites won't be in English which will make my life a lot harder.

I did a quick search on Google, and pulled up a lot of ancient Perl4 like scripts. I tried CPAN for modules, and found two vague possibles, and found nothing on PM.

Seeing is Perl is my language of choice I'll write my own, unless some one happens to know of a recent one that works okay.

What I need has the following loose requirments:

• Runs as CGI live and/or off-line storing output in a static file. I'm using wget to publish the site, so it can be slow
• Produces modern well-formed xHTML fragments- most of the ones I found on Google don't
• As generic as possible, has as little HTML and "site" encoded in it as possible.

Does anyone have something mostly written? If so may I use it? I'd obviously prefer Perl for this job, but I'd accept PHP if someone happend to have a good mapper.

If not then I'll write my own. I think I'll do the following:

• CGI for the CGI and HTML
• HTML::TreeBuilder to get title tags and META Descriptions - other options?
• File::Find to work over the directory tree
• I will pass the results through the sites XSLT-like templating system, so the script will only produce the barest bones of HTML

I'll write it for simplicity, simply traverse the directories one by one, printing a simple indented HTML structure as I go.

I think this should be easy to build, and simple, I only have a few dozen directories, and a few hundred files at the moment. If someone has done this already, I would be very happy to not re-invent the wheel.

As ever thanks in advance...

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.