I wouldn't think that writing a basic HTML4 (no JS) browser in Tk would be that difficult.

HTML4 is rather complex, and contains a lot of details that aren't useful for documentation. You almost certainly won't need frames or forms, for starters.

You would probably be better off with a restricted subset of XHTML. <p>, <b>, <i>, <img> and maybe <table> and <ul>/<ol>, <li> are probably all you need. That will be far easier to parse than full (X)HTML.