Well this may soon become of a formidable complexity. For one thing you should do the parsing of the expressions in the first place. I'm not even trying to do anything like that because admittedly I have no experience at all in this field. However as a general direction you may either roll-your-own solution or design a very simplified grammar with e.g. Parse::RecDescent.

IIRC something roughely along these lines had been discussed on clpmisc short before I stopped regularly following it.

As far as the (Tk) UI is concerned, it's simply up to you and to your skills and tastes, however this seems to me like a task well suited for a cmd-line console based UI, so if you really want to go graphical, then probably something as simple as a window with an input field, a text field for output and an update button would suffice.

UPDATE: Now that I re-read this, I realize that you were most probably referring to the formatting of mathematical expressions. Then in practice you're asking about implementing a typesetting engine, even if you may be content with an elementary one. In any case it won't be an easy task. I recommend you give a peek into the (La)TeX world instead. In particular you may be interested in mimeTeX and preview-latex.