I googled for "text prediction" and found soothsayer (homepage) as one of the first hits. Sounds like a programming library that does exactly what you're looking for.

Another thing that comes to mind is Dasher, although its novelty is not the text-prediction but its accessible interface. However, its text prediction seems good, it can be trained, and it is fast in real-time. It probably uses some hidden Markov mechanism for prediction. Since it is open source, the prediction mechanism can probably be factored out (if it isn't already) and adapted to your interface needs.

Even if you are determined to do things yourself (yikes), you can at least learn a lot by seeing how the others do it.