The best way when adding tests late in the process is in my experience to write an automated test for every new bugreport you get. That way, you automate the (soon to be tedious) retesting and prevent embarrassing regressions.

The unfortunate thing about writing your first test is, that you will find the current internal API of the program to be unwieldly to test, as in general subroutines will be too long and will do too many things at once. For example, a subroutine will likely fall back to hardcoded defaults instead of getting them as parameters, or it will only read them from a hardcoded configuration file instead of that file being an optional parameter. Making such things optionally configurable with the resulting API remaining essentially unchanged is the interesting part that will take you a relatively long time when introducing your first tests, but I find that the safety against regressions pays off.

And there is little more annoying for your users than them finding regressions in your application.