It somewhat depends on your interface. If it's easy to escape from the text version of your program, then it may be ok. I would suggest not trying to buck convention too much, so in the case of a fall-back, pop up a message box something like this:

  +----------------------------------------+
  | DISPLAY set incorrectly.  Falling back |
  | to a text interface.                   |
  |                                        |
  |   [ OK ]                 [ CANCEL ]    |
  +----------------------------------------+
[download]

If you provide on-line help, it should describe the use of the DISPLAY variable to bypass this warning (that is, to set it to a valid X server which you're authenticate to use, or to unset it if there is no such valid X server).

The reason for this is that if I really meant to use the X version of your program, I want a quick and easy way out, so that I can properly set DISPLAY, and go back in. If you default to the "OK" button above, the user which has DISPLAY set incorrectly but is ok with the text interface just hits enter, while the one who wants to reset DISPLAY merely has to hit escape. It's a trivial compromise between all positions, IMO.

Another nice-to-have option is a commandline option, say "-tui", which bypasses the DISPLAY check (and the above warning) and simply goes straight into the TUI, regardless of the DISPLAY setting. In other words, this is the user setting DISPLAY for general X use, but telling your program specifically that she wants you to ignore DISPLAY and go directly into an text-based interface.

Along the same lines is a "-gui" option. It means, use DISPLAY, and don't fall back. If DISPLAY doesn't work, just exit with an error.

For completeness, an "-auto-ui" option which does what you want the whole program to do. It's the default. :-)