I would recommend treading carefully with the 'set -o vi' as it can be a real pain if you are not prepared for it. For example, if you are used to pressing the up arrow for your command history, set -o vi will take this away. You will have to use your vi keys for this (escape k and j for up and down). I prefer 'set -o emacs' to allow me to use the arrow keys for my history.
As for the ^H, as others have suggested stty erase ^H should fix the problem in your current terminal. You should be able to set it in your .bashrc, so that it will be persistent across sessions.
Comment on Re^2: STDIN Odd Bevahior in Linux Environments
vi-mode in zsh 4 works very much like in a vi-like editor (after setting 4-5 custom key bindings for insert mode). In ksh93, bash 3 (& below), it is best to stick with emacs-mode as vi-mode support is horrrid with too many exceptions (wrt an editor). Situation may have changed in bash 4, but I doubt that.