As I recall, I got the list of characters to quote from the bash man page somewhere

On systems with sh, sh (not bash) is used (even if the user's default shell is bash). You need to consult the sh man page. On FreeBSD, the man page for sh says:

Enclosing characters within double quotes preserves the literal meaning of all characters except dollarsign (`$'), backquote (``'), and backslash (`\'). The backslash inside double quotes is historically weird. It remains literal unless it precedes the following characters, which it serves to quote: $  `  "  \  \n