With /e the second half of s/// is an expression, you must manually concatenate your variables. The first works because there is no need to concatenate a single variable (to what?!). It's an easy enough mistake to make.

UPDATE: Note I explicitly did not say use . because you only need effect a concatenation. TIMTWOTDI, you could also use an interpolating quote.

--
perl -pe "s/\b;([st])/'\1/mg"