I also like perlvars, but only if they work. $\ is the output record separator, which means it is printed after each line (and thus before the next). But this leaves the first line without '> ' and introduces an extra '> ' after the last.

And while I'm here let's throw in yet another wtdi perl -i.bak -pe 'substr($_,0,0) = "> "' file

-- Hofmator