I can think of two reasons to use " by default instead of ':

1. To disambiguate when using lots of `s, if your font isn't clear
2. Easier to write text that uses contractions, e.g.

   open(my $fh, '/etc/motd') || die "Can't open /etc/motd: $!";
   [download]

YMMV.