If you know (or test) that the first and last characters are quotes then you can use substr to isolate those from conversion:

my $string = '"I have a 15" latter and a 6" foot arm."';
substr( $string, 1, -1 ) =~ s["]["]g;
print $string;

"I have a 15" latter and a 6" foot arm."
[download]

Or, you can use a lookbehind and a lookahead assertion to prevent the first and last characters being touched:

my $string = '"I have a 15" latter and a 6" foot arm."';
$string =~ s[(?<=.)"(?=.)][&quot;]g;
print $string;

"I have a 15&quot; latter and a 6&quot; foot arm."
[download]

BTW. What is a '15" latter'?