It is my understanding that a line like string =~ s/"//; should replace all occurrences of " with nothing.

That is incorrect. You need to use the g modifier to globally replace all quotes (s)

$string =~ s/"//g;
[download]

Your code only replaces the 1st quote.