If you're using s///, and there's more than one quote per line, I'd use s/"+//g;. That ought to be somewhat faster if there are many quotes in a row.
Of course, typical data doesn't have many quotes in a row, but the general principle hold, if you are deleting sub patterns, s/(?:PAT)+//g is usually faster than s/PAT//g.