I'm not quite sure got this right but heres my try (both dot and comma is used).

s/(?<=\D)([,.]) *(?=\D)/$1 /g;
[download]

Tested with this.

perl -e '$a="1,23, 23, df,sd.asf,\"\"";$a=~s/(?<=\D)([,.]) *(?=\D)/$1 
+/g;print "$a\n";'
1,23, 23, df, sd. asf, ""
[download]