You need to quote your string with \Q\E because it contains | which in a regex means or. What you want is:

$line =~ s/\Q$delimiter\E/\,/g;
[download]

Note that your original code acted as a "match anything" so it interspersed a comma between each character in the target string.