Are you sure you only want these six characters escaped? What about properly escaping the backslash itself?

# escape using quotemeta:
say "\Q$dir\E";
# escape only those six characters plus "\\":
$dir =~ s/(?=[][{~*?\\])/\\/g;
say $dir;
[download]