Try using either $find='C:\\Dir1\\Dir2\\Dir3';

No, that's the exact same as "C:\\Dir1\\Dir2\\Dir3", since single-quoted strings still interpolate a few characters, backslashes being one of them.

The fourth line also interpolates the string.

Not only does it interpolate, it will also interpret any characters special to regexes as such. Your qr// suggestion is good, as an alternative I would suggest s/\Q$find\E/$replace/ to escape the string (see quotemeta).