Something else is happening in your code that isn't demonstrated here, the only way you could get a directory named '$scalar' is if either you used single quotes when creating $dir_name, preventing the variable from getting interpolated, or the variable $scalar contains the word $scalar. If $scalar were simply empty, the code you have here would be checking /path/ and not creating the directory.

If this is a unix machine you should be able to remove that directory by using rm ./\$scalar, or perl -e "unlink('$scalar');"