From perldoc perldata:

String literals are usually delimited by either single or double
quotes.  They work much like quotes in the standard Unix shells: dou-
ble-quoted string literals are subject to backslash and variable sub-
stitution; single-quoted strings are not (except for "\'" and "\\").
[download]

The allowed backslash substitutions are self-explanatory: although people would better use alternate delimiters, it is apparent that for completeness and consistency the possibility of backslashing "'" is given, and once that is provided, it is obvious that the backslash itself needs a backslash substitution as well.