Not just parens, braces and brackets have this property. "<>" is considered such a pair as well. And there's no need for the second pair to be the same as the first one. These are all equivalent:

    s/foo/bar/;
    s!foo!bar!;
    s(foo)(bar);
    s{foo}{bar};
    s[foo][bar];
    s<foo><bar>;
    s{foo}[bar];
[download]

And: the pairs nest poperly. That means that you can use properly nested pairs inside your delimiter pair as well. The same goes for q()/qq(), to which it very similar in rules &mdqh; and that's no coincidence. So even this will work:

    s[(^\d+[.]\s.+)]<<h1>$1</h1>>g;
[download]

(provided the original does what you want. ;-)
For any other, unpaired delimiter inside your string, you'll have to escape it, by prepending it with a backslash.

For the official docs, see quote and quote-like operators in perlop.