If I remember correctly from previous explanations the impact of using one of those variables makes perl copy every string that you match a regex against.

How much that is strongly depend on your application, and might range from "not measurable" to "makes your application crash" (when it runs out of memory).

I find these variables very useful when debugging complicated regex. I put something like this into the regexes at various places: (?{ print "&`«$&»$'\n" }) which shows pretty nicely how far the regex engine proceeded so far. (But you won't find that in production code ;-)

I'd not call it grind its performance to a halt.

The terminology might not be exact in terms of actual performance, but I found that phrase to leave an adequate impression every time the match variable and its kin appear in someone's code.

You might say its fear mongering (and you might be right), but I'd rather exaggerate now, and debate facts after the usage of the match variables has been abolished (another strong word).

Stop saying 'script'. Stop saying 'line-noise'.
We have nothing to lose but our metaphors.

The Devel::SawAmpersand module has a bit more info on why:

There's a global variable in the perl source, called PL_sawampersand. It gets set to true in that moment in which the parser sees one of $`, $', and $&. It never can be set to false again. Trying to set it to false breaks the handling of the $`, $&, and $' completely.

If the global variable PL_sawampersand is set to true, all subsequent RE operations will be accompanied by massive in-memory copying, because there is nobody in the perl source who could predict, when the (necessary) copy for the ampersand family will be needed. So all subsequent REs are considerable slower than necessary.