In a way I hate it when I post a question then manage to answer it myself 10 minutes later. :) I managed to a.) read Programming Perl (LOL), and b.) cut out the recursion:
while ($text=~m/(( \{+) .*? (??{ '\}' x length $2 }))/xg)
{
$text=~s/\Q$1\E/ /;
}