Well curiosity won!

return causes an error and "won't stay shared" lexical variables are problematic too.

Though this works and can be extended:

use strict;
use warnings;


my $start;
my $diff;
my $timeout;


sub tst {
    $timeout=shift;
    my $str = "a"x10000;
    $str   .= "b";
    $start = time;
    $str =~ /^ (( a*
           (?{
               $diff= time-$start;
               die "stopped after $diff sec"
                  if $diff >=$timeout;
           })
           )*)*
       $/x;
}

tst(10);
[download]

output:

Complex regular subexpression recursion limit (32766) exceeded at time
+out_regex.pl line 15.
stopped after 10 sec at (re_eval 1) line 3.
[download]

update

NB: this approach considerably slows down the regex when done in the innermost group-loop, outer loops OTOH might be checked too seldom. YMMV