The expression  \&my::module::exit_routine is evaluated at compile time and so, of course, cannot 'take' an argument (although the LogSimple module, to which you offer no link and which doesn't appear on CPAN or MetaCPAN, may provide a way around this; see its documentation). A typical work-around would be something like (untested)
    exit => sub { return my::module::exit_routine(\%modifier); }