One approach might be to extend the SKIP-FAIL trick for excluding URLs to also exclude dates:
    $text =~ s{ (?: $url | $date) (*SKIP) (*FAIL) | ($digits_3_4) }{...}xmsg

Of course, this leaves you with the headache of trying to define a regex to match every possible format of date that a human bean might imagine. Here's a start, but please be aware that this code is untested and also that the  $date regex does not nearly cover every possible permutation of day-month-year ordering or the many internal separator sequences that might be used; you will have to extend this defnition as needed. (Also note that the  $yr pattern is limited to the 21st century.)

my ($month_name) =
  map  qr{ (?<! [[:alpha:]]) (?: $_) (?! [[:alpha:]]) }xms,
  join q{|},
  map  { $_, ucfirst($_), uc($_) }
  map  { $_, substr $_, 0, 3 }
  qw(january february march april may june
     july august september october november december)
  ;
;;
my $month_number = qr{ (?<! \d) (?: 0? [1-9] | 1 [012]) (?! \d) }xms;
;;
my $month = qr{ $month_name | $month_number }xms;
;;
my $day = qr{ (?<! \d) (?: 0? [[1-9] | [12] \d | 3 [01]) (?! \d) }xms;
;;
my $yr = qr{ (?<! \d) 20\d\d (?! \d) }xms;
;;
my $sep = qr{ [-/] | \s+ }xms;
;;
my $date = qr{
  $yr    $sep $month $sep $day
  |
  $month $sep $day   $sep $yr
  |
  $day   $sep $month $sep $yr
  }xms;
[download]

BTW: You have never said if your version of Perl is 5.10 or later, so I don't even know if the SKIP-FAIL trick is possible for you. What version of Perl are you using?