I'm not sure how to do it in a single regex, but here's one way to delete all except the last semicolon (even if it is not the last character in the string):

use warnings;
use strict;

my $string = "I have multiple ; in my string; however I want to keep t
+he last one;";
my $count = $string =~ tr/;//;
$string =~ s/;// for 1 .. $count-1;
print "$string\n";

__END__

I have multiple  in my string however I want to keep the last one;
[download]