but I tried it and it didn't work

Sure it does:

use strict;
use warnings;

use Test::More tests => 1;

my $db = 'a_foo_b';
my $dbname = 'bar';

$db =~ s/_.+?_/_${dbname}_/;

is $db, 'a_bar_b';
[download]

Update: I see from your reply that you had already figured that out. Apologies!