It doesn't behave that way for me.
The code...

use DBI;
eval {
   $dbh = DBI->connect('dbi:Oracle:dev', 'bad', 'pass', {RaiseError =>
+ 1, PrintError => 0}) || warn "warning";
   print "after connect\n";
};
if ($@) {
   print "Ugh!\n";
}
[download]

will not print out "warning" or "after connect"
it prints out "Ugh!"

# perl testit
Ugh!
[download]