Try this:

#!/usr/bin/perl
use strict;
use warnings;

my $x = '((A & B)\' | (A & C & (A & B & D)\'))'; 

print "Given: $x\n"; 
my ($find)= $x=~/([^()]+?)\)/;
print "FInd:$find\n";
$x =~ s/$find/!$find/g; 

print "Calculated: ",$x,"\n";
[download]

Prints:

Given: ((A & B)' | (A & C & (A & B & D)'))
FInd:A & B
Calculated: ((!A & B)' | (A & C & (!A & B & D)'))
[download]

I could not understand the logic of your substitution requirement.