How about:

$_ = '((transcription AND factor)NOT(control OR regulat) AND (TATA OR 
+TBP) NOT (BLOOD AND marrow))';

my @caps =  m/NOT   # "NOT"
              \s*   # spaces?
              (     # start capturing
              \(    # "("
              [^)]+ # some not ")"s
              \)    # ")"
              )     # end capture
             /xg;

print join "\n", @caps;
[download]

Output:

(control OR regulat)
(BLOOD AND marrow)
[download]

See perlretut for a tutorial and perlre for the details on regular expressions.