As an example, not as a direct response on your specific goal, one way would be to use a negated character class:

C:\>perl -E "use 5.016; use strict; use warnings; 
 my $str='abXcd{X}efXyz'; 
 my @matches; 
 while ($str =~ /[^{](X)[^}]/g) {push @matches, $1;} 
 say @matches;"
XX
[download]