@xreflines=<XREF>;
#remove the blank lines from the araary
@xreflines = grep /\S/, @xreflines;
[download]

You didn't chomp to remove newlines.

    $x=grep /$xrefvalue/, @xreflines;
[download]

if @xreflines contains the values 30368021983, 30368021984 and 30368021985 and $xrefvalue contains the value 3036802198 then it will match all three, incorrectly.

As Corion points out, it is better to use a hash:

my %xreflines = map { /(\d+)/, 1 } <XREF>;


    unless ( exists $xreflines{ "$DISTID1$CUST1" } ) {
        print LOG "\$x value is $x \n";
        print LOG "\$line is $line \n";
        print XFILE "$line \n";
    }
[download]