I've never seen !=~ before but I assume it's functional.

... except it doesn't do what you want. What you've got here is a combination of two operatores: != and ~. What you're asking for is to test that $_ is not numerically the same value as the result of m/^#/g, bitwise inverted. You can test my claim with the next snippet:

use constant NOT1 => ~ 1;
print "~1 is " . NOT1 . "\n";
if(NOT1 !=~ 1) {
    print "Different (huh?!?)\n";
} else {
    print "The same (of course!)\n";
}
[download]

which results in:

~1 is 4294967294
The same (of course!)

For any other number than 1 in the conditional, the test says "different".

p.s. Like everybody else seems to have told you already, you're looking for !~. Or, you can use the inverted test, using unless, instead of if