Also I would have thought that the numeric operator would have considered it as a string (in which case $a should have been 0+1) and evaluated to 1 or as a number 1334 and evaluated to 1335;

$a with its content "1_1334" is not treated as 0 in numeric context. Perl checks the value and sees that the string starts with a number. So it takes and uses that number in numeric context. That's why your $a + 1 results in a 1 + 1 =2 instead of a 0 + 1 = 1.

Other example:

perl -wle '$a = "7__hello"; print "yes" if $a == 7;'
Argument "7__hallo" isn't numeric in numeric eq (==) at -e line 1.
yes
[download]

But beware: $a = "1_334"; is different to $a = 1_334;. While the first one is a string, which is treated as 1 in numeric context, the second one is a fully valid numeric value 1334. The underscore (_) can be inserted to improve readability.