Because it's a bitwise AND.

123 = 1111011
  4 = 0000100
          ^
[download]

The operator '&' returns a value where each bit of one value is is ANDed against each bit of the other value.

The marked digit is 1 for the value 4, and 0 for value 123. The rest of the digits are 1 for value 123 and 0 for 4. When you do an AND of each bit in this case, there are no bits which are 1 in both values. That's why you get a value with no bits set to 1, which is 0.

It'll help if you stop thinking of '&' and '|' as mathematical operators and thinking of them as boolean logical tests against vectors of bits, which is essentially what they are.

From perlop:

Bitwise And

Binary "&" returns its operands ANDed together bit by bit. (See also "Integer Arithmetic" and "Bitwise String Operators".)

Note that "&" has lower priority than relational operators, so for example the brackets are essential in a test like

print "Even\n" if ($x & 1) == 0;
[download]

Bitwise Or and Exclusive Or

Binary "|" returns its operands ORed together bit by bit. (See also "Integer Arithmetic" and "Bitwise String Operators".)

Binary "^" returns its operands XORed together bit by bit. (See also "Integer Arithmetic" and "Bitwise String Operators".)

Note that "|" and "^" have lower priority than relational operators, so for example the brackets are essential in a test like

print "false\n" if (8 | 2) != 10;