Using Devel::Peek to look further into what is going on is more revealing:

#!/usr/local/bin/perl

use strict;

use Devel::Peek;

my $a = 4294967296;
my $b = 4294967295;
my $c = $a & $b;

print '$a:';Dump($a);
print '$b:';Dump($b);
print '$c:';Dump($c);

Results:

$a:SV = PVNV(0x1abe608) at 0x1ab30a0
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,IOK,NOK,pIOK,pNOK,IsUV)
  UV = 4294967295
  NV = 4294967296
  PV = 0
$b:SV = IV(0x1ab2420) at 0x1ab3058
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,IOK,pIOK,IsUV)
  UV = 4294967295
$c:SV = IV(0x1ab2428) at 0x1ab3088
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,IOK,pIOK,IsUV)
  UV = 4294967295

4294967296 has overflowed the integers on the platform I am using, and is appearing as 0xFFFFFFFF.

Whether this is expected behaviour or not is maybe upto Larry or p5p.