Another possible way is to just use looks_like_number

looks_like_number() will tell you that both 1 and "1" look like a number, but the examples you gave using the B module would suggest that you're looking for something that distinguishes between 1 and "1". You can use XS or Inline::C to examine the flags of the variable if you want. (It looks to me that this is all that B is doing, but I haven't looked at the code).
With Inline::C:

#! perl -l
use Inline C => <<'EOC';

SV * is_IV(SV * x) { /* integer */
  if(SvIOK(x)) return newSVpv("yes", 0);
  return newSVpv("no", 0);
}

SV * is_NV(SV * x) { /* float */
  if(SvNOK(x)) return newSVpv("yes", 0);
  return newSVpv("no", 0);
}

EOC

print is_IV(1);   # yes
print is_IV("1"); # no
[download]

However, I would've thought that B (once loaded) would perform the same checks just as quickly.

Cheers,
Rob