I see a lot of code in http://search.cpan.org/src/DAPM/perl-5.10.1/ext/B/B/Xref.pm that is careful to skip sections of code that might call ->NAME() if the class to be used would be "SPECIAL". Indeed, line 162 falls at the end of this chunk of code:

        next unless class($valsv) eq "GV";
        next if class($valsv->STASH) eq 'SPECIAL';
        # these pad GVs don't have corresponding names, so same @pad
        # array can be used without collisions
        $pad[$ix] = [$valsv->STASH->NAME, "*", $valsv->NAME];
[download]

Which seems to make it impossible for either $valsv or $valsv->STASH() to be objects from B::SPECIAL. But you don't say what version of B::Xref you are using so perhaps line 162 is from quite different code for you, but that seems unlikely given that you are using 5.10 and that line 162 actually does contain attempts to ->NAME().

I'd look at the code involved in more detail to figure out how those checks are managing to fail to protect against attempts to use B::SPECIAL->NAME(). I'd start by trying to use "perl -d" for that, but some seem to have pronounced difficulty with that and the "CHECK failed" error might indicate that it would be more challenging than usual in this case. But you can also just add some debugging print statements to the B::Xref code.