The special hash %A:: holds references to the typeglobs, these entries aren't the typeglobs themselves.

No, they are the typeglobs themselves (or other things in some special cases, which means extreme care must be taken when directly accessing a symbol table). But globs are looked up and pointers to them inserted into code that uses variables of that name at compile time. delete does remove the glob from the symbol table hash, but the glob isn't freed if it is referenced by compiled code; any code compiled thereafter will create and reference a new glob, though.

You can see this in action:

$ perl -wle'use Devel::Peek "SvREFCNT"; $A = "foo"; BEGIN { print SvRE
+FCNT($::{A}) } $A = "bar"; BEGIN { print SvREFCNT($::{A}) } delete $:
+:{A}; eval "\$A = 1; print SvREFCNT(\$::{A})"'
2
3
2
[download]

(One refcnt is held by the symbol table; additional ones are added for each reference in the code. The delete removes the glob from the symbol table, but it isn't freed since it still has 2 references.)