In the .c file generated from the .xs files,

(void)newXSproto_portable("TagLib::FileRef::new", XS_TagLib__FileRef_n
+ew, file, ";$$$");
[download]

should be

(void)newXSproto_portable("Audio::TagLib::FileRef::new", XS_TagLib__Fi
+leRef_new, file, ";$$$");
[download]

This is because every MODULE= and PACKAGE= directive is wrong.

Replace every instance of
MODULE = TagLib
with
MODULE = Audio::TagLib

Replace every instance of
PACKAGE = TagLib::...
with
PACKAGE = Audio::TagLib::...