Yes, and that's exactly what the Makefile does: moves things around to the correct place. The entire set of plumbing is there that knows how to take XS and compile it into a loadable module but it's not used; it's just the "move .pm files to the right place" targets that are run in that case.