Do you mean literally "when running Makefile.PL"? It think that shouldn't create those files; you can create the manifest with make manifest and I believe META.yml is created when you do make dist.