perl Makefile.PL just creates the Makefile for you, but that Makefile contains targets you can use to create distributions. Some of those targets include:

• tardist - make a .tar.gz
• zipdist - make a .zip
• uutardist - make a uuencoded .tar
• shdist - make a self-extracting shell archive

So make tardist will give you a .tar.gz as you expect (or just make dist, as tardist is the default dist).