For every release I create a directory structure - or rather Module::Starter does.
Although I do copy over all the test files from the previous version to the new version, the test of the Change Log would copy over in that, wouldn't it?

I think I've just answered my own question...