There are some tricks one could use the see what modules are installed most recently.

You can check the "ctime" of those module files. Unless you did something funky to your filesystem, this should be quite a reliable source for candidate modules to uninstall.

And perldoc perllocal lists the contents of perllocal.pod, which ought to be updated with new module additions, including date of installation, with every installation using CPAN, and hopefully, with replacements of CPAN, too. (I think I recall that that PPM, (originally) ActiveState's alternative module installer for Windows, doesn't update it.)