There are two ways: first, remove the files by hand (you can use the packlist to see which files need to be removed). Second,

   perl Makefile.PL
   make uninstall
[download]

from the directory you got after untarring the module. However, I think uninstalling isn't well supported, and it might be that it will actually advise you to do the first method instead.

I've asked one of the authors of CPANPLUS for an uninstall mechanism, but I didn't get the impression I could convince the author of the desirability of it.

Abigail