Uninstalling is a huge pain, but if you still have your perl build directory around, you can try parsing the output of make -n install ("-n" means "print out what you're going to do, but don't do it"). If that doesn't work, you can try removing stuff from /usr/local/bin/, but perl installs a lot of programs.