In case the last step of the build process (i.e. make install) was successful, you no longer need the temp directory, so feel free to delete it.

Whether you need to set up any paths etc. depends on where you installed it to, and on the details of the particular module. For example, if the module depends on a third party library, and the respective shared object(s) have been installed in a non-standard location, you might need to set up/adjust shared library search paths.

But why not just try and see if it loads properly?  In many cases, a simple

$ perl -MCrypt::Whatever -e1
[download]

will do (i.e., if you get no error, everything is fine).  (There are also modules which load some parts at run-time (on demand); for those you'd in theory need to write a little test script that exercises all required functionality, in order to tell if things are in fact set up properly...)