Since you've installed perl in your home directory, I presume you're using it to run scripts from the command line. Putting a export PERL5LIB="....", with .... the directories your libraries are in ought to do the trick.

There are directories compiled into the binary (assuming a normal build), but they are only used as defaults. It's actually fine if they don't exist (think about it, when running the test suite between building and installing perl, the directories compiled in don't exist yet either).