Supposedly, you can use the otherlibdirs Configure variable to push whatever you like onto @INC when you compile perl, but I haven't been able to get that to work.

You can, however, use the "vendor-supplied add-ons" fields--vendorlib and vendorarch--for this purpose. Check out the INSTALL doc.