but I lost the extra stuffs that 'lib' does (e.g. appending arch/version/auto dirs)

lib.pm is open source, so Use The Source Luke

But wanting to do this almost doesn't make sense :)