Specifically, the usual idiom is to use FindBin with a "use lib" call, like this inside your *.t code:

use FindBin qw( $Bin );
use lib "$Bin/lib";
use My::Special::Purpose::Test::Code qw( :all );
[download]

This is the usual way to specify a module location relative to the script location.