I guess my biggest issue is I am not sure how to load my "plugins" with moose.

I use Module::Pluggable::Object:

package My::Project::HasPlugins;

use MooseX::Role::Parameterized;

parameter 'namespaces', isa => 'ArrayRef', required => 1;

role
{
    my $p          = shift;
    my $namespaces = $p->namespaces;

    has 'plugins', is => 'ro', isa => 'ArrayRef', lazy_build => 1;

    method _build_plugins => sub
    {
        return
        [
            Module::Pluggable::Object->new(
                instantiate => 'new',
                search_path => $namespaces,
            )->plugins
        ];
    };
}

1;
[download]

... and in a class which needs to use plugins:

package My::Project::SomeClass;

use Moose;

with 'My::Project::HasPlugins'
    { namespaces  => [ 'Some::Plugin::Namespace' ] };

...

1;
[download]

You can do this without the parametric role, but it worked really well for my project.