for (@INC) {
  my $path = "$_/MyModule/Plugins";
  for my $m (<$path/*.pm>) {
     $m =~ s{^\Q$_\E/}{}; # strip off everything leading up to MyModul
+e...
     $m =~ s{/}{::}g;     # replace / with ::
     $m =~ s{\.pm$}{};    # remove extension
     eval "use $m;";      # use MyModule::Plugin::Whatever
     die if $@;           # propagate exceptions
   }
}
[download]

That only searches one of @INC's directories, and it has portability issues. Fix:

use Path::Class  qw( dir );
use Scalar::Util qw( reftype );

my $base_pkg = __PACKAGE__ . '::Plugins';

my %plugins;

my @subdirs = split(/::/, $base_pkg);
for (@INC) {
    next if reftype($_);

    ++$plugins{"{$base_pkg}::$_"}
        for
            grep /^\.pm\z/,
            map $_->basename(),
            grep !$_->is_dir(),
            dir($_, @subdirs)->children();
}

( my $base_dir = $base_pkg ) =~ s{::}{/}g;
++$plugins{"{$base_pkg}::$_"}
    for
        map m{^\Q$base_dir\E/([^/]+)\.pm\z},
        keys(%INC);

my @plugins = sort keys(%plugins);

# Load them all!
for my $plugin (@plugins) {
    eval("require \E$plugin\Q")
        or die("Loading plugin $plugin: $@");
}
[download]

You gotta be careful that the file name case matches the package name case on case-agnostic systems. You've got a problem if there's a caseless system out there. Maybe some heuristics would help (such as scanning the .pm for a matching package statement and use its case).

Scanning %INC is useful for getting around the problem of a package manager creating a virtual file system using a reference in @INC.