From the high level, Net::OpenSSH::Parallel does mostly what you are asking for.

The issues you may find, besides being SSH based, is that it does not allows distributing a set of N works over a set of M workers. The relation is always N:N.

In other words, you can say, run this task, then this, then this on every host from this set, then join, then run this, etc.

But you can not say, spread this set of tasks over this set of hosts, then join, then ...