"In general, what are the implications between Foo::Bar and Foo::Baz as opposed to Foo::Bar and Foo::Bar::Baz"

Foo::Bar and Foo::Bar::Baz are within the same second level namespace hierarchy. That means you can have module files under the same directory structure:

Foo-Bar
    |
    /lib
      |
      /Foo
         |
         |--Bar.pm
         |
         /Bar
            |
            |--Baz.pm
[download]

The former, you'd be taking up two slots at the top level namespace, which typically isn't desired from the CPAN perspective unless it coincides with other work. Always try to fit what you have into an existing namespace if possible, and try to keep things contained, instead of spreading them out. For instance, what you're trying to do seems like it would fit under the Net:: top level namespace. If that's the case, a consideration would be:

Net::Circumstance::Automate; # main module
Net::Circumstance::Automate::Command;
Net::Circumstance::Automate::DB;
Net::Circumstance::Automate::Config;
Net::Circumstance::Automate::SSH;
Net::Circumstance::Automate::RDP;
[download]

Then, in your main module, you could suck in all of the lower modules, and return new objects based on parameters to the constructor call, or have separate methods depending on what needs to be done. The config and db modules could be called early on within a constructor to set things up for any following calls.

The dir structure would look like this:

lib/
   - Circumstance
   |- Automate
   |   |- Command.pm
   |   |- Config.pm
   |   |- DB.pm
   |   |- RDP.pm
   |   |- SSH.pm
   |
   |- Automate.pm
[download]

You could also set things up so that you have full-blown inheritance, so that your end-user directly asks for a sub object, and the lower-level items pull in methods and configuration based on its parents, and things may look differently.

When designing something like this that is *bound* to expand exponentially, laying it out in a way that will be extensible is paramount. My suggestion is to start small, use Module::Starter to get set up, muck about, then start your unit tests. While writing tests, you'll quickly find out if your layout and/or your structure makes sense for going forward.

There is no one answer for this, as you are very quickly going to realize. Test, play, scrap, test, play scrap.

The one piece of advice I do have here that my fellow Monks will all agree on, is use a Version Control System (VCS). Github is free, as is Bitbucket. Spending the time to learn git (or hg or svn etc) now will save you enormous amounts of time going forward.

Thank you very much. I didn't have much trouble finding the documentation to write my first module, but finding guidelines on how to organize modules in relation to each other has proven more elusive. <cue the flood gates!> And now that I've rolled one by hand, I'll definitely give Module::Starter a try. So, thank you again.

Version Control. Yes. I'm a zealot actually. I've been religiously stuffing my whole life into various VCS's since the days when RCS was an exciting improvement to SCCS. :)
But always good advice, and never over stated.

I do have another question about the integration of my module hierarchy into the greater CPAN tree, but I'll follow up after I've done a little more research and can ask an intelligent question.

Thanks again,
cbeckley

" For instance, the line where I access the array of commands: for my $cmd (@{$ctxt->{commands}}) { ..."

That depends. In one of my distributions where I knew extensibility would be required, I built an "engine" type system. It was filter based, so I had these phases:

- pre processor
- processor
- post processor
- engine
[download]

That compromised the "core" of the project. How I designed it was that the engine was a part of the core, where the "processor" phase was at its root (the absolute core), and input to it needed to be in an expected format, and output from it would be in an expected format. What the pre-proc did before it reached the proc phase was irrelevant (so long as the proc phase got the proper formatted data), and the output from the post proc and engine was irrelevant as well, as the upper-level methods would output the data, according to the documentations spec.

How I accomplished this, was to create hash tables containing subroutine references to various functionality methods, then 'register' those routines within the main object.

Here's a public method I made available for development and convenience that lists the non-private 'registered' engine methods:

sub engines {
     
    trace() if $ENV{TRACE};
 
    my $self = shift;
    my $module = $self->{namespace} . "::Engine";
    my $engine = $module->new;
  
    my @engines;
 
    for (keys %{$engine->_dt}){
        push @engines, $_ if $_ !~ /^_/;
    }
    return @engines;
}
[download]

The Engine module initialization code (new()), along with the dispatch table that links a command to an actual function:

sub new {
     
    trace() if $ENV{TRACE};
 
    my $self = {};
    bless $self, shift;
 
    $self->{engines} = $self->_dt;
 
    return $self;
}
sub _dt {
     
    trace() if $ENV{TRACE};
 
    my $self = shift;
 
    my $dt = {
        all => \&all,
        has => \&has,
        missing => \&missing,
        lines => \&lines,
        objects => \&objects,
        search_replace => \&search_replace,
        inject_after => \&inject_after,
        dt_test => \&dt_test,
        _test => \&_test,
        _test_bad => \&_test_bad,
    };
 
    return $dt;
}
[download]

...here's an example of an Engine itself:

sub all {
     
    trace() if $ENV{TRACE};
 
    return sub {
         
        trace() if $ENV{TRACE};
 
        my $p = shift;
        my $struct = shift;
 
        my $file = $p->{file};
 
        my @subs;
 
        for my $name (@{ $p->{order} }){
            push @subs, grep {$name eq $_} keys %{ $struct->{$file}{su
+bs} };
        } 
 
        return \@subs;
    };
}
[download]

...and finally, the _core() code that looks for the engine specific code and loads it, whether it's an already registered engine, or a user-supplied code reference of a new/custom one:

sub _engine {
     
    trace() if $ENV{TRACE};
 
    my $self = shift;
    my $p = shift;
    my $struct = shift;
 
    my $engine
      = defined $p->{engine} ? $p->{engine} : $self->{params}{engine};
 
    if (not $engine or $engine eq ''){
        return $struct;
    }
 
    my $cref;
 
    if (not ref($engine) eq 'CODE'){
 
        # engine is a name
 
        my $engine_module = $self->{namespace} . "::Engine";
        my $compiler = $engine_module->new;
 
        # engine isn't in the dispatch table
 
        if (! $compiler->exists($engine)){
            confess "engine '$engine' is not implemented.\n";
        }
 
        eval {
            $cref = $compiler->{engines}{$engine}->();
        };
 
        # engine has bad func val in dispatch table, but key is ok
 
        if ($@){
            $@ = "\n[Devel::Examine::Subs speaking] " .
                  "dispatch table in Devel::Examine::Subs::Engine " .
                  "has a mistyped function as a value, but the key is 
+ok\n\n"
            . $@;
            confess $@;
        }
    }
 
    if (ref($engine) eq 'CODE'){
        $cref = $engine;
    }
 
    if ($self->{params}{engine_dump}){
        my $subs = $cref->($p, $struct);
        print Dumper $subs;
        exit;
    }
 
    return $cref;
}
[download]

Now, when someone calls the all() method on the main object, this is what is executed:

sub all {
     
    trace() if $ENV{TRACE};
 
    my $self = shift;
    my $p = $self->_params(@_);
 
    $self->{params}{engine} = 'all';
     
    $self->run($p);
}
[download]

...run() is the method that actually instigates the whole process. The above all() is the most simplistic as it gets. If more phases were involved, this method kicks it off, it'll process through all of the phases, then gets it back for return to the method that originally called it for return to the user:

sub run {
 
    trace() if $ENV{TRACE};
 
    my $self = shift;
    my $p = shift;
 
    $self->_config($p);
 
    $self->_run_end(0);
 
    my $struct;
 
    if ($self->{params}{directory}){
        $struct = $self->_run_directory;
    }
    else {
        $struct = $self->_core;
        $self->_write_file if $self->{write_file_contents};
    }
 
    $self->_run_end(1);
 
    return $struct;
}
[download]

Then, when _core() is called, it iterates through the phases, calling each one at a time (if they are needed), and in this case, it'll eventually see an engine is located. This is how that phase is executed. $engine is the subroutine callback (code reference) specified, and that's how we can call it as a function like below.

    my $engine = $self->_engine($p, $subs);
 
    if ($self->{params}{engine}){
        $subs = $engine->($p, $subs);
        $self->{write_file_contents} = $p->{write_file_contents};
    }
[download]

After that, there's further processing, but since "engine" is the last phase, eventually, $subs gets put back into $struct, and when things are done, that is what is returned back to run(), which is returned back to all() which is returned back to the user caller.

So, that was kind of fun going back through some of the code I wrote that I had fun designing that allowed the shifting in and out of code blocks for different purposes.

That's just one way to make things a bit modular. Note, I said one way. There are many, and the deeper and more complex you get, the more convoluted your code may become, unless you have a great test suite, and a whole crapload of good documentation. Those two things are key to building a system that you're looking at approaching :)

fwiw, the code above belongs to my Devel::Examine::Subs.

update: Good thing is when you go back and examine your code a few years later, you can see a whole lot of "NOPE" that you wouldn't do now and it looks out of place, which means refactoring ;)

I was going to suggest looking at Rex and also suggest avoiding Net::FullAuto, the horror (some context, and do more digging).

Definitely, definitely, definitely avoid Net::FullAuto.

More context.

Thank you both, I will certainly give Rex a look.

I can't overstate the value this forum provides regarding module advice.

First, for the suggestions of applicable modules. This isn't the first time I've come up dry searching CPAN both through CPAN and google.

Beyond that, and far more valuable, the inside knowledge that comes along for the ride: "theres X and Y but avoid Y" or "X, Y, and Z, but Z is unsupported and only Y installs easily on your OS."

Amazing.

Thanks,
cbeckley

I've written a module, nascent, that makes it a little easier to execute a list of commands on a group of servers. Basically I create an array of servers, which I refer to as contexts in the code, each of which has an array of commands and all the commands get executed in all the contexts and all the output is gathered and returned for processing.

That's very similar to what Net::OpenSSH::Parallel does. The mayor difference is that it doesn't capture the output directly, you have to save it into files that you can read afterwards for processing.

Thank you very much for the suggestion. I've looked through Net::OpenSSH::Parallel now, and it does seem to do a lot of what I'm looking for. I've always been rather averse to leaving execution artifacts behind, but if it gets the job done, I'll get over it!

Thanks again for taking the time.

Thanks,
cbeckley

Initially, I did not include in the module the capacity to capture output directly in order to simplify its implementation and also because I was not able to imagine an API supporting it that would make sense while still being simple to learn an use...

Over time, I have come to appreciate that way of working. Using the file-system to store the generated data escalates quite well, allows easier debugging and in most cases, the performance penalty is negligible (usually the bottleneck is in the network). Also, the mechanism is easy to understand and to use.