Handling configuration for multiple modules

jhourcle has asked for the wisdom of the Perl Monks concerning the following question:

I'm currently working on a mid-sized project (4 programmers, with 14 components, in 42 scripts and 53 Perl modules, not including stuff from CPAN). Due to the nature of the application, different servers may have one or more of the components installed.

We've attempted to move server specific variables to a configuration module, from which the other modules can get their necessary information. Each of the modules that use the configuration module need to have a chance to register their variables with the configuration module when they're installed.

... And that's where I'm running into problems.

I currently call an 'AddVariable' function from within Makefile.PL:

        eval {
                require Physics::Solar::VSO::Config;
        };
        if ($@) {
                die "Physics::Solar::VSO::Config not installed";
        }
        Physics::Solar::VSO::Config::AddVariable (
                'REG_DIR',
                desc    => 'location of the registry data.  Will be gu
+essed if left undefined',
                default => undef,
                module  => 'registry',
        );
        Physics::Solar::VSO::Config::AddVariable (
                'REG_DUMP',
                desc    => 'location to store pre-parsed registry data
+, if defined',
                default => undef,
                module  => 'registry',
        );
[download]

(I don't think the actual code for storing the variable is significant to the question, as it's more a logic thing, but if you want, I can provide that, too.)

And so we get to the real problem -- how would you handle the configuration such that it's available for a 'make test', but not seen by any other module until you do a 'make install'?

I'm currently storing the configuration info in an xml file in the same directory as the configuration module:

use XML::Simple;
my $parserOptions = {
        cache                   => [ 'memshare' ],
        NoAttr                  => 1,
        SuppressEmpty   => undef,
};

require Exporter;
our @ISA = qw(Exporter);
our @EXPORT = ();

( my $configFile = __FILE__ ) =~ s/\.pm$/.xml/;
my $config ||= XMLin( $configFile, %$parserOptions );

our %CONFIG : unique = ( %{ $config->{'CONFIG'} || {} } );
our %C      : unique = ( %{ $config->{'C'     } || {} } );
our %PERL   : unique = ( %{ $config->{'PERL'  } || {} } );
our %SHELL  : unique = ( %{ $config->{'SHELL' } || {} } );

sub new { 
    my $class = shift;
    return bless \%CONFIG, ref($class) || $class;
}

sub AUTOLOAD {
    (my $constant = our $AUTOLOAD) =~ s/^.*://;
    no strict 'refs';
    
    if (exists($CONFIG{$constant})) {
        *{$AUTOLOAD} = sub { return $CONFIG{$constant} };
        return &{$AUTOLOAD};
    } elsif (exists($PERL{$constant})) {
        *{$AUTOLOAD} = sub { return $PERL{$constant} };
        return &{$AUTOLOAD};
    } else {
        require Carp;
        Carp::croak "Unknown constant : $constant";
    }
}

# this needs to stay below where %CONFIG and %PERL are populated
our @EXPORT_OK = ( keys %CONFIG, keys %PERL );
[download]

I've trimmed out the majority of the module, but it should give you the basic idea. Basically, we couldn't decide how to access the values, so some are importing them as constants, others create a config object, etc -- and there are functions to export the configuration as XML, C headers, or shell scripts ... %CONFIG holds the language indendent items.

My current thoughts on how to handle the problem include --

Write the new xml file to the local directory, write it into blib when we do a 'make', and have the module determine the path if grep { $_ eq 'blib/lib' } @INC
Write the new xml file to the local directory, and copy it and the config module to blib when 'make' is run.

Are there any better suggestions, either for the narrow problem, or to replace my whole configuration logic?

Comment on Handling configuration for multiple modules Select or Download Code

Replies are listed 'Best First'.
Re: Handling configuration for mutliple modules by kvale (Monsignor) on Apr 08, 2005 at 03:18 UTC
I would go with your second option: create a new XML file during 'make' or 'make test' in the proper direcotry for testing, then copy the XML file to the proper location for installation. Production modules should not have to worry about the test harness. Removing ambiguity in the search path may also reduce headaches later on. -Mark	[reply]
Re: Handling configuration for mutliple modules by tilly (Archbishop) on Apr 09, 2005 at 01:56 UTC
Here is what I did in a vaguely similar situation. I wrote a custom config module. This module had a search path that it knew to follow of places that it could find configuration information. When you called that module's import method, it looked at caller to figure out the module being loaded and then search for the appropriate configuration files. For a module named Foo::Bar, it would look for files named Bar.pc in that path. Each would be read into RAM, a package command would be inserted, and then eval called. When it was done it would then look through the import request, and would export them to the caller's package if they were loaded, plus would tell strict.pm that they were now OK. If they were not loaded I would die a horrible flaming death. To figure that out use the *foo{THING} syntax described in perlref. You'll need to require that all scalars must be defined. The configuration path had a directory for good development defaults, a directory with as little as possible in it for overrides for production (presumably you'd want another for testing overrides), and a directory outside of source control for per developer overrides. Together with a script to load the full configuration and then dump it in various formats, this should meet your needs. Another solution that I've used is to build up a hash of different configurations for different situations where the configurations are built up from each other. Then require people to use special access functions for their configuration data. There are cases where this is more flexible than the other, and cases where the other is more flexible than this. On the whole I think I preferred the first approach - actually having variables exported into your namespace is a nice convenience touch. But it seems that everyone has different desires for a configuration module. Which is one reason why there are so many configuration modules on CPAN, most of which are worse for my desires than raw Perl. Figure out what you want and pick a configuration solution that makes sense. But, absolutely, you must settle on one. Even if it is a configuration solution that you dislike, a mediocre solution used consistently is better than several much better solutions competing with each other in the same code.	[reply]


The stupid question is the question not asked
	PerlMonks