http://qs1969.pair.com?node_id=562873

jimt has asked for the wisdom of the Perl Monks concerning the following question:

I have a few modules on CPAN (some older than others), and a few years back I switched from having all of the default information within the modules to instead placing it into separate config files. So the user doesn't need to go hack up the module to get their defaults into place. I consider this a good thing.

But I've never gotten it quite where I think it should be because the user still needs to at least specify the location of the config file. So every time they install a new version, they clobber that pointer to their personal config file and need to re-edit. Admittedly, the user will need to change 1 line instead of 50 (or whatever), but it's still something that needs to be done and adds opportunity to screw something up and waste people's time.

So what's a good way to deal with this? Some ideas I've had about this are...

  1. Always toss the config file into some global location (/etc/conf or whatever) and hope the user doesn't mind me stomping on their file system. Also hope the user has such a global location.
  2. Always include in the same install location as the module and hope the user doesn't mind having it there and can find it.
  3. Somehow prompt the user to choose the location of their config file as part of the make install process. Which leads to the subquestions:
    1. Can this be done? and
    2. How? and
    3. Is it considered good form anyway?
    I've never done more advanced make files like this, so a good make resource would be nice, too. A more dynamic build approach like this also has the advantage of ensuring that an update never stomps on an existing config file.
  4. Remove any default config information from the module and require the user to specify their own config file in their script. Yuck. While I like the idea of a runtime override, I don't want to require it.
  5. Leave it as is. If the code was hard to write, it should be hard to upgrade. This keeps me out of the business of finding their conf file and ensures I never overwrite it, but they need to remember to update their pointer.

So what'd be considered good form for this thing? Any suggestions of existing CPAN modules that handle this in a particularly slick fashion?

Replies are listed 'Best First'.
Re: Config files in CPAN modules?
by blue_cowdawg (Monsignor) on Jul 21, 2006 at 14:51 UTC
        So what's a good way to deal with this?

    One way I've solved this for some modules for general use that I've authored for my job is to use XML::Simple to store my configs. If I have a program that is invoking a module that needs to get to a config file named "foo.pl" I simply create a file in the directory the script lives in called "foo.xml" and put the configuration information in there. For instance I have some database access modules that I use a lot and I store the information as such:

    <disco_database> <host>database-host.foo.com</host> <dbname>disco</dbname> <user>disco</dbname> <password>duck</password> </disco_database>
    in my xml file. The module s written to invoke XML::Simple thusly:
    package my_package; use Class::DBI::Loader; use XML::Simple; | much handwaving... sub new { | more handwaving... my $config = new XMLIn(); $self->{loader} = Class::DBI::Loader -> new ( dsn => sprintf("dbi:%s:dbname=%s;host=%s", $config->{disco_database}->{driver}, $config->{disco_database]->{dbname}, $config->{disco_database}->{host}), user=>$config->{disco_database}->{user}, password=>$config->{disco_database}->{password} | and so forth...
    This has the added bonus that if I have change any of the parameters that the script and its associated modules need to live on change I don't have to search through all the source code of the modules and script to change variables.

    In other cases (and actually, the project I call "disco" is a good example of this) I put a config file that is central for that module. If I ever have to override it I make that a potential parameter for instantiating the module. For instance:

    | blah blah... use IBM::Disco::HostEntry; | } | blah | | my $he = IBM::Disco::HostEntry->new{ config_file => qq(/path/to/file) ); |
    When the module is instantiated it uses a default normally but when the parameter config_file is passed to the constructor I use that value instead.


    Peter L. Berghold -- Unix Professional
    Peter -at- Berghold -dot- Net; AOL IM redcowdawg Yahoo IM: blue_cowdawg
Re: Config files in CPAN modules?
by eric256 (Parson) on Jul 21, 2006 at 15:30 UTC

    Allow the user to sub-class your module and specify there configuration there (either as a reference to a file or the actual configuration

    package My::Whatever; use base Whatever; sub config_file { "/usr/home/whatever" };

    Seems like this gives the most power to everyone involved, even means that i could have several configurations and easy access to each.


    ___________
    Eric Hodges
Re: Config files in CPAN modules?
by philcrow (Priest) on Jul 21, 2006 at 14:37 UTC
    Using Module::Build you can easily implement option 3. <shameless_plug> You could consider Gantry::Conf which gives considerable control to you and the user. </shameless_plug>

    Phil

Re: Config files in CPAN modules?
by eXile (Priest) on Jul 21, 2006 at 18:31 UTC
    Lots of software looks at several places for configfiles. Ie. a systemwide config and a user-specific config. Usually the user-specific config can override the systemwide config (exception is security-type options for instance). User-specific configs usually live in a users home directory (on UNIX as .<appname>rc files, like .cvsrc ).

    My advice would be to consider using a 2 configfile type of approach, make it work in all three situations ( system-config + user-config, system-config + no user-config, no system-config + user-config ) and document what overrides what.

      This suggestion (provide both a system-wide and user-specific config) is very sensible and helps alleviate the central problem, but I don't think it eliminates it. What should the installer do when confronted with an existing system-wide configuration file? Overwriting it is harmless only if all customization has been confined to the user-specific files. Most likely, you're stuck with doing something like silently creating a "config.new" file, and leaving "config" alone. For extra-credit, you could examine the existing config file to see if it differs from the default (compare MD5 hashes?) and overwrite if no customization has been done. And if you want to go above and beyond, you could try to merge your changes into the existing file.

      Note: asking the user about this in the middle of an installation is to be avoided on penalty of death. Think about someone using CPANPLUS to install a module with a cascading series of dependencies involving dozens of modules. What if every other package wanted to politely ask a question about something?

        I like the opposite -- save the original as 'config.orig' or 'config.old', or perhaps 'config.2006-07-21' for example. Then, save the new one as 'config'. It's pretty common, and it gives me a point of reference when I go to customize the new one.

        What's really best, though, is when a package recognizes an obsolete config file and saves a 'config.old', a 'config.defaults', and a 'config' that has all the options from 'config.old' updated to match the new format and adds any settings from 'config.defaults' that used to be hardcoded and now aren't. Preferably, the new config file will be set up with things specified as much like the old version worked even if the default behaviors have changed, and with notes in the config file's comments stating what's changed and why. I can't name a CPAN module that does this currently, but I've used a few server and desktop applications that do this. That's one of the biggest things I like about the CMS I recommend for my clients, actually. When they upgrade it (or hopefully when they pay me to upgrade it), the upgrade is almost seamless.


        Christopher E. Stith
      I think it's important to adhere to the system philosophy of placing config files. Some systems use /etc/, some systems use /usr/share, some systems use C:\USER_SETTINGS. Unfortunately, there is no CPAN module that makes this transparent.
Re: Config files in CPAN modules?
by kwaping (Priest) on Jul 21, 2006 at 14:59 UTC
    My vote goes to option 2. It might be hard to find initially, but both the user and the software will always know where it is (after they find it once).

    I don't like option 1 because I don't see it as being very portable. I don't like option 3 because it introduces the possibility of user error (typo, etc) during install, and it's more of a pain to install. I like no-fuss, no-muss installations and I'm sure I'm not alone. I actually don't mind option 4 very much, but it seems that's not your favorite. Option 5 I think is the worst solution, as you don't want to keep changing the module source just for a config change. Not only is that a big problem if your module gets released into the wild, but also it introduces the possibility of inadvertantly creating syntax errors in the code.

    ---
    It's all fine and dandy until someone has to look at the code.
Re: Config files in CPAN modules?
by fmerges (Chaplain) on Jul 22, 2006 at 10:50 UTC

    Hi,

    Normally what I do is asking the user to provide the configuration, could be the config itself, or the path to the configuration file:

    MyModule->new({ config_file => 'file.conf'}); # or MyModule->new({ config => \%Config }); # or MyModule->load_config($file); # etc...

    This way it's up to the users script to decide which method he uses to provide the modules with the configuration settings.

    Or going for well known places where the config files should be (linux -> /etc or /usr/local/etc) could be with subdir for the application itself or so; or the other way is to create a hierarchy for the application itself:

    MyApp | +-- bin/ +-- etc/ +-- lib/ +-- docs/

    But the most important thing of all is to DOCUMENT it. So that the user know how he must do instead of figuring it out. ;-)

    Regards,

    fmerges at irc.freenode.net
Re: Config files in CPAN modules?
by adamk (Chaplain) on Jul 22, 2006 at 12:57 UTC
    File::UserConfig might be one way to at least handle the initial case.

    It installs to the module's "auto" share directory a default copy of the configuration file/directory.

    When the program runs the first time for a new user, it copies it to the appropriate place.

    For the upgrade case, perhaps what Ruby on Rails does for SQL schema. You have the config in a format you can manipulate, then write a series of classes that upgrade or downgrade from one version to the next.

    Upgrading over a number of releases just means running each upgrade sequentially (with a backup first).
Re: Config files in CPAN modules?
by Anonymous Monk on Jul 24, 2006 at 08:14 UTC
    If you want to prompt, use ExtUtils::MakeMakers prompt function.

    There is no reason to install a default config file.

Re: Config files in CPAN modules?
by hobbs (Monk) on Jul 24, 2006 at 18:49 UTC
    Here's a "Clever" solution ;)
    • Define a config format that begins with "1;\n__END__\n", and continues with whatever sort of config data you like.
    • Have your users put "myapp.conf" anywhere in @INC. Optionally recognize a special environment var (say, MYAPP_HOME) to temporarily push onto @INC, just in case they don't feel like fooling around with PERL5LIB :)
    • In your code:
      { local @INC = @INC; unshift @INC, $ENV{MYAPP_HOME} if defined $ENV{MYAPP_HOME}; require 'myapp.conf'; } open my $conf, '<', $INC{'myapp.conf'} or die 'Flaming death'; 1 while <$conf> ne "__END__\n"; enjoy($conf);
Re: Config files in CPAN modules?
by pemungkah (Priest) on Jul 25, 2006 at 22:52 UTC
    I don't know if you'd consider it "slick" or not, but CPAN.pm does something like this.

    It installs a basic set of config data in CPAN/Config.pm, which can be overridden by data stored by the user in .cpan (which CPAN.pm silently creates and manages).

    Whether this is sufficiently cross-platform enough for you I don't know, but it's certainly an alternative.

    I'd be tempted to suggest Config::Std for config files, except for the fact that it doesn't handle a dynamic choice of config file well (you can always eval a 'use', though).

Re: Config files in CPAN modules?
by Anonymous Monk on Jul 25, 2006 at 08:55 UTC
    Use option 2. And define an include-statement in your format. And your default config should be "include /etc/config" ;-)
    All bases covered.