Data checking configuration files

Amblikai has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks! I have a quick question regarding configuration files

I almost always use Apache style configuration files in partnership with Config::General in my scripts

The thing is, i always end up writing lengthy subroutines or methods to check that the config file is in the correct format or has the data i'm expecting before i use it. I've been known to make this the majority of the length of my script!

Is this madness? What is the general opinion on checking config files are as expected?

Error out upon trying to use an aspect of the configuration if its wrong?

Completely check that everything is in place with the config so you have complete confidence in moving on and using the config data?

Are there any modules out there to help with this or do i just accept that the end user might completely screw up the config data/format and just bail with an error when i don't get what i expect?

Cheers!

Edit for Completeness!

Thanks for all the replies, i looked into Config::Validator but i couldn't work out how to do it should the config have unknown values in it

So long story short, i ended up creating a child class of Config::General and having a "CFG_check_cfg" method which runs all the other little standalone checks i wrote.

The Package being:

package ConfigObjects;
use strict;
use warnings;
use Exporter 'import';
use parent 'Config::General';
our $VERSION='0.1';
our @EXPORT_OK=qw(CFG_check_cfg);
# Main config checker
sub CFG_check_cfg {
  my $self=shift;
  my $return=1; # True by default (all good)
  $return&=CFG_check1($self);
  $return&=CFG_check2($self);
  return $return;
}
# Unit checks:
sub CFG_check1 {
  my $self=shift;
  return (#Some property of the $self->{config} data structure);
}
1;
__END__
[download]

Then i call the config parser in the main script like:

my $cfg_obj=ConfigObjects->new(-ConfigFile => $opt_config_file);
pod2usage(-verbose => 1, -exitval => 1, -msg => "ERROR: Config file ha
+s errors") unless ($cfg_obj->CFG_check_cfg);
[download]

Any feedback would be very much appreciated. Particularly if doing it this way is madness!!

Comment on Data checking configuration files Select or Download Code

Replies are listed 'Best First'.
Re: Data checking configuration files by SimonPratt (Friar) on Sep 17, 2015 at 13:45 UTC
I typically ignore checking config file contents when it is first loaded. Instead I get the functions using those settings to validate the data. This way, if I choose to switch the input origin for the function (such as from config file to user input through a website), I know the input is still being validated before being consumed, without having to rebuild the cart, so to speak. I don't think your way is wrong at all, just different :) EDIT: In addition to the above, moving data validation into the consuming function makes the function complete within itself, which allows for greater code-reuse (if you're that way inclined).	[reply]
Re^2: Data checking configuration files by BillKSmith (Monsignor) on Sep 17, 2015 at 20:03 UTC
I assume that your comments do not apply to validation issues related to the file itself, such as the files existence and syntax. Errors of this type should not propagate into using functions. Bill	[reply]
Re^3: Data checking configuration files by SimonPratt (Friar) on Sep 18, 2015 at 11:42 UTC
Of course not :) Whatever is loading the config file is responsible for validating its own inputs (in this case the file), exactly the same way as other functions consuming the various config values are responsible for validating them.	[reply]
Re: Data checking configuration files by nevdka (Pilgrim) on Sep 18, 2015 at 00:23 UTC
There's Config::Validator, but I haven't had a chance to use it. There's an example using it with Config::General in the documentation.	[reply]
Re^2: Data checking configuration files by Anonymous Monk on Sep 18, 2015 at 00:57 UTC
Or even Config::Model / cme	[reply]
Re: Data checking configuration files by KurtSchwind (Chaplain) on Sep 18, 2015 at 14:00 UTC
I use configuration files a lot. What works for me is to check that the config file exists and immediately exit if it isn't. After that, for any given configuration key/value pair, it's a case-by-case instance. Many times you can have a sensible default and the configuration file is there to over-ride that default. So not the mere non-existence of key/value pair isn't necessarily an error. If a value is required, I die (hopefully gracefully). Hand in hand with config files is good logging. Most any program that is going to use configuration files should have decent logging to the screen and/or a file. I'd definitely log whenever a key was looked for and not found. -- “For the Present is the point at which time touches eternity.” - CS Lewis	[reply]
Re: Data checking configuration files by locked_user sundialsvc4 (Abbot) on Sep 17, 2015 at 11:49 UTC
Within reason, I would keep on doing what you’re doing. You need to check that a key exists `or die()` in a meaningful way. The fact that a script ran to completion should be a positive indication that it did so correctly, because only the script is in a proper position to detect any problem that might occur. If there is something wrong with a settings-file ... or with any other input or output ... it must be the one to say.
Re^2: Data checking configuration files by Amblikai (Scribe) on Sep 17, 2015 at 11:53 UTC
I see your point, i wonder if i can do what i'm doing in a more efficient manner though. Are there any modules that you're aware of that could help? Maybe a direct comparison with an example config or similar? Thanks very much for your reply!	[reply]
A reply falls below the community's threshold of quality. You may see it by logging in.
Re: Data checking configuration files by Amblikai (Scribe) on Sep 29, 2015 at 11:16 UTC
Thanks for all your replies folks. I've edited my post to show how i've ended up doing it. I hope it's not a bad way to do it.	[reply]