[one_server]
save_dir = /var/spool/bla/files

[one_server ABC]
dir = /var/spool/gugus/bla/save
glob = .
archive = p2gspsam.gsp_toAMS.save.tar.gz

[one_server XYZ]
dir = /var/spool/kw4711/bla
glob = .
archive = kw4711.bla.tar.gz

[one_server DEF]
dir = /var/spool/quatsch/bla/save
glob = .
archive = quatsch.bla.save.tar.gz
[download]

my $cfg = Config::IniFiles->new( -file => "./$p_cfgfile" );
foreach my $group ($cfg->Groups) {
    ##  do something with: $cfg->val($group, 'save_dir');

    foreach my $member ($cfg->GroupMembers($group)) {
        ##  do something with: $cfg->val($member, 'dir');
        ##  do something with: $cfg->val($member, 'glob');
        ##  do something with: $cfg->val($member, 'archive');
    }
}
[download]

If you want to keep the description "intuitive" (for the users of it, who have enough domain knowledge I assume), I think you will have to put "sensible" logic into your program.

For example, you should do away with the user-configurable listing of ftp_instances and dynamically generate it from each ftpXX section - if there are three ftpXX sections, you set ftp_instances to 01 02 03.

I can't recommend YAML as a configuration file format, as it is very sensitive to whitespace at the end of the file.

You could look at how fetchmail "structures" its config file format, but an .ini-style format isn't too bad either. Also consider the possible errors the users can/will make, when you phone in the changes they need to make - whitespace (except maybe newlines) shouldn't matter too much, and the script should output very good diagnostic messages when it encounters stuff it doesn't understand (and also output a log file for post mortem analysis). It should have a facility to comment out a single line. Putting whole sections into comments is an interesting thing but unnecessary in my opinion, and it creates too many nonobvious problems, if for example both, the comment begin and the comment end are outside of the displayed area.

Which is/would be a more typing for the user, and thus more to get wrong..

This isn't always a bad thing ... in fact, it's frequently a good thing.

By having a format with a little extra markup, you help reduce the number of missunderstandings about what was intended. In simple formats (like KEY=VAL) how does a person represent a value with a newline in it? or what about a key with an equals sign in it ... or a space?

The user might try these things, and your program might run just fine without any errors, because what they've given you is a valid file, it just may not be valid in the way they think it is. By having a few mandatory requirements that require a little additional typing, you can help ensure that there isn't any confusion.

Your INI file approach already does a little of this ... assuming you are validating that the only sections in the file are [main] and [ftpNN] where NN appears in the list of ftp_instances, and that everything in the list has a section -- that way you reduce the risk that they add an [ftp99] section without adding it to the list. But you should also worry about the possiblity of duplicate sections (happens a lot when people cut/paste key/val pairs and change the value but forget to change the key)

The biggest downside to an INI type solution is that it does require them to duplicate data ... you could avoid that by saying that the set of sections determines the number of FTP hosts, and eliminate the "ftp_instances" list completely -- but that's probably not very practical if you want to have lots of sections for lots of sets of config info.

Which is where XML style config files come in handy ... the biggest complaint against XML configs is duplicate markup; but nestability, quoting, and DTDs make it a really great choice to eliminate confusion and identify mistakes...

<config>
  <root>/path/to/root</root>
  <files>/path/to/files</files>
  <ftps>
     <instance id="00">
       <source>/blah/path/blah</source>
       <save>/blah/path/blah</save>
     </instance>
     <instance id="01">
       <source>/blah/path/blah</source>
       <save>/blah/path/blah</save>
     </instance>
     ...
  </ftps>
</config>
[download]

Sorry, but if you have to make manual changes to such a file, and have a non-XML-competent user, this will become really ugly. You can't "phone in" any changes to such a file, and there is far too much unnecessary fluff that clutters the relevant data - you can't easily put a whole section into comments unless you use the XML comments, which will lead to confusion regarding which section is active or inactive.

Hmm, sorry, I disagree. The XML looks even worse for a human to edit, sure, its readable, but to create by hand? Yuck.

OTOH you did give me another idea: The one with leaving out the ftp_instances variable, and just getting the user to give information for each ftp instance. I can create the value of ftp_instances dynamically, the IDs too, since they just need to be unique.

Another way to do it would be to have variables with a list of values for each ftp_instance variable, but I think that's uglier:

hostnames=one,two,three
sourcepaths=/path/one,/path/two,/path/three
[download]

(and less usable)

As for your other concerns, the variables can't contain newlines, the split only splits on the first '=' and variable names cant contain an '=' or a space. (These are unix environment vars, so I'd be quite surprised if that even worked, the config files contain mostly a bunch of "export VAR=X", anyway, even if it was possible, this software doesn't have any).

These won't be idiots, that will be running this, after all, they usually install the software by editing all of it's config files, and getting them lined up with each other. I'm just trying to make the process easier, without making it more complex in the process.

Duplicating data is not a bad thing, if it provides more clarity.. IMO

C.

ConfigReader::Simple is nice.

# you can include comments
ftp1_param foobar
ftp1_root  /local/foobar
ftp1_string "this is foobar"
# 
ftp2_param foobar
ftp2_root  /local/foobar
ftp2_string "this is foobar"
[download]

You can even use multiple files, one for program settings and one for user settings, so they have less chance of screwing something up, or in your case you could create a separate file for each ftp.

"root"          => "/path/to/root",
"filepath"      => "/path/to/files",
"ftp_instances" =>
[
    {
        "sourcepath" => "/source/path/on/server",
        "savepath"   => "/save/files/here",
    },
    {
        "sourcepath" => "/source/path/on/server",
        "savepath"   => "/save/files/here",
    },
]
[download]

eval "%CFG = (" . readfile("cfg") . ");";
[download]

eval "%CFG = (" . readfile("cfg") . ");";

Is there any specific reason for not using my %CFG = do 'cfg'; instead?

Juerd # { site => 'juerd.nl', plp_site => 'plp.juerd.nl', do_not_use => 'spamtrap' }

if any, then because i didn´t know it could be done that way ,-)

I'm using the approach with Config::IniFiles. A lot of people are "Windows-wise" and able to edit an ini-file, while directed over the phone.

Non-printable characters are represented bei some escapes like \n for a newline or \s for a blank in the right hand value. Even my most ... um ... inexperienced users have learned those escapes.

Since my *nix boxen live within an ADS-environment, I'm looking into something like Bundle::Net::LDAP for (mis-)using the existing ADS/LDAP as a general authentification/configuration-tool. I've no practical experience with that approach yet.

BTW, it's one of the goals evaluating a LDAP-based solution to restrict the ability for configuration to some users -- i.e. teamleaders -- who have also the rights to reset password, assign printers and such.

root: '/path/to/root'
filepath: '/path/to/files'
ftp_instances:
  - sourcepath: '/source/path/on/server'
    savepath: '/save/files/here'
  - sourcepath: 
    savepath: 
  - sourcepath:
    savepath:
[download]

Reading that YAML file returns this perl structure:

$VAR1 = {
  'ftp_instances' => [
    {
      'savepath' => '/save/files/here',
      'sourcepath' => '/source/path/on/server'
    },
    {
      'savepath' => '',
      'sourcepath' => ''
    },
    {
      'savepath' => '',
      'sourcepath' => ''
    }
  ],
  'filepath' => '/path/to/files',
  'root' => '/path/to/root'
};
[download]

I prefer YAML, also. It has the high signal-to-noise ratio of traditional config files, while also providing an elegant way of representing heirarchical data -- even quite complex data. And the reference support is cool, too. And I like Ingy's YAML module.

But then, I tend to see the world monochromatically... I tend to grab one tool (a hammer) and view the rest of the world as a nail...

And while I'm thinking about it, I'll reference a warning in this thread: 347905

1. perl config file
2. Config::ApacheFormat or Config::IniFiles
3. XML with XML::Simple

Second rule is that it should be simple, which again would rule out XML, Yaml and Perl code.

I might format the config file like this:

root     = /path/to/root
filepath = /path/to/files

server = ftp1
    sourcepath = /source/path/on/server
    savepath   = /save/files/here

server = ftp2
    savepath   = /files/go/here
    sourcepath = /some/path/somewhere

Some parsers I've used are simply line based, using hardly anything more than split. Sometimes, I write a Parse::RecDescent grammar.

I quite like this suggestion, since it still just involves X=Y lines and not sections, and if I give the variables meaningfull names (similar to how they are referenced in the documentation), it will probably work quite well.

I agree, by the way, the config file should be as simple and liberal as possible, and accept a variety ways of doing things, even if it makes the parsing code complex. (It should be easy for them, not me), which is why I'm also ruling out using Perl, XML, YAML and so on.

Thanks, C.

I like curly-bracket block formatting, myself:

main {
    root=/path/to/root
    filepath=/path/to/files
    ftp_instances=01 02 03
}

ftp01 {
    sourcepath=/source/path/on/server
    savepath=/save/files/here
}
[download]

If you've never used Parse::RecDescent before, the above is a great way to learn. The grammar for the above is fairly simple.

I don't really like the look of XML files, but they do have the advantage that, using XML::Simple, most of the work is already done for you.

One thing I dont get about this is why you need the "ftp_instances" key at all. I'd just have a bit of code that assumed that all Ini file sections whose name matches /^ftp/ are valid instances. Its easy enough to loop through the sections as needed, and it means that when somebody decides to delete one of the sections they dont have to remember to update the instances list, and it also means that if the users want to give the instances more descriptive names they can.

Also, an alternative for things like this is to use a DB for the config data. Of course that still leave the problem of where to put the DB connection details but allowing the user to manage their config through a web client or litte-gui may tickle their fancy more than using a config file.

Basically, I was stuck in 'I need to do X' mode, and didnt realise until I started reading my answers, that of course I can just construct that value, and not make people enter it at all.

To confuse things some, this variable (and its associated config files), changed name between two program versions (now it does ftp or scp, and is called transfer)..

A DB plus a GUI would be lovely, but as I said, this is a script to install a piece of software, on various machines (remote ones, usually), so actually having the DB accessable, even an SQLite one, isn't all that practicable.

. o O(Although a nice little side project would be to extract this information from our Lotus Notes delivery DB, and precreate some of the config file, or have a CGI to create it in a DB, and a button to create the config file from that info, in which case it could well be XML or whatever.. ) O o ... I can dream..

C.