Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

While Config::General with Tie::IxHash works very well for me, there is an issue I'm struggling with; The order of the keys are not retained when the configuration file is saved.

Sample perl script

#!/usr/bin/perl use strict; use warnings; use Config::General qw{ParseConfig}; use Tie::IxHash; my $file = 'test.cfg'; tie my %CFG, "Tie::IxHash"; %CFG = ParseConfig( -ConfigFile => $file, -Tie => "Tie::IxHash", ); print "When freshly loaded:\n"; print join "\n", keys %CFG; print "\n\n"; (new Config::General( -ConfigHash => \%CFG, -Tie => "Tie::IxHash", ))->save_file($file); # SaveConfig appears "equally bad" at this.. %CFG = ParseConfig( -ConfigFile => $file, -Tie => "Tie::IxHash", ); print "When saved & reloaded:\n"; print join "\n", keys %CFG;

And the test.cfg

<abc1> opt val </abc1> <abc2> opt val </abc2> <cba1> opt val </cba1> <cba2> opt val </cba2> <test1> opt val </test1> <test2> opt val </test2>

Results in:

When freshly loaded: abc1 abc2 cba1 cba2 test1 test2 When saved & reloaded: test1 cba2 abc2 abc1 test2 cba1

I would like the original order of those keys retained when the config file is saved and loaded again. Is this possible, or do I need to implement my own "save tied hash order to file" function?

Replies are listed 'Best First'.
Re: Retaining hash order with Config::General?
by kennethk (Abbot) on Dec 15, 2008 at 16:25 UTC

    While I am not familiar with the Config::General you are working with, a fundamental property of hashes is that order of key-value pairs should be irrelevant. Do you care about the order for some internal reason, or is this about the order in the saved file?

    Update: The Config::General->save_file method calls the _store method, which in turn traverses the hash with a keys method on line 1252. There also appears to be a save sorted version of the same loop, which you can use by setting the -SaveSorted flag and will reliably output your keys in alphabetical order. It still doesn't change the fact that a hash's order should be irrelevant.

    Update 2: Forward/backward slash typo. Thanks Arunbear.

      Update: The Config::General->save_file method calls the _store method, which in turn traverses the hash with a keys method on line 1252.

      And thereabout is where the problem lies. The module takes extra care of preserving ties on the data (thus on the order, if tied to Tie::IxHash), up to calling this method, store_. And there, it foolishly throws it away:

      sub _store { # # internal sub for saving a block # my($this, $level, %config) = @_; ... }
      Ouch.

      This assignment of the sorted hash data to an ordinary hash %config is where the semi-random order comes from — only to traverse it later on.

      Tieing to any hash is useless this way.

      I'd call that a bug. I think it'd be better if _store accepted a hashref instead of flat data, to traverse the data tree.

        Thank you all for your replies.

        I will see if I can hack this by passing a hash ref or something. Christmas vacation coming up ;-)

      yes, I need it for internal purposes. Some of the entries in the file are listed to the user as available options for run-time configuration. from the user perspective, it looks silly when the menus change order for no apparant reason. Granted, I could force an alphabetical listing, or even maintain a separate sort list. Alphabetical is essentially random in this case, since anything except the structured order will not appear sane.. Cheers

        "Alphabetical is essentially random in this case, since anything except the structured order will not appear sane "

        So use a label field and sort by that. Consider a database. I don't care how the database stores the internals, only that I can retrieve that data. If i want to sort that data, i have to make sure that the data is sortable in a meaningful way. Personally, i would not store information into a Config file that would be better suited in a database, such as data that changes often. YMMV.

        jeffa

        L-LL-L--L-LL-L--L-LL-L--
        -R--R-RR-R--R-RR-R--R-RR
        B--B--B--B--B--B--B--B--
        H---H---H---H---H---H---
        (the triplet paradiddle with high-hat)
        

        Based on the code I've seen, you are best off just resorting the hash keys to the order you want just prior to display. Alphabetical is easiest, but you can resort the hash order with a custom test like this:

        #!/usr/bin/perl use strict; use warnings; use Config::General qw{ParseConfig}; use Tie::IxHash; my $file = 'test.cfg'; tie my %CFG, "Tie::IxHash"; %CFG = ParseConfig( -ConfigFile => $file, -Tie => "Tie::IxHash", ); print "When freshly loaded:\n"; print join "\n", keys %CFG; print "\n\n"; (new Config::General( -ConfigHash => \%CFG, -Tie => "Tie::IxHash", ))->save_file($file); # SaveConfig appears "equally bad" at this.. %CFG = ParseConfig( -ConfigFile => $file, -Tie => "Tie::IxHash", ); print "When saved & reloaded:\n"; print join "\n", sort {myfunction($a, $b)} keys %CFG; sub myfunction { my($key1, $key2) = @_; if (substr($key1,length($key1)-1,1) == substr($key2,length($key2)- +1,1)) { return ($key1 cmp $key2); } else { return (substr($key1,length($key1)-1,1) cmp substr($key2,lengt +h($key2)-1,1)); } }

Re: Retaining hash order with Config::General?
by Anonymous Monk on Dec 15, 2008 at 16:07 UTC
    Try rewrite your test like so
    { tie ... #init, save } { tie ... #reload }
      I fail to see how this will affect the result, but perhaps I am mistaken. I do these in different blocks in the real application, and it doesn't work there either. I will give it a try when I return to my computer. Thanks.
Re: PATCH - Retaining hash order with Config::General?
by Anonymous Monk on Dec 16, 2008 at 16:38 UTC

    I cooked together a simple patch based on bart's suggestion, submitted to the module author also. My module version is 2.40 (latest

    WARNING: Barely tested. Use at your own risk. Works for Me(tm).

    1107c1107 < $config_string = $this->_store(0, %{$this->{config}}); --- > $config_string = $this->_store(0, \%{$this->{config}}); 1114c1114 < $config_string = $this->_store(0,%{$config}); --- > $config_string = $this->_store(0,\%{$config}); 1140c1140 < return $this->_store(0, %{$this->{config}}); --- > return $this->_store(0, \%{$this->{config}}); 1147c1147 < return $this->_store(0, %{$config}); --- > return $this->_store(0, \%{$config}); 1158c1158 < my($this, $level, %config) = @_; --- > my($this, $level, $config) = @_; 1169,1171c1169,1171 < foreach my $entry (sort keys %config) { < if (ref($config{$entry}) eq 'ARRAY') { < foreach my $line (sort @{$config{$entry}}) { --- > foreach my $entry (sort keys %{ $config }) { > if (ref($$config{$entry}) eq 'ARRAY') { > foreach my $line (sort @{$$config{$entry}}) { 1173c1173 < $config_string .= $this->_write_hash($level, $entry, $li +ne); --- > $config_string .= $this->_write_hash($level, $$entry, $l +ine); 1176c1176 < $config_string .= $this->_write_scalar($level, $entry, $ +line); --- > $config_string .= $this->_write_scalar($level, $$entry, +$line); 1180,1181c1180,1181 < elsif (ref($config{$entry}) eq 'HASH') { < $config_string .= $this->_write_hash($level, $entry, $config +{$entry}); --- > elsif (ref($$config{$entry}) eq 'HASH') { > $config_string .= $this->_write_hash($level, $$entry, $$conf +ig{$entry}); 1184c1184 < $config_string .= $this->_write_scalar($level, $entry, $conf +ig{$entry}); --- > $config_string .= $this->_write_scalar($level, $$entry, $$co +nfig{$entry}); 1189,1191c1189,1191 < foreach my $entry (keys %config) { < if (ref($config{$entry}) eq 'ARRAY') { < foreach my $line (@{$config{$entry}}) { --- > foreach my $entry (keys %{ $config }) { > if (ref($$config{$entry}) eq 'ARRAY') { > foreach my $line (@{$$config{$entry}}) { 1200,1201c1200,1201 < elsif (ref($config{$entry}) eq 'HASH') { < $config_string .= $this->_write_hash($level, $entry, $config +{$entry}); --- > elsif (ref($$config{$entry}) eq 'HASH') { > $config_string .= $this->_write_hash($level, $entry, $$confi +g{$entry}); 1204c1204 < $config_string .= $this->_write_scalar($level, $entry, $conf +ig{$entry}); --- > $config_string .= $this->_write_scalar($level, $entry, $$con +fig{$entry}); 1269c1269 < $config_string .= $this->_store($level + 1, %{$line}); --- > $config_string .= $this->_store($level + 1, \%{$line});