Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much

Preserve the order in JSON

by user786 (Sexton)
on Jun 16, 2015 at 20:16 UTC ( [id://1130703]=perlquestion: print w/replies, xml ) Need Help??

user786 has asked for the wisdom of the Perl Monks concerning the following question:

I have written a perl script to edit the a json file.

I need to check if cp is 10000 and set custcap to 99999 if so. If the cp is not equal to 10000, just exit out of the file without making changes.

The script works as expected. It makes the necessary changes and saves the files.

however after the changes,the order of the parameters in JSON file changes>

I read the json spec, My understanding is json is unordered.

can some one help me out with preserving the order

config file

{ "version": "15320029", "global": { "ap": { "log": "info", "hyd": { "log": "info", "qo": false } }, "cus": [ { **"cp": "10000"**, "ser": "XYZ", "usesr": false, "services": { "acc": { "ips": { "usesr": false } } }, "q": { "policy": "CAP", **"custcap": 3000000** } }, { "cp": "10441", "ser": "abc", "usesr": false, "services": { "acc": { "ips": { "usesr": false } } }, "q": { "policy": "CAP", "custcap": 3000000 } } ] }
#!/usr/bin/perl use strict; use warnings; use JSON; my $json; { open my $fh, "<", "cfg.txt" or die("Can't open file \"/Users/hsivaram/cfg.txt\": $!\n"); local $/; $json = <$fh>; } my $data = decode_json($json); for my $cus (@{ $data->{global}{cus} }) { $cus->{q}{custcap} = 99999 if $cus->{cp} == 10000; } $json = JSON->new->utf8->pretty->encode($data); { open my $fh, ">" ,"cfg.txt" or die("Can't open file \"/Users/hsivaram/cfg.txt\": $!\n"); local $/; print $fh $json; }

After script execution the order of the file changes. is there a way to preserve the order ?

Replies are listed 'Best First'.
Re: Preserve the order in JSON
by roboticus (Chancellor) on Jun 16, 2015 at 20:33 UTC


    Inside a dictionary/hash, there's no meaning to the order of the keys. That's what it means to be unordered. If you want to maintain order, I'd suggest switching to an array. Arrays will maintain the order.


    When your only tool is a hammer, all problems look like your thumb.

Re: Preserve the order in JSON
by afoken (Chancellor) on Jun 17, 2015 at 07:17 UTC
    is there a way to preserve the order

    Not exactly, but JSON can sort JSON object keys. You need to call $json->canonical() before $json->encode($data). Note that this adds some significant overhead.

    The big question remains: Why do you need sorted keys in the object?


    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
      Note that this adds some significant overhead.

      Have you measured it? I have. It is far from "significant" unless you have at least one hash that holds a very large number of keys. I find such to be pretty rare in the JSON I've had to deal with.

      I have found much more overhead resulted from the pain of having to deal with random order of keys in JSON, making comparisons or just visual searching much more difficult. And overhead to human processing costs a ton more than overhead to computer processing, IME.

      - tye        

Re: Preserve the order in JSON
by Laurent_R (Canon) on Jun 16, 2015 at 21:03 UTC
    Key and value pairs are always unordered in a hash. But you can also apply a sort to your data elements if you need to visit/modify them in a certain specific order. You're not saying really enough, tell us why you want a specific order.
Re: Preserve the order in JSON
by GotToBTru (Prior) on Jun 16, 2015 at 20:40 UTC

    I'm curious why you need the order preserved? Perhaps you need to add a field to store a sorting key?

    Dum Spiro Spero
Re: Preserve the order in JSON
by u65 (Chaplain) on Jun 17, 2015 at 11:41 UTC

    why retain order?

    If I were modifying a configuration file programmatically, I would want to preserve the original order to ease later visual inspection--very confusing to me to see a config file changed unnecessarily.

      & the unnecessary changes create headaches with version control when written to config or other files.
Re: Preserve the order in JSON
by Anonymous Monk on Jul 11, 2018 at 10:52 UTC
    Interesting that so many people have asked why you would need to preserve the order of keys when in fact it would be a very useful feature. Right now I would need it to modify the configuration of some charts, they are displayed in PHP in the order they are found in the JSON which is in turn stored in a postgres text field. I would've wanted to create a plperl function to modify the chart config but it looks like I'm better off reading the config, modifying it in PHP then saving it back to the db.
      If you need to preserve the order, don't use "JSON object" (aka hash in Perl) as the data structure, use an array (or include an ordering attribute to the object).

      ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
      Every time someone says they want to preserve the order of JSON keys, people jump on them for it, saying the keys are unordered by definition and why should they want that anyway? There are use cases for this!! Take the example of a JSON configuration file that happens to be stored in a git repository. Suppose you want to modify the JSON programmatically to make a slight change. If the keys are kept in the original order, a "git diff" of the changes will show a tiny change clearly. If all the hash keys are in a new randomized order, the diff will be mostly noise and the change will be very difficult to find.

      Anyway, I came up with a good solution for this using Monkey::Patch with JSON::PP to make it use Tie::IxHash for all objects decoded from JSON. This solves the problem, allowing a JSON file to be loaded, decoded from JSON, modified, encoded into JSON again and written back to the file, without changing the key ordering at all. Just make sure you're using JSON::PP (not JSON::XS) and that the $handle variable remains in scope:

      use Monkey::Patch qw[patch_package]; # Monkey-patch JSON::PP::object() subroutine to use Tie::IxHash. my $handle = patch_package 'JSON::PP' => 'object' => sub { my $orig = shift; my %obj; tie %obj, 'Tie::IxHash' or die "tie(\%obj, 'Tie::IxHash') failed!\n"; $orig->(\%obj) };
        Hello Deven,

        nice! as side note you can force JSON::PP to always order keys alphabetically using canonical or giving it a custom order function: JSON::PP#sort_by thus avoiding monkey patching.


        There are no rules, there are no thumbs..
        Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

        I wrote JSON::MultiValueOrdered to cope with JSON like the following:

        { "foo": 1, "bar": 2, "foo": 3 }

        If you parse the above into $data then $data->{foo} will be 3, but tied(%$data)->get('foo') in list context will return ( 1, 3 ), and if you serialize the structure, you'll get this, preserving the original order of the keys:

        { "foo": 1, "bar": 2, "foo": 3 }
        > saying the keys are unordered by definition and why should they want that anyway

        AFAIR is this disputed, because different RFC tell differently.

        JS Objects are unsorted tho and JSON stands for "JS Object Notation".

        The curlies in JS { ... } create literal objects (kind of hashes plus inheritance chain).

        Different definitions will lead to much confusion...

        Cheers Rolf
        (addicted to the 𐍀𐌴𐍂𐌻 Programming Language :)
        Wikisyntax for the Monastery

      I have a reason why. I have a vim command that I use that Tidies different files. :Tidy When it's perl code, it uses the Perl::Tidy. If it's JSON, then it formats it nicely using JSON::PP. Recently I've been doing comparisons of serialized JSON data (to make sure it's formatted correctly) and it would be nice if the order was maintained when being Tidied. Performance isn't critical, it's an editor command; not used in production.
        I've been using a one-liner. json_pp -f json -t json -json_opt pretty. I may end up implementing it using Hash::Ordered
        Use the ordering/sorting feature then nobody is stopping you
A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1130703]
Approved by Paladin
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (3)
As of 2024-04-21 05:24 GMT
Find Nodes?
    Voting Booth?

    No recent polls found