Adjusting variable context when workign with JSON data?

unmatched has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

Today I decided to practice some Perl by porting a Python script. What took me the most to wrap my head around was working with JSON data using JSON::XS::decode_json. In my example below, this returns an array of hashes, i.e.: [{}, {}, {}]. Therefore, I would've thought that the return value could be inferred as a list, and declared using:

@decoded = decode_json($result);
[download]

But this didn't work as expected, and this threw me off when trying to do something simple like looping on the resulting value. I found a few similar questions online and managed to finish the script, but I was wondering if there's a better way of doing this. In particular, the syntax for my $workspace (@$decoded) seems really weird to me. Do you have any thoughts on this?

For context, I'm using the i3 window manager and I often want to move windows over to the next empty workspace. This problems translates to finding the next non-consecutive number in a sequence of numbers. For example, given the workspaces 1, 2, 3, 5 and 6, while currently focusing workspace 1, this returns workspace #4. If I'm focused on workspace #5, however, it would create a new workspace at #7.

#!/bin/perl

use v5.10;
use strict;
use warnings;

use JSON::XS qw/decode_json/;
use feature qw/say/;

sub main {
    my $result = qx{ i3-msg -t get_workspaces }; # returns [{},{},{}]
    my $decoded = decode_json($result);
    my $current = 0;

    for my $workspace (@$decoded) {

        my $num = $workspace->{'num'};

        if ($workspace->{'focused'}) {
            $current = $num;
            next;
        }

        # Ignore workspaces below the currently focused one.
        if ($current == 0) { next; }

        # Found the gap!
        if ($current + 1 != $num) { last; }

        # Keep increasing the soon-to-be current workspace.
        $current = $num;

    }

    say $current + 1;
}

main();
[download]

And the Python version, in case you want to give some advice on this too! :P

#!/bin/python3

import json
import subprocess

def main():
    result = subprocess.run(
        ['i3-msg', '-t',  'get_workspaces'],
        capture_output=True,
        check=True
    )

    i3_workspaces = json.loads(result.stdout)
    current_workspace = next((x['num'] for x in i3_workspaces if x['fo
+cused']))
    last_workspace = i3_workspaces[-1]['num']
    remaining_workspaces = (x['num'] for x in i3_workspaces if x['num'
+] >= current_workspace)

    for i in range(current_workspace, last_workspace):
        if i != next(remaining_workspaces):
            print(i)
            return

    print(last_workspace + 1)

if __name__ == '__main__':
    main()
[download]

And for some fun, I decided to compare the performance of each. I know it's a really simple script and the test is basically meaningless, but I was curious, and actually the results are surprising to me:

time $(for i in {1..100}; do ./move_to_next.pl> /dev/null; done;)

Perl:
real    0m1.113s
user    0m0.843s
sys     0m0.266s

Python:
real    0m1.852s
user    0m1.516s
sys     0m0.323s
[download]

But then I changed the code to run the loop inside the program itself, basically calling `main` 100 times, and running the test again like so:

time ./move_to_next.pl > /dev/null

Perl:
real    0m0.231s
user    0m0.116s
sys     0m0.116s

Python:
real    0m0.099s
user    0m0.078s
sys     0m0.018s
[download]

So, it seems that spawning the Python interpreter itself is quite costly, but it runs faster on the long run? I also saw a video about how the JSON module in Perl is quite slow, could that be it?

Anyway, thank you in advance!

Comment on Adjusting variable context when workign with JSON data? Select or Download Code

Replies are listed 'Best First'.
Re: Adjusting variable context when workign with JSON data? by hv (Prior) on Dec 04, 2024 at 02:36 UTC
In particular, the syntax `for my $workspace (@$decoded)` seems really weird to me. It does not seem weird to me, but for me it is very normal to choose to work with references rather than real arrays. In particular for function calls, it is much more efficient for a function to return a reference than to return a full list (of arbitrary length) to the caller. It takes some getting used to, the core concept is that when declaring a variable the sigil specifies the type of container, but when accessing a variable the sigil specifies the type of structure we are asking for. Thus: `my @a = (1 .. 5); say $a[0]; # '$' sigil, we are picking out a scalar say @a[3,1,2]; # '@' sigil, we are picking out a list (an array-like + structure) my %h = (a => 1, b => 2, c => 3); say $h{a}; # '$' sigil, we are picking out a scalar say @h{qw{b c}}; # '@' sigil, we are picking out a list say %h{qw{a c}}; # '%' sigil, we are picking out a hash slice (a hash- +like structure)` [download] With references it is similar, except that the reference itself is always a scalar that needs dereferencing `my $aref = [ 1 .. 5 ]; say $aref->[0]; # or ${$aref}[1] say @{$aref}[3,1,2]; my $href = { a => 1, b => 2, c => 3 }; say $href->{a}; # or ${$href}{a} say @{$href}{qw{b c}}; say %{$href}{qw{a c}};` [download] Given this, it is natural that given an array reference `$aref` we would reference the whole array as `@$aref`, and given a hash reference `$href` we would reference the whole hash as `%$href`.	[reply] [d/l] [select]
Re^2: Adjusting variable context when workign with JSON data? by unmatched (Sexton) on Dec 04, 2024 at 17:21 UTC
I see, this is what I'm trying to understand better about Perl, and what I mean by "variable context". It'll take some getting used to but I'll get there, eventually. Thanks!	[reply]
Re: Adjusting variable context when workign with JSON data? by karlgoethebier (Abbot) on Dec 03, 2024 at 15:47 UTC
One remark - just as an aside: Try hyperfine. To measure such things, it is certainly a better solution than `time`. man hyperfine ŤThe Crux of the Biscuit is the Apostropheť	[reply] [d/l]
Re^2: Adjusting variable context when workign with JSON data? by unmatched (Sexton) on Dec 03, 2024 at 21:11 UTC
Thank you, I didn't know about this one, I'll be sure to check it out.	[reply]
Re: Adjusting variable context when working with JSON data? by hippo (Archbishop) on Dec 03, 2024 at 17:52 UTC
I also saw a video about how the JSON module in Perl is quite slow, could that be it? No, because you are not using the JSON module, you are using the JSON::XS module. The two are quite different and the latter is purposely much faster than the former. And even if it weren't we are not talking about a particularly large chunk of JSON to decode. I have not profiled your Perl code because as posted it fails to compile: `Missing right curly or square bracket at 11162986.pl line 36, at end o +f line syntax error at 11162986.pl line 36, at EOF 11162986.pl had compilation errors.` [download] While I can guess what is wrong, this shows that the code you have posted is not the code you are running, so who knows what else might actually be in there to slow it down. Regardless, a fair guess would be that the slow part is this line: `my $result = qx{ i3-msg -t get_workspaces }; # returns [{},{},{}]` [download] Shelling out 100 times is going to be slow however you do it. Try obtaining the JSON once and then loop 100 times over the rest of the processing and see what difference that makes. 🦛	[reply] [d/l] [select]
Re^2: Adjusting variable context when working with JSON data? by unmatched (Sexton) on Dec 03, 2024 at 21:31 UTC
Oops, I think I accidentally removed that closing curly brace for the loop there. I originally posted the without using any HTML tags and went back to adjust the post manually... got a little too carried away. You are probably right, spawning new processes is always expensive, and making this only once shows that Perl runs much faster. I would like to play with hyperfine as suggested above, so I might try that when I have something a bit more meaningful to test on. The "benchmark" was just for fun, thought, to see how Python and Perl compare. What I'm more concerned is with the correctness of the script, specifically how the JSON is processed. As I said, I find this syntax a bit strange: `(@$decoded)`. I couldn't figure it out without looking for answers online and I'm not sure if there's perhaps a better way to tell Perl that this is an array (assuming that I know in advance the structure of the JSON to be returned)?	[reply] [d/l]
Re^3: Adjusting variable context when working with JSON data? by choroba (Cardinal) on Dec 03, 2024 at 21:38 UTC
You can start with an array at the beginning: `my @decoded = @{ decode_json($result) };` [download] `map{substr$_->[0],$_->[1]\|\|0,1}[\\|\|{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^ARGV,3]`	[reply] [d/l] [select]
Re^4: Adjusting variable context when working with JSON data? by unmatched (Sexton) on Dec 04, 2024 at 17:24 UTC
Re^3: Adjusting variable context when working with JSON data? by Danny (Chaplain) on Dec 05, 2024 at 00:57 UTC
I find this syntax a bit strange: (@$decoded) That's something you will have to get used to as it's a fundamental syntax in perl. $decoded is a reference to an array. To dereference it you just put a @ at the front. e.g. `$x = [1,2,3]; @array = @$x`. You can also put squiggles around the reference if that makes it clearer and in some cases this is needed, like @{$decoded}.	[reply] [d/l]
Re^4: Adjusting variable context when working with JSON data? by unmatched (Sexton) on Dec 08, 2024 at 10:52 UTC
Re^5: Adjusting variable context when working with JSON data? by hippo (Archbishop) on Dec 08, 2024 at 11:32 UTC
Some notes below your chosen depth have not been shown here
Re: Adjusting variable context when workign with JSON data? by karlgoethebier (Abbot) on Dec 04, 2024 at 11:55 UTC
Just one more thing: You might also like black as well as Perl::Tidy and perlreftut. ŤThe Crux of the Biscuit is the Apostropheť	[reply]
Re^2: Adjusting variable context when workign with JSON data? by unmatched (Sexton) on Dec 04, 2024 at 17:26 UTC
Yes, I need to start looking into things like that... I promise I'll get to that at some point :D	[reply]
Re: Adjusting variable context when workign with JSON data? by karlgoethebier (Abbot) on Dec 03, 2024 at 21:12 UTC
`subprocess.run` doesnt subshell per default - you may replace the backticks with `run` from IPC::Run to avoid subshelling - see also for an older discussion. ŤThe Crux of the Biscuit is the Apostropheť	[reply] [d/l] [select]
Re^2: Adjusting variable context when workign with JSON data? by ikegami (Patriarch) on Dec 04, 2024 at 20:30 UTC
`capturex` from ~~core~~ module IPC::System::Simple would be a simpler replacement for backticks.	[reply] [d/l]
Re^3: Adjusting variable context when workign with JSON data? by karlgoethebier (Abbot) on Dec 05, 2024 at 10:24 UTC
That I didn't know that - that's how I realize that I'm slowly becoming outdated. ŤThe Crux of the Biscuit is the Apostropheť	[reply]
Re^4: Adjusting variable context when workign with JSON data? by ikegami (Patriarch) on Dec 05, 2024 at 17:37 UTC
Re^5: Adjusting variable context when workign with JSON data? by ikegami (Patriarch) on Dec 05, 2024 at 17:46 UTC
Re^4: Adjusting variable context when workign with JSON data? by soonix (Chancellor) on Dec 05, 2024 at 11:52 UTC
Re^5: Adjusting variable context when workign with JSON data? by karlgoethebier (Abbot) on Dec 05, 2024 at 13:49 UTC
Re^2: Adjusting variable context when workign with JSON data? by unmatched (Sexton) on Dec 03, 2024 at 21:34 UTC
That is very good to know, thank you. I figure it's not important in this particular case as it's quite a simple script that I only run occasionally, but I'll try to remember that in the future.	[reply]