If you've discovered something amazing about Perl that you just need to share with everyone, this is the right place.

This section is also used for non-question discussions about Perl, and for any discussions that are not specifically programming related. For example, if you want to share or discuss opinions on hacker culture, the job market, or Perl 6 development, this is the place. (Note, however, that discussions about the PerlMonks web site belong in PerlMonks Discussion.)

Meditations is sometimes used as a sounding-board — a place to post initial drafts of perl tutorials, code modules, book reviews, articles, quizzes, etc. — so that the author can benefit from the collective insight of the monks before publishing the finished item to its proper place (be it Tutorials, Cool Uses for Perl, Reviews, or whatever). If you do this, it is generally considered appropriate to prefix your node title with "RFC:" (for "request for comments").

User Meditations
Performance of hash and array inserts
4 direct replies — Read more / Contribute 		by bliako
on Sep 29, 2021 at 07:45

    After reading A short meditation about hash search performance because of Hash Search is VERY slow I wanted to investigate the performance of inserting into a hash and array as the number of existing elements increases. The methodology is to Benchmark::timeit() a number of insertions into an empty hash or an empty array. For array, insertion means "push at the end". My aim was to see if the time to insert is constant, linearly increasing with the number of items inserted. I believe this is confirmed. Inserts of N elements into a fresh hash are timed and recorded for different N. I did not just time only 1 insert because there's the suspicion that existing size may affect each new insert. I.e. it's faster to insert into an empty container. So I inserted N items and timed that in total.

    Linear regression is used to build the relationship T = aX+b where T is the total time to insert X items into a fresh container. The error of the regression will be indicative of how well the hypothesis of constant insert time holds.

    I interpret the results as follows: inserting into a hash or array takes constant time and is independent of size: e.g. if insertion of 1000 items into a fresh hash takes 1 unit of time then 2000 items take 2 units of time. Same with array. Caveat: hash keys provided by random_key() were extremely random. The confidence of this conclusion is stronger for arrays than for hash (the error of the linear regression). Inserting into an array is faster than inserting into a hash. Twice as fast in my computer/OS. Perl version used 5.32.1, Linux Fedora 5.13.19.

    Caveat #2 The methodology is simplistic so please add on if you have better inspiration.

    5' Edit: The ascii plot looks volatile but in reality the numbers show a nice linear relationship, which is confirmed by the low error of the least squares. Also "linear regression" and "least squares" mean the same in this post.

    There have been some more edits within 20 minutes of posting. E.g. to clarify that array insert means push.

    10000 array insertions took 1.00004911422729 seconds.
11000 array insertions took 1.09900379180908 seconds.
12000 array insertions took 1.20203304290771 seconds.
13000 array insertions took 1.32625269889832 seconds.
14000 array insertions took 1.42015385627747 seconds.
15000 array insertions took 1.50754404067993 seconds.
16000 array insertions took 1.60211801528931 seconds.
17000 array insertions took 1.70919871330261 seconds.
18000 array insertions took 1.80735301971436 seconds.
19000 array insertions took 1.90634799003601 seconds.
20000 array insertions took 1.99797105789185 seconds.
21000 array insertions took 2.09790301322937 seconds.
22000 array insertions took 2.20532703399658 seconds.
23000 array insertions took 2.30040597915649 seconds.
24000 array insertions took 2.40066504478455 seconds.
25000 array insertions took 2.50436305999756 seconds.
26000 array insertions took 2.59865713119507 seconds.
27000 array insertions took 2.70806193351746 seconds.
28000 array insertions took 2.79667997360229 seconds.
29000 array insertions took 2.89690399169922 seconds.
30000 array insertions took 2.99919009208679 seconds.
Array inserts:
Y = 9.94971160764818e-05 * X + 0.0141616115322365 with error 0.0014576
+9586141692
Y : total time to insert X elements.
10000 hash insertions took 2.02198815345764 seconds.
11000 hash insertions took 2.21846413612366 seconds.
12000 hash insertions took 2.43116092681885 seconds.
13000 hash insertions took 2.62970495223999 seconds.
14000 hash insertions took 2.83560991287231 seconds.
15000 hash insertions took 3.04865097999573 seconds.
16000 hash insertions took 3.23556900024414 seconds.
17000 hash insertions took 3.44259905815125 seconds.
18000 hash insertions took 3.65537810325623 seconds.
19000 hash insertions took 3.85966873168945 seconds.
20000 hash insertions took 4.09639000892639 seconds.
21000 hash insertions took 4.35650420188904 seconds.
22000 hash insertions took 4.53428721427917 seconds.
23000 hash insertions took 4.74114084243774 seconds.
24000 hash insertions took 5.05467200279236 seconds.
25000 hash insertions took 5.18268871307373 seconds.
26000 hash insertions took 5.41770219802856 seconds.
27000 hash insertions took 5.68814396858215 seconds.
28000 hash insertions took 5.90737128257751 seconds.
29000 hash insertions took 6.0479040145874 seconds.
30000 hash insertions took 6.26542806625366 seconds.
Hash inserts:
Y = 0.000215204607047044 * X + -0.176900404356259 with error 0.0075377
+646200501
Y : total time to insert X elements.

    [download]
      7 +-----------------------------------------------------------------
+-----+   
    |                 +                 +                +            
+     |   
    |                                                       'arr2'    
+A    |   
  6 |-+                                                    'hash2'    
+BB +-|   
    |                                                           B   B 
+     |   
    |                                                        B        
+     |   
    |                                                 B  B            
+     |   
  5 |-+                                                               
+   +-|   
 S  |                                          B  B                   
+     |   
 E  |                                      B                          
+     |   
 C4 |-+                             B   B                             
+   +-|   
 O  |                           B                                     
+     |   
 N  |                        B                                        
+     |   
 D3 |-+               B  B                                            
+   +-|   
 S  |             B                                             A   A 
+ A   |   
    |      B   B                                      A  A   A        
+     |   
    |   B                                  A   A  A                   
+     |   
  2 |-+                         A   A   A                             
+   +-|   
    |                 A  A   A                                        
+     |   
    |      A   A  A   +                 +                +            
+     |   
  1 +-----------------------------------------------------------------
+-----+   
  10000             15000             20000            25000          
+   30000 
                                   number of items

    [download]
    Read more... (4 kB)

    bw, bliako
Organizational Culture (Part VII): Science
6 direct replies — Read more / Contribute 		by eyepopslikeamosquito
on Sep 14, 2021 at 00:02

    Publish or Perish

    I remember my dad lamenting his scientific publish or perish workplace culture, lightheartedly explaining to me one day how promotions were awarded:

    1. Each applicant's publications are placed on a weighing scale
    2. The promotion is awarded to the candidate with the heaviest weight

    Using that simple method, I doubt that Albert Einstein would've gained a promotion in 1905, due to the (meagre) weight of his Annus Mirabilis papers:

    • On a Heuristic Point of View Concerning the Production and Transformation of Light External (Photoelectric effect). 16 pages.
    • On the Movement of Small Particles Suspended in Stationary Liquids Required by the Molecular-Kinetic Theory of Heat External (Brownian motion). 11 pages.
    • On the Electrodynamics of Moving Bodies External (Special Theory of Relativity). 30 pages.
    • Does the Inertia of a Body Depend Upon its Energy Content? (E=mc**2). 3 pages.

    In fact, Albert did not score a promotion in 1905, as detailed in his 1905 Performance Appraisal:

    This is a patent office, Albert. Your job is to transform written patent applications into clear and precise language, and to study applications and pick out the new ideas of an invention. These are the priorities. Where does it say that your priorities are rewriting the rules of the Universe, unifying space and time, unifying radiation and matter, or demonstrating the existence of atoms?

    Regrettably, I had to put you down as "poor" for "works well with others" and "shares credit appropriately". You had no co-authors on your five papers, and your citations were quite skimpy: no citations at all in your June and September paper, only one citation in your April paper, and not much better on the others. You wrote that your special theory of relativity came to you after a discussion with your friend Michele Besso. But you didn't even acknowledge him in your June paper. This is an area for improvement.

    You seem to lack a flare for self-promotion. Lucky for us our PR department stepped in and changed your L/c2 equation into the much more marketable E = mc2.

    Based on his performance as a patent clerk, I cannot recommend Albert for a promotion at this time.

    -- from Einstein's Patent Clerk Third Class Performance Appraisal of 1905

    Curiously, Einstein was passed over for promotion for three long years, until he "fully mastered machine technology", remaining a lowly Patent Clerk Third Class at the Swiss Patent Office until 1 April 1906, when he was finally promoted to Technical Expert Second Class.

    See also: Performance Appraisals from the Agile Imposition series.

    The Sad Story of Giordano Bruno

    In 1600 Giordano Bruno was found guilty of heresy by the Roman Inquisition. He was then humiliatingly paraded naked through the streets of Rome (with his tongue lashed to prevent him speaking) and finally burned at the stake, with his ashes thrown in the Tiber river. His crime? Declaring that the stars are distant suns surrounded by their own planets.

    I'm still amazed at how far ahead of his time Bruno was, his bold conjecture only recently verified by exoplanet detections by the Kepler space telescope. If there is a heaven, I hope Bruno smiles every time Kepler finds a new exoplanet and I look forward to paying my respects to him at his statue, located at the site of his execution.

    Galileo was only placed under house arrest for the lesser crime of suggesting that the Earth and planets revolve around the Sun (rather than the other way around).

    Scientific Culture

    An important scientific innovation rarely makes its way by gradually winning over and converting its opponents: it rarely happens that Saul becomes Paul. What does happen is that its opponents gradually die out, and that the growing generation is familiarized with the ideas from the beginning: another instance of the fact that the future lies with the youth.

    -- Planck's Principle (Scientific autobiography, 1950, p.33,97)

    The process in the scientific method involves making conjectures (hypotheses), deriving predictions from them as logical consequences, and then carrying out experiments or empirical observations based on those predictions. A hypothesis is a conjecture, based on knowledge obtained while seeking answers to the question. The hypothesis might be very specific, or it might be broad. Scientists then test hypotheses by conducting experiments or studies. A scientific hypothesis must be falsifiable, implying that it is possible to identify a possible outcome of an experiment or observation that conflicts with predictions deduced from the hypothesis; otherwise, the hypothesis cannot be meaningfully tested.

    -- from Scientific method (wikipedia)

    Thankfully, Science has come a long way since the days of Galileo and Bruno. Today, the scientific community expects:

    • Rigorous scrutiny. In science, all ideas (especially the important ones) must stand up to rigorous scrutiny. The culture of science does not value dogma.
    • Honesty, integrity, and objectivity.
    • Credit where credit is due. Scientific research articles always provide a list of citations, crediting other scientists for ideas, techniques, and studies that were built upon by the current research. The number of citations a paper receives can help indicate how influential it was, since important research influences how other scientists think about a topic and will be cited many times in other papers.
    • Adherence to ethical guidelines.

    Update (see erix response below): the four Mertonian norms (often abbreviated as the CUDO-norms) are:

    • Communism: all scientists should have common ownership of scientific goods (intellectual property), to promote collective collaboration; secrecy is the opposite of this norm.
    • Universalism: scientific validity is independent of the sociopolitical status/personal attributes of its participants.
    • Disinterestedness: scientific institutions act for the benefit of a common scientific enterprise, rather than for the personal gain of individuals within them.
    • Organized skepticism: scientific claims should be exposed to critical scrutiny before being accepted: both in methodology and institutional codes of conduct.

    Because it undermines science, scientists take misconduct very seriously. In response to misconduct, the scientific community may withhold esteem, job offers, and funding, effectively preventing the offender from participating in science. There are also strict rules around scientific publications, as detailed at publishing process and responsible referencing:

    • After erasing all signs of your identity, your paper will be sent to 2–3 other scholars who have done research in the same field for review; they will not know who you are or who the other readers or reviewers are.
    • The reviewers will spend a few months reading your submission, double-checking your methods and or claims/citations from other source; they will then send it back to the manuscript editor with commentary and one of the following votes: accept for publication; revise and resubmit (the most common outcome); reject for publication. The reviewers may further write some commentary, for example, a need to redo the experiment using slightly different equipment, or ask you to read through certain books or articles others have published and incorporate that into your work.
    • The reviewers have to judge the work based on what you have written and what you use as evidence, not who you are or whether you have a fancy degree.

    Though the traditional Scientific method has served us well, it seems we need new methodologies to tackle the urgent problems facing us today.

    Crisis Disciplines

    On an evolutionarily miniscule timescale, cultural and technological processes transformed our species’ ecology. These changes that have transpired over this period have come about largely to solve issues at the scale of families, cities, and nations; only recently have cultural products begun to focus on solutions to worldwide problems and wellbeing. Yet we lack the ability to predict how the technologies we adopt today will impact global patterns of beliefs and behavior tomorrow ... social interactions and external feedback make it difficult, if not impossible, to reason about cross-scale dynamics through argument alone (i.e., these are complex adaptive systems).

    Humanity faces global and existential threats including climate change, ecosystem degradation, and the prospect of nuclear war. We likewise face a number of other challenges that impact our wellbeing, including racism, disease, famine, and economic inequality. Our success at facing these challenges depends on our global social dynamics in a modern and technologically connected world. Given our evolved tendencies combined with the impact of technology and population growth, there is no reason to believe that human social dynamics will be sustainable or conducive to wellbeing if left unmanaged.

    Other crisis disciplines thrive on a close integration of observational, theoretical, and empirical approaches. Global climate models inform, and are informed by, experiments in the laboratory and the field. Mathematics describing disease dynamics suggest treatment paradigms in medicine, which can be tested and validated.

    A consolidated transdisciplinary approach to understanding and managing human collective behavior will be a monumental challenge, yet it is a necessary one. Given that algorithms and companies are already altering our global patterns of behavior for financial reasons, there is no safe hands-off approach.

    -- from Stewardship of global collective behavior (cited by erix)

    As argued convincingly above, the traditional scientific method, requiring laborious peer review for example, is too slow for crisis disciplines and better alternatives must therefore be urgently sought. On a more positive note, we've been forced to do this sort of thing before - the Manhattan Project and rapid vaccine development during our current COVID-19 pandemic spring to mind.

    Other Articles in This Series

    References

    Historical References

    References Added Later

    Updated 26-Sep-2021: Added paragraph on Mertonian Norms.

Similarities of Perl and Python?
8 direct replies — Read more / Contribute 		by Bod
on Aug 30, 2021 at 06:07

    Learned and wise Monks...

    Within my business we use the usual front-end web technologies of HTML/CSS/native Javascript/AJAX and the back-end is entirely Perl. With a very small amount of Java for two Android apps.

    Here in the UK there is currently a government scheme to get young people into work. The Kickstart Scheme pays the young person minimum wage for 25 hours per week for 6 months. We have applied to take on two young people under the scheme. Most applicants have played with React at most but I am interviewing a graduate tomorrow. On his CV he has C, Python and R which has been done as part of his Physics degree plus Python and Shell Scripts on a Raspberry Pi as a hobby.

    I know absolutely nothing about R.
    From my very limited knowledge of Python, I believe it is broadly in the same group of languages as Perl. Therefore, the skills this applicant already has in Python should be relatively easily transferrable to Perl
    - is that a fair assessment???

    Any suggestions for me bearing in mind the applicant has had no real workplace experience?
RFC: Mock::Data::Regex
No replies — Read more | Post response 		by NERDVANA
on Aug 27, 2021 at 04:13
    A few weeks ago I asked about iterating the characters of a character class, as part of a Yak-Shaving exercise for my not-yet-released Mock::Data module. Well, I went further, and built a regex-reverser! (generates random strings that match a user-supplied regex) I'd like some feedback on what people think of the API.
R.I.P. Charlie Watts
No replies — Read more | Post response 		by kcott
on Aug 25, 2021 at 16:00

    Charlie Watts died today at the age of 80. A sad event, but obviously a good innings.

    At first, I thought this probably had little relevance here. Then I thought of the many hours that I had coding with the Rolling Stones playing in the background: Charlie brought what they said was the engine to the Stones; he also brought that same engine to my coding.

    He was a great man and had a great life: R.I.P.

    — Ken
Organizational Culture (Part VI): Sociology
3 direct replies — Read more / Contribute 		by eyepopslikeamosquito
on Aug 11, 2021 at 05:09
Organizational Culture (Part V): Behavior
1 direct reply — Read more / Contribute 		by eyepopslikeamosquito
on Jul 16, 2021 at 06:34
Prefer Pure Perl Core Modules
9 direct replies — Read more / Contribute 		by Leitz
on Jul 13, 2021 at 10:03

    "I prefer to use pure Perl core modules instead of depending on the CPAN."

    Saying that on IRC usually causes a flurry of negative comments. I agree that the CPAN is resource intense, and I've put stuff there. If you use lots of code from the CPAN, I'm not going to make fun of you. However, I would ask that you give me the same respect. Here's why I choose this path.

    1. Compatibility Pure Perl modules are portable, the target node doesn't need a set of compiler tools. While XS based modules might improve performance, not all nodes have compiler tools. Some nodes are precluded from having compilers based on resources or security mandates. Depending on a module that may not be installable creates a production risk.

    2. Idempotence Well engineered software can be installed and removed cleanly. When I worked for a telco, we would install, remove, and then reinstall software during QA. If it failed at any of those, it failed. Period. There is no "cpan" command to uninstall a module. The "cpanm" -U option for "uninstall" is marked "EXPERIMENTAL". The few times I've tried to use it to install modules that were installed via "cpan", it could not find the modules. Even when I told the command where the module was. If a module cannot be installed and removed cleanly then it does not belong on a production system.

    3. Upgradeability The "cpan" -u option (upgrade) comes with the warning: "Blindly doing this can really break things, so keep a backup." This conflicts with the concept of keeping software current to reduce security vulnerabilities and bugs. If a system's software cannot be cleanly upgraded it should not be in production.

    4. Security -- See Addendum below -- One of Perl's common object oriented modules is Moose. Installing Moose adds roughly 900 modules to the node. Who is security checking all those dependencies? Who wants to explain each and every one of those modules to a security auditor? In truth, how many of us could explain the risks and benefits of all nine hundred dependencies? And are we being paid to check someone else's code or are we paid to keep a production system running?

    5. Immiscibility Most Linux distributions require some version of Perl for operation. This sounds good for Perl, until you realize that the versions are often very out of date. If you want to use a semi-recent Perl you usually have to compile your own and install it somewhere. You also have to install any CPAN modules separately, which means your backups are now taking longer and you have more to sift through when trying to make space. And, of course, anyone who wants to use your code has to concoct the same environment.

    Solution

    My personal solution is to use pure Perl core modules or pure Perl CPAN modules that do not have a large dependency list. Large, in this sense, is "Am I willing to deal with these dependencies manually?" At some point in time I hope to be Perl-smart enough to help improve CPAN, but I'm not there yet.

    Addendum

    An earlier version of this page referenced YAML::Tiny in the security section. Investigating, based on hippo's comment (below) about "suggested_options" ('suggests_policy' in MyConfig.pm) removed YAML::Tiny as a culprit. YAML::Tiny has no non-core module dependencies.

    Chronicler: The Domici War (domiciwar.net)

    General Ne'er-do-well (github.com/LeamHall)
Cliques solution pertinent to my use case
3 direct replies — Read more / Contribute 		by Sanjay
on Jul 07, 2021 at 11:57

    This refers to the hardness of finding cliques in a graph when the number of nodes becomes high. My use case is that I am trying to form cliques within a cluster where the sum of relationships is maximum.

    Background & definition (anthropomorphized for ease of understanding):

    Cluster: A number of persons where each person is related to at least one another person. The strength of the relationship is a number, i.e. Si-j is the strength of the relationship between person i & person j (Symmetric: Si-j = Sj-i). All combinations need not be present - if person x & person y have no relationship then Sx-y does not exist. No relationship with self - Si-i does not exist as well.

    Problem: Given a list of relationships Si-j: i from 1 to N, for each i some j from 1 to N, i!= j; (graph with the edges giving the strength of the relationship). Find those groups (cliques) where each member is connected to each other and the sum of the relationship is maximum, e.g. if the persons are as follows:

    Person-1  Person-2  Relationship-strength
A         B         92
A         C         7
B         C         2
C         D         88

    [download]

    Then the groups formed would be (A,B) & (C,D) for a sum of relationships of 180 (92 + 88).

    Any other combination would have a lesser sum, e.g. (A,B,C) & (D) would have a sum of relationships of 101 (92 + 7 + 2).

    How I did it: Convert the list of Si-j's into an integer linear programming problem and call a free solver lp_solve. Got good results - out of about 40,000 problem sets (clusters) only about 20 timed out. This on an ancient 3'rd gen i7 laptop with 4GB RAM. About another 10 were "very" large (more than 2,000 edges). Gives me confidence that use of a commercial solver and a larger timeout period may give even better results.

    Linear programming problem formulation left out here as it is more an algorithm issue rather than a Perl issue. We can discuss if interested. Off forum perhaps?

    Hope this is interesting. And, perchance, if it is useful for anyone then it would be the icing on the cake!
Organizational Culture (Part IV): Perl Culture
No replies — Read more | Post response 		by eyepopslikeamosquito
on Jul 05, 2021 at 09:25
Perl OOP (POOP) - a use case for Util::H2O (hash to object)
1 direct reply — Read more / Contribute 		by perlfan
on Jun 30, 2021 at 13:15
    Intuitively, I've been strongly drawn Util::H2O, and I finally have a solid example that has informed me explicitly what I have been feeling implicitly. My goal is to be laconic this time.

    Recently, I was writing a commandline tool in Perl, one of my favorite things to do. I needed Getopt::Long. I also was being nagged internally to use Util::H2O. Why? I simply wanted "real" accessors for the options hashref that I really like using with GetOptions. I ended up with something like this: 

    use strict;
use warnings;

use Getopt::Long qw/GetOptions/;
use Util::H2O qw/h2o/; # <~ here

my $opts = {};
GetOptions(
  $opts,
  qw/
      option1=s
      option2=i
      option3
      option4!
      option5+
    /
);
h2o $opts;             # <~ here

#...
exit;

__END__
## now we get the following, but only for the options actually passed 
+(updated)
# $opts->option1
# $opts->option2
# $opts->option3
# $opts->option4
# $opts->option5

    [download]
    Almost magically, with the addition of a single line (well, 2 if you count the use Util::H2O qw/h2o/; line,) I had accessors for $opts. I was pleased, to say the least. Note, I have not tried this with callbacks, but I also don't use them much. I suspect that works just fine. (update) Also note, in this case one gets accessors only for the options passed (or more precisely for only the keys that exist in $opts). I suspect this could be leveraged for more interesting and perlish ways of checking what options were actually passed.

    And now I can start to quantify where and when I'd like to use Util::H2O - not as an actual way to have a bunch of boiler plate at the top of my script to declare classes (which it does support), but perlishly (and iteratively) adding accessors to hash references - which my code tends to be absolutely full of. I've used it in some other places to clean up existing code and in new code, knowing full well I am going to be slinging hash references all over the place.

    In summary - check out Util::H2O. If you're like me, then you'll find little benefits to you that POOP offers can actually be satisfied in this natural way. It also occurs to me, this topic would be an ideal entry for the Perl Advent Calendar - so maybe I'll work out some more examples and make a submission later this year.
Benchmark: Storable, Sereal and JSON
3 direct replies — Read more / Contribute 		by stevieb
on Jun 30, 2021 at 11:38

    After I get a distribution very stable, I go back and try to make it better, faster or more efficient. In IPC::Shareable, I've been benchmarking various serialization techniques to see which one works the fastest. I knew that Sereal was quite a bit faster than Storable, but there are some gotchas with it that I couldn't work around. This morning I tested with JSON, and to my surprise, it blew both out of the water!

    Results:

    Benchmark: timing 5000000 iterations of json, sereal, store...
      json: 17 wallclock secs (17.53 usr +  0.00 sys = 17.53 CPU) @ 28
+5225.33/s (n=5000000)
    sereal: 22 wallclock secs (21.78 usr +  0.00 sys = 21.78 CPU) @ 22
+9568.41/s (n=5000000)
     store: 49 wallclock secs (49.55 usr +  0.01 sys = 49.56 CPU) @ 10
+0887.81/s (n=5000000)
           Rate  store sereal   json
store  102312/s     --   -56%   -64%
sereal 233863/s   129%     --   -18%
json   286862/s   180%    23%     --

    [download]

    Benchmark code:

    use warnings;
use strict;

use Benchmark qw(:all) ;
use JSON qw(-convert_blessed_universally);
use Sereal qw(encode_sereal decode_sereal looks_like_sereal);
use Storable qw(freeze thaw);

if (@ARGV < 1){
    print "\n Need test count argument...\n\n";
    exit;
}

timethese($ARGV[0],
    {
        sereal  => \&serial,
        store   => \&storable,
        json    => \&json,
    },
);

cmpthese($ARGV[0],
    {
        sereal  => \&serial,
        store   => \&storable,
        json    => \&json,
    },
);

sub _data {
    my %h = (
        a => 1,
        b => 2,
        c => [qw(1 2 3)],
        d => {z => 26, y => 25},
    );

    return \%h;
}
sub json {
    my $data = _data();
    my $json = encode_json $data;
    my $perl = decode_json $json;
}
sub serial {
    my $data = _data();
    my $enc = encode_sereal($data);
    my $dec = decode_sereal($enc);
}
sub storable {
    my $data = _data();
    my $ice = freeze($data);
    my $water = thaw($ice);
}

    [download]

    I would never have expected that. Next up, a new serialization option for IPC::Shareable!
Perl tools for making code better
3 direct replies — Read more / Contribute 		by Leitz
on Jun 29, 2021 at 10:02

    I have bodies of code to maintain and improve. Using tests is a given, and TDD is a wonderful thing. I am still learning Perl culture, and would appreciate guidance on tools and processes.

    My parameters are hopefully simple. I prefer clear and verbose code so I can understand it the next time I read it. When producing tools I prefer to stay as close to Core/StdLib Perl as possible. When working *with* tools, more options are open. For example, using a Perl::Critic plug-in while developing code is fine, shipping those add-ons is something I would avoid.

    My current plan is a cycle of:

    1. Write Tests
    2. Kwalitee
    3. Perl::Tidy
    4. Perl::Critic

    Repeating as necessary, or until I have to ship the code. Are there other tools to add to the mix? Better processes?

    Chronicler: The Domici War (domiciwar.net)

    General Ne'er-do-well (github.com/LeamHall)
Organizational Culture (Part III): Spaceflight and Aviation
2 direct replies — Read more / Contribute 		by eyepopslikeamosquito
on Jun 28, 2021 at 02:37

    Let's continue this seemingly never-ending series by examining culture change in two domains near to my heart:

    Space Shuttle Challenger (1986)

    As soon as the button was pressed to mute NASA from our meeting, the managers said "we have to make a management decision", said Boisjoly.

    The general manager of Thiokol turned to his three senior managers and asked what they wanted to do. Two agreed to go to a launch decision, one refused. So he (the general manager) turns to him and said "take off your engineering hat and put on your management hat" -- and that’s exactly what happened, said Boisjoly. He changed his hat and changed his vote, just thirty minutes after he was the one to give the recommendation not to launch. I didn’t agree with one single statement made on the recommendations given by the managers.

    The teleconference resumed and NASA heard that Thiokol had changed their mind and gave a recommendation to launch. NASA did not ask why.

    I went home, opened the door and didn’t say a word to my wife, added Boisjoly. She asked me what was wrong and I told her "oh nothing hunny, it was a great day, we just had a meeting to go launch tomorrow and kill the astronauts, but outside of that it was a great day".

    -- from Remembering the mistakes of Challenger

    For a successful technology, reality must take precedence over public relations, for nature cannot be fooled.

    -- Richard Feynman (in Rogers Commission Report on the Challenger disaster)

    Space Shuttle Columbia (2003)

    Although changes were made by NASA after the Challenger accident, many commentators have argued that the changes in its management structure and organizational culture were neither deep nor long-lasting.

    After the Space Shuttle Columbia disaster in 2003, attention once again focused on the attitude of NASA management towards safety issues. The Columbia Accident Investigation Board (CAIB) concluded that NASA had failed to learn many of the lessons of Challenger. In particular, the agency had not set up a truly independent office for safety oversight; the CAIB decided that in this area, "NASA's response to the Rogers Commission did not meet the Commission's intent". The CAIB believed that "the causes of the institutional failure responsible for Challenger have not been fixed", saying that the same "flawed decision making process" that had resulted in the Challenger accident was responsible for Columbia's destruction seventeen years later.

    -- from Space Shuttle Challenger disaster (wikipedia)

    Incredibly, NASA, with so many talented people, had somehow (twice!) failed to remedy a glaring Conflict of interest organizational structure/culture problem. It seems they failed to find a way to ensure that the technical team making launch safety decisions were allowed to do so without pressure or coercion from outside interests.

    (Which BTW reminds me of the poor old Release Manager in commercial software companies being relentlessly pressured by senior management to ship the next release ... less common perhaps in the open source world, though release management remains stressful).

    Perhaps the root cause of NASA's Space Shuttle woes was an unhealthy mix of commercial/financial and engineering/scientific culture. Certainly, NASA have achieved many breathtakingly brilliant successes in the purely engineering/scientific domain, such as the venerable Hubble Space Telescope and the recent stunning New Horizons mission to Pluto and beyond. Update: and the James Webb Space Telescope which became operational in July 2022 after a mind-bogglingly complex commissioning.

    It will be interesting to see in the coming decade if Elon Musk or Jeff Bezos or Sir Richard Branson can create a more effective culture for commercial space flight.

    National Cultures

    Social psychologist and former IBM researcher Geert Hofstede identified five dimensions of culture in his studies of national work related values:

    • Power Distance. How accepting are people of unequal power distribution?
    • Individualism. Is individual achievement or collective achievement emphasized?
    • Masculinity. How much does the culture accept gender roles?
    • Uncertainty Avoidance. How anxious are folks about uncertainty and risk?
    • Long-term Orientation. Do people take a long-term view of their work?
    Hofstede found that these five dimensions varied considerably between countries. The United States, for example, rated highest on Individualism, while Russia and India rated highest on Power Distance. Japan scored the highest on the Masculinity index. Russia and Japan have high Uncertainty Avoidance, while Denmark has a low score. Japan, South Korea and India have a much stronger Long-term Orientation than Canada, the UK and the USA.

    How this affects software teams was discussed previously at Nobody Expects the Agile Imposition (Part IV): Teamwork.

    Korean Airlines

    The captain made the decision to land despite the junior officer's disagreements, eventually bringing the plane down short of the runway, highlighting how a pilot can contribute to a disaster. In high power distance cultures, it is uncommon for subordinates to question their superiors. High power distance can be seen as the willingness to be in an unequal position, making it a challenge for an officer lower in the hierarchy to question the decisions of the one in power.

    -- Impact of culture on aviation safety (wikipedia)

    Korean Air had many fatal accidents between 1970 and 1999, during which time it wrote off 16 aircraft in serious incidents and accidents with the loss of 700 lives. The last fatal accident, Korean Air Cargo Flight 8509 in December 1999 led to a review of how Korean cultural attitudes had contributed to its poor crash history. Since then safety has improved.

    -- Korean Air incidents and accidents (wikipedia)

    In the cases of Korean Air flight 801 and Air Florida flight 90 one clear statement could have saved more than 300 lives. In each case, a series of leadership errors led to the deaths of hundreds of passengers. Cultural and contextual factors contributed to the leaders unknowingly creating unsafe conditions. A lack of cultural awareness and understanding of the environment was a culprit in both case studies, along with dozens of other airplane crashes. These two case studies serve as warnings for all leaders on the importance of including cultural dimensions in their leader(ship) development programs as well as adapting programs to the specific environmental factors. By taking a lesson from the aviation industry, other organizations can prevent similar mistakes, create better leaders and leader(ship) development programs, and possibly save lives.

    As a result of this and other accidents, Korean Air’s new CEO set about an ambitious organizational change. He made English the official language of the airline, redefined roles in the cockpit, and provided extensive communications training, among other initiatives to change the culture (Wee, 2013). Other airlines also made changes in response to the reoccurring communication challenges. One notable change made by many airlines was to adopt a program known as cockpit resource management (CRM). CRM requires the entire crew to work together and communicate (D’Amato, 2013). Training programs provide specific skills and procedures to maximize communication and effectiveness. CRM has proven to be very effective in reducing airplane crashes due to communication, regardless of cultural backgrounds.

    -- Copilot leadership (medium.com)

    It is gratifying (at long last!) to find a convincing example of where changing Organizational Culture actually worked!

    Other Articles in This Series

    References
Organizational Culture (Part II): Meta Process
3 direct replies — Read more / Contribute 		by eyepopslikeamosquito
on Jun 17, 2021 at 06:03

    Googling Organizational Culture revealed many folks offering (often pricey) Organizational Culture workshops based on theories concocted by a pair of enterprising boffins, quietly contemplating at the University of Michigan, located in the picturesque village of Ann Arbor.

    Though the definitive reference on their work is available for purchase from amazon, you can also get a feel for their process by reading this early paper:

    A Process for Changing Organizational Culture by Kim Cameron, University of Michigan 2008
    Handbook of Organizational Development, 2008: 429-445 (cited by 345)

    Abstract: This chapter outlines a process for diagnosing and changing organizational culture. It uses the Competing Values Framework to describe a validated approach to helping an organization change from a current culture to a desired culture.

    ... and from the (mostly youtube) links in the References section below. As you might expect, I was too cheap to pay for advice on this topic, so instead watched some youtube videos and read Kim's original paper. Though not necessarily the best organizational change process (alternative citations welcome), at least this is a concrete thing that can be discussed and analysed, and thus serve as a starting point for discussing specific ways to improve Organizational Culture (and Perl organizational culture too).

    For those seeking a Perl Monks connection to this academic paper, notice that Ann Arbor Michigan is a mere two hour scenic drive from Hope College in Holland Michigan, the sacred birthplace of Perl Monks (if you drive via Portage Michigan, you can further pick up some COVID-19 vaccines from Pfizer’s huge manufacturing facility on the way).

    Definition of Organizational Culture

    Although many definitions of culture have been proposed, the two main disciplines are:

    • Sociological (organizations have cultures). Assumes you can identify differences among organizational cultures, can change cultures, and can empirically measure cultures.
    • Anthropological (organizations are cultures). Assumes that nothing exists in organizations except culture, and one encounters culture anytime one rubs up against any organizational phenomena.

    In her 2008 paper, Cameron gave a popular and practical definition of culture as:

    the taken-for-granted values, underlying assumptions, expectations, and definitions present which characterize organizations and their members
    which:
    • serves as the social glue binding an organization together; and
    • represents how things are around here, affects the way members think, feel, and behave.

    and further perceptively noticed that:

    With very few exceptions, virtually every leading firm has developed a distinctive culture that is clearly identifiable by its key stakeholders

    This distinctive culture is sometimes created by the initial founder: Walt Disney, Bill Gates, and Larry Wall, for example. All three of these legends developed something special, something more vital than corporate strategy, market presence, or technical advantages: the power that arises from a unique and spirited culture.

    Curiously, most people are unaware of culture until suddenly confronted with a different one: travelling to Vietnam, for example, finding yourself immersed in different noises and smells and unable to understand a word of the local lingo ... or asking a question on SO after years of posting at Perl Monks. :)

    The culture of most organizations is invisible, most members have a hard time describing it, let alone consciously changing it -- that is why you need tools, such as The Competing Values Framework and the Organizational Culture Assessment Instrument (OCAI), developed by scholars Cameron and Quinn.

    Notice that Organizational Climate is distinct from Organizational Culture: Climate is temporary, Culture enduring.

    The Competing Values Framework

    This framework, used to assess the dominant characteristics of organizations, differentiates on two vertical dimensions:

    1. Flexibility, Discretion, Dynamism (effective if: changing, adaptable, organic, e.g. Google, Nike)
    2. Stability, Order, Control (effective if: stable, predictable, mechanistic, e.g. Universities, Government agencies, Boeing)

    and two horizontal dimensions:

    1. Internal orientation, Integration, Unity (harmonious internal characteristics, e.g. IBM, the IBM way)
    2. External orientation, Differentiation, Rivalry (effective if: competing with others outside their boundary, e.g. Toyota, Honda)

    Together these two dimensions form four quadrants:

    1. Clan culture (Collaborate)
    2. Adhocracy culture (Create)
    3. Hierarchy culture (Control)
    4. Market culture (Compete)

    Hierarchy Culture:

    • Formalised and structured workplace. Procedures and controls govern what people do. Leaders: coordinators, organizers, monitors.
    • Maintaining a smoothly running organization is important.
    • Long term: stability, predictability, efficiency, formal rules and policies hold the organization together.
    • Success is defined in clear lines: authority, control, accountability, e.g. McDonalds, Govt agency on airport controls (strict guidelines for every small detail).

    Market Culture:

    • Competing environment, results-oriented workplace, external environment is hostile, consumers are selective, want value.
    • Leaders are hard-driving producers; competitors are tough and demanding.
    • Productivity, results and profits; glue is emphasis on winning.
    • Success is market share and penetration, e.g. Ikea, Walmart.

    Clan Culture:

    • Friendly workplace, people share a lot of themselves, leaders are mentors, facilitators, team builders, parent figures, held together by loyalty and tradition.
    • Commitment is high, long-term benefit of individual development, high cohesion and morale important.
    • Success is defined as internal climate and concern for people, premium on teamwork, participation and consensus, e.g. small family-owned companies, doctors without borders, wikipedia.

    Adhocracy Culture:

    • Dynamic, entrepreneurial culture, people take risks, leaders: visionary, innovative, risk-oriented.
    • Glue: Commitment to experimentation and innovation, to be at the leading edge, readiness for change is essential.
    • Long term emphasis is rapid growth.
    • Success is creating unique & original products and services, e.g. start-ups.

    Relationship between the four quadrants:

    • Each side represents opposites
    • Flexibility vs Stability
    • Internal vs External
    • Competing on the Diagonal: Clan (internal focus) v Market (external focus); Adhocracy (external organic) v Hierarchy (internal control)
    The competing on opposite sides of each quadrant give rise to the name of the model.

    Why Change Organizational Culture?

    The Competing Values Framework Introduction youtube (around the 12:15 mark) gives a fascinating case study of how Organizational Cultures change over time. Steve Jobs, a charismatic entrepreneurial leader, was a great cultural fit for the (Startup culture) Apple of the 1970s ... only to get fired in 1985 for his chaotic management style, when Apple needed more controls and standard procedures ... then finally re-hired in 1997 to heroically resurrect the company after it started having a hard time inventing new products! That is, Apple culture evolved from Adhocracy (Apple II era) to Adhocracy/Clan (Macintosh era) to Hierarchy (John Scully era) to Hierarchy/Market then balanced Hierarchy/Market/Adhocracy/Clan (on Jobs return).

    How does a language win? By being compelling enough to be used for new things. It's not solely a technical concern; it's a concern of the language community and ecosystem.

    -- Why Perl Didn't Win (essay from outspeaking.com)

    As indicated in the essay cited above, a good reason for Perl to change its culture may be to make it more attractive for new projects (compared to competing languages). Update: see this response for a present-day example of a domain in which Perl is less compelling than Python.

    Of course, if Perl was a commercial enterprise, one business strategy to cope with losing market share may be to seek a merger with Python ... thus allowing our new customers to write some truly astonishing code: 

        # copy stdin to stdout, except for lines starting with #
    while left_angle_right_angle:
        if dollar_underscore[0] =eq= "#":
            continue_next;
        }
        print dollar_underscore;
    }

    [download]

    Sorry, couldn't resist. :)

    Note that if you choose not to attempt to explicitly change your organization's culture, it will change anyway. Culture is evolving all the time.

    The Seven Steps To Culture Change

    1. Clarifying meaning.
    2. Identifying stories.
    3. Determining strategic initiatives.
    4. Identifying small wins.
    5. Crafting metrics, measures, and milestones.
    6. Communication and symbols.
    7. Leadership development.

    The Organizational Culture Assessment Instrument (OCAI)

    It might be fun to create a Perl Monks poll to see how people rate P5P culture or Perl Monks culture.

    1. DOMINANT CHARACTERISTICS
    The organization is:
    A. a very special place. It is like an extended family. People seem to share a lot of themselves.
    B. a very dynamic and entrepreneurial place. People are willing to stick their necks out and take risks.
    C. very production oriented. A major concern is with getting the job done. People are very competitive and achievement oriented.
    D. a very formalized and structured place. Bureaucratic procedures generally govern what people do.

    2. ORGANIZATIONAL LEADERS
    The leaders of the organization are generally considered to be:
    A. mentors, facilitators, or parent figures.
    B. entrepreneurs, innovators, or risk takers.
    C. hard-drivers, producers, or competitors.
    D. coordinators, organizers, or efficiency experts.

    3. MANAGEMENT OF EMPLOYEES
    The management style in the organization is characterized by:
    A. teamwork, consensus and participation.
    B. individual risk-taking, innovation, flexibility, and uniqueness.
    C. hard-driving competitiveness, goal directedness, and achievement.
    D. careful monitoring of performance, longevity in position, and predictability.

    4. ORGANIZATION GLUE
    The glue that holds the organization together is:
    A. loyalty and mutual trust. Commitment to this organization runs high.
    B. orientation toward innovation and development. There is an emphasis on being on the cutting edge.
    C. the emphasis on production and goal accomplishment. Marketplace aggressiveness is a common theme.
    D. formal rules and policies. Maintaining a smooth running organization is important.

    5. STRATEGIC EMPHASES
    The organization emphasizes:
    A. human development. High trust, openness and participation persist.
    B. acquiring new resources and meeting new challenges. Trying new things and prospecting for new opportunities are valued.
    C. competitive actions and achievement. Measurement targets and objectives are dominant.
    D. permanence and stability. Efficient, smooth operations are important.

    6. CRITERIA OF SUCCESS
    The organization defines success on the basis of:
    A. development of human resources, teamwork, and concern for people.
    B. having the most unique or the newest products. It is a product leader and innovator.
    C. market penetration and market share. Competitive market leadership is key.
    D. efficiency. Dependable delivery, smooth scheduling, and low cost production are critical.

    Other Articles in This Series

    References

    See Also
<Newer entries  (166-180) of 300 Older entries> 

Add your Meditation
Title:
Meditation:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.