Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Meditations

( #480=superdoc: print w/replies, xml ) Need Help??

If you've discovered something amazing about Perl that you just need to share with everyone, this is the right place.

This section is also used for non-question discussions about Perl, and for any discussions that are not specifically programming related. For example, if you want to share or discuss opinions on hacker culture, the job market, or Perl 6 development, this is the place. (Note, however, that discussions about the PerlMonks web site belong in PerlMonks Discussion.)

Meditations is sometimes used as a sounding-board — a place to post initial drafts of perl tutorials, code modules, book reviews, articles, quizzes, etc. — so that the author can benefit from the collective insight of the monks before publishing the finished item to its proper place (be it Tutorials, Cool Uses for Perl, Reviews, or whatever). If you do this, it is generally considered appropriate to prefix your node title with "RFC:" (for "request for comments").

User Meditations
Reading Perl documentation in Emacs
2 direct replies — Read more / Contribute
by haj
on Aug 08, 2022 at 13:53

    These days I came round to package an Emacs command which I am using since some time now: <M-x> perl-doc. It is a viewer for POD in Emacs, and it has been accepted in GNU ELPA, so it can be installed with <M-x> package-install.

    But why? There are already plenty of ways to read POD. They work (sort of), but I was not totally happy with any of them:
    • The perldoc command in a shell is nice. It knows what I've installed in my Perl, and I can add own projects to its search path by adding to PERL5LIB. It is not so useful with documents like the Moose::Manual which contain many cross-references.
    • Pod::Webserver is a nice way to get the same information in your browser. But it needs you to run two extra programs and does not provide an equivalent of perldoc -f split.
    • Pod::Perldoc::ToTk is supposed to display POD in a GUI, but the command in the synopsis perldoc -o tk Some::Modulename & fails with Undefined subroutine &Pod::Perldoc::ToTk::MainLoop called. I have Tk::Pod and Pod::Perldoc installed and don't want to chase that error.
    • Emacs has <M-x>cperl-perldoc which per default tries to get information for the thing where your cursor is, and displays the document in another window. But internally it uses man which isn't available on Windows, and until now I was too lazy to install any of the replacements (man for Windows, or woman.el. Also, it needs man pages installed, so I need to build those when I want to read documentation from sources I'm working on.
    • Emacs::PodMode (available via CPAN, not ELPA) is targeted for writing POD, it shows all the markup.
    • The Perl menu in CPerl mode still refers to Perl documentation in info format, which is no longer shipped with Perl and never was available for CPAN modules. Eventually these items should be deleted from the menu.
    • Edited to add (2022-08-09): In the meantime I found perl-pod-preview.el which also provides a man formatted view of POD. I like the fact that it works on (unsaved) buffers and might add a similar feature to perl-doc.el.

    So, trying to combine the good parts, <M-x>perl-doc defaults to the thing where the cursor is, respects PERL5LIB to find your POD (also accepts file names), does not need extra programs and has a decent formatting which allows to follow links between your documents.

How has your coding style changed over the years?
9 direct replies — Read more / Contribute
by stevieb
on Aug 06, 2022 at 21:42

    Since I started coding C and C++ in 2000, and Perl very shortly afterwards, my style hasn't fundamentally changed. If anything, I've simply become more pedantic about certain things. I'm bored so I thought I'd put a list together off the top of my head. What's your list look like?

    - Four space tabs!

    - I like no space between function parens:

    function($param1, \%param2);

    - I'm very much a K&R style person who likes 'else' type statements on their own line:

    sub function { if ($_[0] == 1) { do_something(); } else { do_something_else(); } }

    - When dereferencing a list, I always use whitespace:

    my @list = keys %{ $href->{thing} };

    - I always use the deref indicator where it is needed:

    my $thing = $hash{one}->{two}; my $other = $href->{one};

    - In my personal Perl code (60+ CPAN distributions), I always put the POD at the bottom of the file after all code (at $work, it's inline):

    package Blah; ... 1; =head1 NAME Blah - The thing with the guy and the place ...

    - I *ALWAYS* write unit tests before or at least during writing each function/method. I use the test scripts as test runs to prototype the code I'm writing instead of having a separate script. The test scripts are inherently part of the test suite. I also *ALWAYS* review my unit test suite for each piece of functionality and update it if necessary if patching/updating subs days, weeks or years later.

    - I (almost) *ALWAYS* write POD documentation as I'm writing the code (rarely before, but sometimes I do that too).

    - I frequently peruse the documentation of a random publication of my own software (regardless of language), and make fixes or produce relevant updates I may have missed.

    - I use the same editor all the time (vi/Vim) when I don't have my IDE handy, intelliJ IDEA (with vim support, of course). (VSCode for some of my work projects).

    - I rarely have to use the perl debugger, because I almost always find base issue cause through Data::Dumper statements. If I do use a debugger, it's more often for C code than it is for Perl code.

    - One of my favourite topics for new development is writing code to help other developers (including me). Introspection and unit test software is a big thing for me.

    - I love PPI, and am fluent enough with it that I rarely need to refer to the documentation when I want to use some of its functionality.

    - For my Perl code, I strive with great effort to achieve 95%+ unit test coverage, and that coverage generally covers that 95% 16 ways from Sunday. I often write additional code just so that tests can test code it just cant cover. This includes special constants, env vars etc. Some of my software has complete routines as add-ons just to get a single statement covered that otherwise couldn't have been.

    - I use Continuous Integration testing (CI) for almost everything. Mostly Github Actions (formerly Travis CI until they pissed me off tremendously), but some of my code can't run there, so I use my own Test::BrewBuild for such software.

    - I used to, but not so much anymore, review CPAN latest releases to try to find bugs to fix, even if its just documentation.

    - I am very specific about honouring other artist's copyright information. To further, I regard and honour the license of projects I've taken over from other authors. I'm a published author, poet, lyricist and music producer so the copyright thing in ingrained and imprinted. Appreciating other's art ownership isn't a desire to me, it's a core instinct.

    - I am diligent in acknowledging contributors to my software. My Changes files and commits generally have the persons name and/or CVS username embedded.

    - I take criticism very well; that said, I *ALWAYS* give credit where it is due, and *NEVER* claim credit for things I did not myself do

    - I take bug/issue/feature requests very seriously, and do my utmost to respond in as timely a manner as I humanly can (sometimes I don't, but that's very rare).

    - I use a bug tracker for almost everything I find myself; new features, real life bugs, security issues or even POD typos. If I'm perusing a random distribution of my own and I see a typo in the SYNOPSIS, I create a ticket for it.

    - I never use shift in an OOP module, I always use my ($self, ...) = @_;

    - I *ALWAYS* do parameter validation in each and every function/method.

    - I use pure perl OOP; very, very rarely do I ever use any of the helpers. The only time that happens is if I'm requiring a distribution that has that required already.

    - My POD format for subs is typically:

    =head2 method($param) Contemplates the reason for life, using common sense as a modifier. my $explanation = My::Continuity->new; my $thing = 'Reason for living'; my $reasoning = $explanation->method($thing); I<Parameters>: $param I<Mandatory, String>: The explanation of the formation of humanity in +a single string. I<Return>: Hash reference, where each key is a continent, and its valu +e is a colour. C<croak>s on failure.

    - I very sparsely use comments. Almost always my comments within code refer to *why* something is happening. It very rarely (if ever anymore) refer to 'what' is happening. The code says what is happening. Said comments when I make them are generally one to two lines only, and I use them only when I, myself will need to be reminded why in the fsck I did something so bizarre.

    I'm sure I can add a hundred other particulars I've formed over the years, but that's a start. How about you?

    Edit: Oh dear, I completely forgot. If it isn't blaringly obvious, Perl is my language of choice. Always has been, and I'm sure always will be. I'm decently fluent in C, C++, C#, wrote code in Python for four years as part of a job, can dabble my way through Javascript/JS, but I always lean back to Perl. Need an API for something (eg. Raspberry Pi)?, I'm making it available in Perl! New unofficial API for a new toy (eg. Tesla)? I'm ensuring it can be accessed with Perl! My priorities in life: My health, contentedness and happiness, my sobriety, my wife and children, Perl, everything else :)

RFC: A guide to installing modules for Win32 (2022 Edition)
9 direct replies — Read more / Contribute
by pryrt
on Jul 25, 2022 at 16:05

    Pursuant to Re^3: Can't locate Convert/BER.pm , here is a suggested version for "A guide to installing modules for Win32 (2022 Edition)"

    After any suggested updates (I don't pretend to be an expert, so feel free to correct and nitpick), would this go better as a reply to holli's original A guide to installing modules for Win32, or as a new top-level post in the Tutorials section?

    I included the case-sensitivity section because that was an issue in the recent thread Can't locate Convert/BER.pm, even though I'm not sure how general-use that note really is, or whether it really belongs in this tutorial.


    Updates: Here is a history of the edits made as a result of suggestions.

    1. Fix cpan arguments (per brian_d_foy's reply)
    2. Add perl Makefile.PL to the "standard recipe" (per syphilis's reply)
    3. Switch to a table, to avoid & (avoiding single & per many recommendations)
    4. Fix formatting syntax (per Discipulus's reply)
    5. Add a note about alternative build recipes (to cover Build.PL and possibly others)
    6. Add a note about incompatible modules
    7. s/skill/system/ in the last paragraph (per hippo's comment

    A guide to installing modules for Win32 (2022 Edition)

    Nearly two decades later, holli's excellent A guide to installing modules for Win32 could use some updated information.

    ActiveState phased out using PPM in 2021 (¹), so starting with ActivePerl 5.28, PPM is no longer included.

    And since 2008 (²), there's been an alternative distribution, Strawberry Perl, which comes with it's own gcc/g++ compiler and build environment. Modern Strawberry Perl versions not require using PPM for installation either (though it still ships with a PPM client, if you can find PPM repositories to use it with), so the original Win32 Guide's PPM instructions are not as useful as they once were.

    ActiveState's ActivePerl

    The modern method of "installing" modules on ActivePerl, as announced in The ActiveState Platform and Perl 5.32, is to make a binary build with the State Tool, where you tell it all the modules you need, and it will provide a binary build with all of those modules (and their dependencies) already installed.

    Strawberry Perl

    One of the benefits of Strawberry Perl is that they include a working gcc/g++ compiler and build environment, complete with a variant of make (dmake for older versions, gmake for newer versions; see perl -V:make to find out which your copy of Strawberry Perl uses) that means you can easily build and install modules similarly to how it's done on Linux and other OS:

    The default CPAN.pm client comes with Strawberry Perl, so installing Some::Module as easy as cpan Some::Module. Strawberry Perl also comes pre-installed with cpanm, an alternative CPAN client that handles dependencies, allowing installation with cpanm Some::Module. You can install cpanplus or other of the advanced CPAN clients on Strawberry Perl as well.

    Finally, if you are a traditionalist and want to manually build using the traditional recipe, you can look at the output of perl -V:make and then pick the appropriate variant of the recipe:

    traditionaldmakegmake
    perl Makefile.PL make make test make install
    perl Makefile.PL dmake dmake test dmake install
    perl Makefile.PL gmake gmake test gmake install

    Some modules may specify their own recipe for building and installing. If so, then try following their directions; the Strawberry build environment is pretty good. But if they specify a different recipe, it doesn't work, and the CPAN-client options don't work, you should file a bug report with the author, because any distributed module should be installable using cpan, cpanm, cpanplus and the like.

    For any of these installation techniques: If you have an installed copy of Strawberry Perl, your path should point to the Perl and C binaries already; if you have a portable copy of Strawberry Perl, you may need to run portableshell.bat to get the environment set up correctly.

    Other Alternatives

    If you have built your own Perl, or for Windows Subsystem for Linux, or cygwin, or other Windows tools that provide bash or bash-like environments, you should still be able to follow the instructions in the original A Guide to Installing Modules for installing, or use cpanm or cpanplus or other clients not mentioned in the original guide.

    Win32 Caveat: Module Case Sensitivity

    File names on Windows are not case-sensitive, so some Windows users are used to typing PATHS IN ALL CAPS. Do not type module names in all caps, even when using a CPAN client from the Windows command line (cmd.exe or powershell), as Perl and the CPAN tools will not treat SOME::MODULE and Some::Module the same, even if they resolve to the same ...\Some\Module.pm file.

    Windows Incompatibility

    Please note that some modules have been created in such a way that they are incompatible with the Windows operating system. This guide cannot help you install and use a module that is not compatible with your OS. (You can check the CPAN Testers reports linked from the metacpan.org page for each module: if it doesn't show any passing results on the mswin32 platform, you may have difficulty installing the module.)

I don't like annotation syntax
3 direct replies — Read more / Contribute
by Liebranca
on Jul 13, 2022 at 06:25

    Hello,

    By 'annotations' I mean 'attributes'. And by 'do not like', 'hate' and 'utter disgust that makes me wish for my own unborn demise' I actually mean it just doesn't click right (hey, that's a way to put it).

    And I wonder if anyone actually likes the :annotation(:attribute) syntax because I seldom see it outside of a *certain* OOP proposal that at least to me smells an awful lot like weird C++ fetish roleplay during which someone must've forgotten the safe word.

    Alright, hey, listen. I know that's too harsh. But if I want weird octopus dog for dinner, then I'll just have that. A 'class' keyword and all derived farts incumbent on the ever philosophizing theorymongers of objectification, I really do not care for.

    So why am I going on about this? I'm writing a preprocessor. It doesn't really introduce new syntax, just fake attributes that are removed before the compiler even gets to see them.

    The purpose of these attributes is simple: marking subroutines for inlining and variables as blessed references to one or another package -- just because I need to know where to pull a definition from before I get to inlining anything. It's a pinky-promise, not a typecheck.

    But this syntax...

    my $eyes :bleed = '<-whenever I see :this and :that';

    ... is frankly not my cup of tea. But I *do* wonder who likes it, if anyone, and why. I mean, I get it. I'm an outcast, a widower and I'm pissed. To say I'm out of touch is an understatement. So alright, I'm willing to listen to reason. Where is reason?

    I can transform an entire file, and therefore am entirely in possession of the arcane power to change the language itself into hexspeak spaghetti; I'd much rather work on something other people can read. Or more like, I'd much rather work on something that doesn't require you to basically relearn the language in order to use it.

    It must FEEL like it's still the same thing. That is why I care to do this:

    use inline; MyPackage::MySub(); # contents of sub will be inlined no inline; MyPackage::MySub(); # will not be inlined

    ^because it's what feels more natural, more Perl-esque, far as I'm able to tell. It's the same to perl the binary because the preprocessor in question strips those two use/no lines anyhoo, so they could just be whatever I say. I could just enforce Turbo C on 16-bit Windows rules because that's how I was taught way back when and it makes me feel nostalgic to #include <conio.h> and define functions inside a macro just because I can. Wouldn't that be silly of me?

    Yes, it would be silly. Sorry, this is a lot of ranting. I'm trying to think of a better way to make the pinky-promise to the preprocessor that a given variable is a blessed ref that isn't just simply adding the pinky-promise keyword and calling it a day. This is driving me crazy.

    If people feel annotations for this are alright then I must defer to their judgement. Mostly because I can't think of anything better that doesn't look like C. And that's that, you know.

    Have a good day.

Good day fellow monks. Back after a while
3 direct replies — Read more / Contribute
by Charles_okhumode
on Jul 13, 2022 at 05:31

    It's been a while i have being a member of perl monks. I have being on and off in my participation in here. I guess administrators would have picked up on my in ability to recaĺl my username, password,and email to my previous account.

    This is because i have being in and out of hospital and the mobile number i was using in the past for the email account i misplaced it and the sim pack. Making it difficult to retrieve the number.

    Now i am slowlly recovering and sooner rather than latter ill be back on my foot and able to be more active in here.

    I consider myself an intermediate perl programmer and have a working knowledge of unix/linux. I am also comfortable in systems asministration. My goal now is to major on web application development using perl.

    I have began my search for the right MVC i can use for this. I know there are quite a few. Please your guidiance and mentoring will be gladly welcomed.

    I hope my time in here will be fun and relatively peaceful. I wish to have a friend whom i can grow and share my learning experience with.

    Thanks and God bless.

Reflections after going through all PDL-featuring nodes on PerlMonks
3 direct replies — Read more / Contribute
by etj
on Jul 08, 2022 at 16:58
    As a few will have seen (I assume they look at "Newest Nodes" - and kindly upvoted some of my replies, for which many thanks), I've been posting replies to some old nodes on here. This was after deciding a couple of months ago that after looking at the front page each day and voting on various posts and replies, I'd then go back over all the nodes featuring the text "PDL" and vote on (to use the rest of that day's votes) / reply to ones where that seemed useful. I kept track of where I was up to by looking at "Nodes (I) Wrote" and seeing the date of the original thread I was replying to on my most-recent post.

    Some of the things I have learned:

    • of a few non-reported bugs in PDL that are now fixed
    • of a few former keen PDL users who now apparently don't visit PerlMonks anymore
    • back in 2001, there was a thing called "Program Design Language" which gave some false positives on my searching
    • protein sequences using the https://en.wikipedia.org/wiki/Protein_primary_structure coding gave some false positives when folks using Perl for biology stuff included some data with the string "PDL"
    • Despite PDL being around since 1997 in various forms, it only started getting discussed on here in mid-to-late 2000 or so
    It's been really interesting to take this journey back through time, and see how Perl, and the PerlMonks site, and PDL have developed over time.
perl 5.36 and the for_list feature - a simple speed comparison
1 direct reply — Read more / Contribute
by swl
on Jun 21, 2022 at 05:32

    Perl 5.36 has a new feature that iterates over multiple values at a time in a for loop (perldelta entry here). Any number of values can be specified, but the pairwise case is the focus here.

    The for_list feature is a useful alternative to a while-each idiom over a hash when working with key-value pairs. It also allows one to natively work with arrays of key-value pairs without converting them to a hash (and thus losing values associated with duplicate keys).

    Speed is one factor in the decision to updated code, so I figured I would see how fast the new feature is compared with while-each and some other approaches using List::Util functions. Some benchmark code is below (inside readmore tags), with results following.

    Benchmark labels starting with a_ operate on array data while those with h_ operate on hash data.

    I note that it is fairly well documented that the while-each idiom is best avoided because the hash iterator has global state and thus can cause action at a distance. In the benchmark code the hash is only accessed this way in one sub.

    The summation of the hash values in the loops is merely something for the loop to do that would have relatively low overhead compared with the looping itself.

    The hash_vals sub is done as a point of comparison. There is no reason why one would iterate over both keys and values when only the values are needed.

    Code was run under Ubuntu 20.04 under WSL2 for Windows.

    The main conclusion is that the new feature is faster than all the others when an array is used. It is faster than all of the hash approaches that use both keys and values. Profiling shows the cause of the slowdown when using List::Util::pairs is the dereference of the pair array, which is not surprising. Using the declared_refs feature does not help in this case.

    Overall I quite like the new for_list feature. Aside from being faster than many of the alternatives, it is also much cleaner.

    Results:

    2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 2500050000 Rate a_pairs_alias a_pairs h_by_key h_each h_for_list +h_vals a_pair_vals a_for_list a_pairs_alias 87.9/s -- -9% -19% -35% -51% + -82% -83% -89% a_pairs 96.5/s 10% -- -11% -28% -47% + -80% -81% -88% h_by_key 109/s 24% 13% -- -19% -40% + -77% -78% -87% h_each 135/s 53% 40% 24% -- -25% + -72% -73% -84% h_for_list 181/s 106% 88% 66% 34% -- + -62% -64% -78% h_vals 477/s 443% 394% 338% 254% 164% + -- -6% -42% a_pair_vals 506/s 476% 424% 364% 275% 180% + 6% -- -39% a_for_list 827/s 841% 757% 659% 513% 357% + 73% 63% --
20 is a round number
4 direct replies — Read more / Contribute
by Discipulus
on Jun 13, 2022 at 07:54
    Hello dear community,

          Happy Monkday!!1!
          You've been here 20 fruitful years.
          Haven't you grown up yet?
    

    is the message I've found today entering the halls of our lovable place.

    I planned for today a spectacular obfuscation, but lacking currently the inspiration I just have for you a little resume of my partecipation.

    Yes: I grown up here. It was in 2002 when I moved my first baby steps in the Programming Art, Perl declination of it, so poor in English and in Perl that I wrote my code in Italian.

    During these early years I attempted to automate as much as possible my daily job with windows, IIS et similia. My scripts incredibly worked even in this ugly environment. My mad vein was already there, for exmple I wanted to modify my source at runtime: substitution in $0.

    In this early stage I used a lot of CGI and I started having fun with Tk under the mentorship of master zentara.

    At that time node reputation was in hundreds of votes: many brilliant hackers here and my lack of understanding them.. nice days :)

    Then, starting from Murder of a Perl coder (announced), I had bad times with Perl at work, with only eleven posts here during 2005-2008.

    Then, since 2009, a personal rebirth: Perl is too much addictive to be killed by job bosses: I published here the first naive CUFP: when no ping say 'moo!' and I used it to spot the right eth cable in the datacenter, detaching them in serie, in the time no cable were numbered.. a mess :) This was the first of 23 CUFP.

    New techniques, modules usage and a modern Perl for me: I started coding my own modules.

    Among my personal Cool Uses for Perl you can find the acme of my heresy: the biggest string eval ever attempted by a coder: CPAN Namespace Navigator evals the whole 02packages.details.txt from CPAN. I'm still proud of it.

    In 2012 my Ten (years) Here marked the beginning of yet another era: I found that I had something interesting to share with you and that I'm able to give a little help to newcomers, always as humble amateur.

    The majority of my Meditations are since then. As heretic perl programmer I started an incomplete serie: Antiquitates - liber I and Antiquitates - liber II aiming to link ancient wisdom with our beloved perl art. Worth to notice also Binary Comedy -- 01 - 000001 where I poetically met Larry in Dante's Inferno.

    After fifteen years here it started again a time of troubles, with work involving in a mess with absolutely no more room for my perl, now that I'm almost ready to become a professional perl coder, simply the perl job marked is something not existing anymore.

    But dont give up! During this golden age you can find my 12 Obfuscations and my Poetry.

    In the same years I became a oneliner addicted with 76 oneliners published here, let me feeling the younger of the old hacker generation and the acme of ameteur perl coders.

    Thanks you all for these 7305 amazing days!!

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
Work-around for Gmailpocalypse
1 direct reply — Read more / Contribute
by Aldebaran
on Jun 02, 2022 at 20:41

    Hi Monks,

    I just wanted to write up what happened with smtp.google.com on the 30th of May. Auth 2.0 came into effect, which deprecated use of a username and a password, requiring instead an app key. An effective work-around is described in Re^3: sending an email from my rpi to my droplet [gmail workaround].

    I found the error message instructive:

    $ echo "BEEP BEEP" | mailx -s "Subject: This is a test!" tallharry84@g +mail.com send-mail: authentication failed (method PLAIN) send-mail: server message: 534-5.7.9 Application-specific password req +uired. Learn more at send-mail: server message: 534 5.7.9 https://support.google.com/mail/ +?p=InvalidSecondFactor s8-20020a62e708000000b0051835ccc008sm2037121pf +h.115 - gsmtp send-mail: could not send mail (account default from /home/pi/.msmtprc +) send-mail: cannot log to /var/log/msmtp.log: cannot open: Permission d +enied send-mail: log info was: host=smtp.gmail.com tls=on auth=on user=tallh +arry84 from=tallharry84@gmail.com recipients=tallharry84@gmail.com sm +tpstatus=534 smtpmsg='534-5.7.9 Application-specific password require +d. Learn more at\n534 5.7.9 https://support.google.com/mail/?p=Inval +idSecondFactor s8-20020a62e708000000b0051835ccc008sm2037121pfh.115 - +gsmtp' errormsg='authentication failed (method PLAIN)' exitcode=EX_NO +PERM Can't send mail: sendmail process failed with error code 77 $

    Typing in the 16-letter app output in the place of the password does the trick.

    I went ahead and opened up the site they mention and found it worth a read: Signing in with App Passwords. The world is a-changin' as far as authentication goes. Passwords are no longer for sticky notes on the wall. 2-factor auth is necessary, and if you don't have a phone number to give google, they're not gonna let you play.

How to get programming help
7 direct replies — Read more / Contribute
by jdporter
on May 23, 2022 at 11:21
    "Every time I have a programming question and I rly need help, I post it on PerlMonks and then log into another account and reply to it with an obscenely incorrect answer. Ppl don’t care about helping others but they LOVE correcting others. Works 100% of the time"
    -- @soychotic
Libraries and security
2 direct replies — Read more / Contribute
by davies
on May 23, 2022 at 10:25

    An article in today's Register, https://www.theregister.com/2022/05/23/npm_dependencies_vulnerable/ has got me thinking. One quote from it is "These 2,415 people with these email addresses are currently more trusted than your own software engineers as they can effectively run any code they want on your production infrastructure without review from anyone". When the Log4J vulnerability appeared, I found out (I hardly ever use Java) that the offending module could be in any .jar without it being obvious to the programmer. At least, when I install a CPAN module, I can see a list of the modules it brings with it, even if I don't usually pay enough attention. But sometimes I need to be very security conscious.

    Looking for vulnerabilities in CPAN took me to https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=cpan. There's not a lot there, and many of them are closed in the latest version. But looking at one of them, https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-4184, I see that the module was updated <update>last month two years ago, seven years after the vulnerability was reported</update>, but that there is nothing in the changes file to indicate that the vulnerability has been addressed even though it's nine years old. Nor is there anything in the docs even acknowledging the existence of the vulnerability.

    The changes file does, however, reveal another worry.

    1.148 Thu Nov 16 10:21 2006
    - Debian has chosen to distribute their own Data::UUID, which has a different interface and breaks other modules.

    What, then, do people do in the real world? Are checks for CVEs enough? Do people do code reviews of every module they use in every language? It sounds beyond the bounds of reasonability to me. Are there certifications for modules (and if so, who certifies the certifiers - that old question)? I know people who prefer to install Perl packages on Debian via apt because it makes them easier to delete, but the quote above makes me fear that the CPAN and apt packages need not be the same.

    Regards,

    John Davies

    Update: I read the date carelessly & got it wrong. Thanks, Hippo.

RFC: Creating Dancer2 validation plugin.
3 direct replies — Read more / Contribute
by AlexP
on May 04, 2022 at 10:30

    Hello monks, this is requests for comments, so if you are interested in this topic, please leave any.

    Preface

    Dancer2 is a lightweight framework that provides us with rich routing and serialization features out of the box. The rest is left to the developer, including working with the database, models and data in general.

    When working with data, especially received from a user, you want to be sure of their consistency. In other words, I don't want to get a number where the username should be and the word in the age field (like age => twenty).

    Perl is a language with very powerful regular expressions and many hackers use them. However, this approach has the following disadvantages:

    • It is not declarative, you need to validate all fields manually
    • It's hard to reuse
    • Code is cumbersome and hard to read

    Instead of writing if ($age !~ /^\d+$/) {…} else {…} and so for each field received from the user, I want to use a construction like (age => 'integer').

    And here you should pay attention to special modules that simplify the validation process.

    Modules

    I've found some useful modules on cpan:

    They all have their pros and cons, but none of them met all my criteria.

    Criteria

    1. Control flow the field to the validator, and not vice versa

    This means that the code used fields as first class entities:

    { email => qw(required, email), age => qw(integer, min:18), }

    And not validators:

    { required => qw(email), email => qw(email), integer => qw(age), min => { filed => ‘age’, num => 18 }, }

    2. Required is validator too

    Very often you can see:

    { required => … , validators => { … } }

    Requred is the same validator and it is not clear to me why to extract it into a separate entity.

    3. Has a set of built-in validators

    Not all modules have built-in validators and you need to implement them yourself.

    4. Easily extendable with new modules or custom code

    This feature is provided by most modules.

    5. It has a minimal set of dependencies and does not require the mandatory use of DBIc or Moose

    Because we have a Dancer2 project and maybe we don't use DBIc models at all.

    6. Multilingual support

    Didn't find it anywhere.

    7. Tightly integrated with Dancer2 and provides a set of handy dsl words and html templates tokens

    I do not want to constantly get and transfer data to the template like errors or old input.

    Dancer2::Plugin::FormValidator

    Based on the above criteria, I decided to write my plugin specifically for Dancer2. Since this is my first big OS project, I ask respected monks to share their opinion on what I got as a result.

    use Dancer2; use Dancer2::Plugin::FormValidator; package RegisterForm { use Moo; with 'Dancer2::Plugin::FormValidator::Role::Profile'; sub profile { return { username => [ qw(required alpha_num_ascii length_min:4 + length_max:32) ], email => [ qw(required email length_max:127) ], password => [ qw(required length_max:40) ], password_cnf => [ qw(required same:password) ], confirm => [ qw(required accepted) ], }; } } post '/form' => sub { if (validate profile => RegisterForm->new) { my $valid_hash_ref = validated; save_user_input($valid_hash_ref); redirect '/success_page'; } redirect '/form'; };

    Plugin can be found on Metacpan => Dancer2::Plugin::FormValidator.
    Github repo => dancer2-plugin-formvalidator.

    I am think about the following questions:

    • How convenient is the dsl keywords interface? Is the word validated redundant? Moreover, it uses a global variable. But it looks helpful.
    • How complete is the set of basic validators, is it worth expanding it, with what?
    • In tests, pieces of code are often repeated, is there a generally accepted method of reusing them?
    • Is the documentation clear, what should be added or removed?
    • Are the interfaces friendly to extensions: Extension.pm, Validator.pm
FY411: Missing Perl on package managers for Mac and Windows
1 direct reply — Read more / Contribute
by perlfan
on May 03, 2022 at 14:12
Pangram challenge: greed and Scrabble
2 direct replies — Read more / Contribute
by Athanasius
on Apr 24, 2022 at 10:43

    Task 2 of the current Perl Weekly Challenge is to generate a pangram from the English dictionary provided. “A pangram or holoalphabetic sentence is a sentence using every letter of a given alphabet at least once.”1 However, for this challenge the requirement that the words form a sentence is removed: a list of unrelated words is all that is required.

    The bonus challenge is to “Constrain or optimize for something interesting,” and the first suggestion given is:

    Shortest possible pangram (difficult)

    Ok, so just how difficult would it be to perform an exhaustive search? I don’t know the answer, but since the search space increases exponentially as words are added to the dictionary, a search for sequences (of varying lengths) drawn from a pool of 23,610 words (see below) would appear highly impractical. I haven’t attempted it.

    Before we proceed, it should be noted that the dictionary can be significantly reduced in size before the search begins. Where groups of words share the same set of letters, only one word from the group (the shortest) need be kept. For example, “abdicate,” “diabetic,” and “abdicated” all have the letter set {a, b, c, d, e, i, t}, so “abdicate” is retained but “diabetic” and “abdicated” are filtered out. This reduces the dictionary size from 39,172 to 23,610 — a saving of 15,562 words!

    As noted above, 23,610 is still far too many words for an exhaustive search. So we’re looking for a way to get a close-to-optimum result in a reasonable time. And as we’ve just noted, each dictionary word can be considered as a set of its component letters. So the task is to find a minimum set cover, a well-studied problem in combinatorics and computer science.2 And there is an algorithm that is known to give good results in polynomial time: the greedy algorithm:

    at each stage, choose the set that contains the largest number of uncovered elements.

    My initial implementation of this algorithm produced the following pangram:

    oversimplification ladybug hawk jazz equinox (40 letters)

    I wasn’t satisfied with this, so began looking for ways to improve the search. It occurred to me that words with rare letters should be preferred over words with common letters. This immediately suggested the Scrabble3 points system:

    const my %ALPHABET => ( a => 1, b => 3, c => 3, d => 2, e => 1, f => 4, g => 2, h => 4, i => 1, j => 8, k => 5, l => 1, m => 3, n => 1, o => 1, p => 3, q => 10, r => 1, s => 1, t => 1, u => 1, v => 4, w => 4, x => 8, y => 4, z => 10 );

    With this I was able to give each word a weighting based on its component letters, and prefer words with higher weightings. This produced the following pangram:

    sympathized quarterbacking reflexive jawbone (41 letters)

    — no improvement at all! The problem appears to be that the selected words contain too much deadweight — too many redundant letters. So would it help to adjust the weighting for each word by deducting points for redundant letters? I experimented with different values of $DUP_WEIGHT, the amount to deduct for each redundant letter. Here are the results:

    $DUP_WEIGHT Pangram Letter Count
    1 sympathized quarterbacking jinx overflow 37
    2 sympathized quacking overflow jinx be 33
    3 sympathized quacking fox jaw verb all 32
    4 sympathized unblock xv jaw qt frog 29

    Eureka! At 29 letters, the last pangram is only 3 letters above the theoretical best-possible result of 26. (For values of $DUP_WEIGHT above 4, the result does not change.)

    It should be emphasized that my improved results are not an improvement on the greedy algorithm. That algorithm applies to sets, whereas my search for pangrams is a search over words, which are essentially bags because they contain multiple instances of the same letters.

    For the implementation, I found the Set::Tiny module very useful. I highly recommend it for working with sets of strings.

    Wikipedia1 gives the following as a short pangram:

    Waltz, bad nymph, for quick jigs vex. (28 letters)

    Since all these words occur in the given dictionary, it would be possible for a search to improve on my best solution by at least one letter. Challenge: can anyone implement a search which finds this pangram (or a shorter one) within a reasonable time?

    Full code listings for my solution (in Perl and Raku) will shortly be are available on the Weekly Challenge’s GitHub repository at https://github.com/manwar/perlweeklychallenge-club/tree/master/challenge-161/athanasius.

    Cheers,

    1“Pangram, ” Wikipedia, https://en.wikipedia.org/wiki/Pangram
    2“Set cover problem,” Wikipedia, https://en.wikipedia.org/wiki/Set_cover_problem
    3 “Scrabble letter distributions,” Wikipedia, https://en.wikipedia.org/wiki/Scrabble_letter_distributions#English

    Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

Trading compile time for faster runtime?
7 direct replies — Read more / Contribute
by melez
on Apr 20, 2022 at 09:54

    Perl optimizations are pretty basic, and it compiles fast - which is great for utility scripts, but sometimes it feels like a waste of potential. A lot of Perl code actually runs persistently, either behind mod_perl, plack server, mojolicious server or just as a daemon.

    I've been thinking - what if we had a switch (executable flag?) that would turn on additional set of compile time optimizations? It could hypothetically be a big performance win, and not affect programs that need to boot fast. I have close to no experience in XSUB, so that's a big if.

    For example, one of my projects has I think close to 100 .pm files and compiles in 800 ms. It will then run for a long time (in production) and require performance. Other than testing and development, I wouldn't mind spending 10 seconds extra if that would mean some crucial parts would run 50% faster. Or maybe I could care less about using small subroutines in tight loops, as they would get inlined.

    Comments? Was it attempted already?


Add your Meditation
Title:
Meditation:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (4)
As of 2022-08-09 16:52 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?