Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

[NTF] Nice Perl ideas I have no time for

by Discipulus (Canon)
on Sep 07, 2023 at 09:32 UTC ( [id://11154284] : perlmeditation . print w/replies, xml ) Need Help??

Hello dear community,

Being our venerable halls quiet nowadays I propose this meditation to share ideas of programs, modules and everything you want we have no time enough to develop them further.

In my world ideas have no copyright and should instead circulate freely as there is the chance they are grasped by an enlightened soul who can squeeze the best from them.

Even more: there are amateur programmers with nice ideas and professional ones with few ones. It is not something to complain about, we are different brains with different skills, inclinations and.. free hours :)

I'd like to see at least some demo code for these ideas with the goal well explained as any possible path of implematation or critical parts, not just: /I'll save the world with a oneliner/.

We can use a tag for these post like [NTF] (No Time For) and post them in reply at this post or as new Meditation.

I'll start with a first one if are ok with this ..nice perl idea :)


There are no rules, there are no thumbs..
Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

Replies are listed 'Best First'.
Re: [NTF] to check md5 of loaded modules
by Discipulus (Canon) on Sep 07, 2023 at 11:28 UTC the first [NTF] born from a chatboard idea discussed among bliako, Corion and me about..


    ..checking the md5 checksum of every module loaded.

    A possible solution and tools for

    Probably checking the md5 of every path in @INC would be easier and faster (using tar you can also check permissions) but checking every module is more granular, allows to specify a white/black list and is funnier :)

    Basically as explained in require documentation prior to 5.37.7 we can use the hook putting a sub inside @INC like in: unshift @INC, sub { my ($coderef, $filename) = @_; ... } (and is the easier case ;) to have something done just before paths are searched for the requiered module. It is a trick I forgot.

    In perldelta for v5.38.0 there is the new %{^HOOK} API and actually require__before and require__after are available: nice and fun!


    How to store the file containing the md5 checksum to check against? It should be protected to be useful. My idea (uniplemented) is that brings up a password request as first thing to decipher a protected md5-checksums.txt file, or a SQLite db.

    The module needs to use some module to run, so these are checked AFTER they are loaded and this can be a security hole in paranoic world.

    The @INC array can be maliciously modified by other modules so should be saved soon and used to scan for file to load (not implemented, olny my @original_INC = @INC; in my code). Probably the module should check the existence of the file using @original_INC and do it returning the value and then populate %INC ..just to be very paranoic :) My traverse_INC sub is almost empty.

    The nice starts with HOOKs and this part is totally unimplemented.

    Code has a bounch of subs and a big BEGIN block. After the initial check I check brutally $^V ge '5.38.0' to spot which hook to use.

    The md5-check.txt is a simple file

    ..and the script is simple as:

    use strict; use warnings; use List::Util;

    ..finally the command invocation is: perl -I. -MParanoic to be paranoic as soon as possbile.


    For a month the demo will be available at the nice PerlBanjo website. The checksum are correct only for the 5.36.1 version, so you'll see errors (have to be die in the code) for 5.38

    Here the output for future reference:

    BEFORE any hook I will check md5 of already loaded module: OK at /usr/local/lib/perl5/5.36.1/ has the ex +pected md5: 9ac6b836ee45f6e08e5c8a84cee5e619 OK at /usr/local/lib/perl5/5.36.1/ has the ex +pected md5: 7167a8489aafb9faddbbe48c6480f47c WHITELIST for at [f929845aba01aa4bf162a15cc2 +54c123] OK at /usr/local/lib/perl5/5.36.1/ has the expect +ed md5: 31b6105d6dc1cde54154291b86c8b285 OK at /usr/local/lib/perl5/5.36.1/x86_64-linux/ has the + expected md5: 8f620379a0649ad32f14f1ce50b88bc0 OK Digest/ at /usr/local/lib/perl5/5.36.1/Digest/ has +the expected md5: b5de2696c583dfec247af39b45288735 OK at /usr/local/lib/perl5/5.36.1/ has the ex +pected md5: 56cde6eba0f667ab56196613df3933c1 OK File/Spec/ at /usr/local/lib/perl5/5.36.1/x86_64-linux/Fil +e/Spec/ has the expected md5: bf252d457a243d20eabbd91292fcf3f4 OK Digest/ at /usr/local/lib/perl5/5.36.1/x86_64-linux/Digest/ has the expected md5: d75a3d708ce93ad8d99fcbdefa2c8429 OK File/ at /usr/local/lib/perl5/5.36.1/x86_64-linux/File/Spe has the expected md5: 7be482dda6bd364dd65e286b24cd8691 OK warnings/ at /usr/local/lib/perl5/5.36.1/warnings/regi has the expected md5: 2d8f6ce093a2176b982c0e12c0194b3b OK at /usr/local/lib/perl5/5.36.1/ has the ex +pected md5: 74a2550b5b0731996c0c825930003013 AFTER I will use some hook to check md5 of modules loaded by the calli +ng program ====> Perl v5.36.1 using @INC here.. Paranoically considering [List/]

    The 5.38.0 output is different in the final part:

    AFTER I will use some hook to check md5 of modules loaded by the calli +ng program ====> Perl v5.38.0 using $^HOOK here.. SKIP [] already processed SKIP [] already processed Paranoically considering [List/] SKIP [] already processed SKIP [] already processed SKIP [] already processed SKIP [] already processed SKIP [] already processed


    Have fun developping this Perl idea and share your progress!


    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

      MD5 is a pretty old hash format and hasn't been considered especially secure for about a decade.

      Module::Signature switched to SHA256 about five years ago, so switching to that too might be a good idea. Especially as this means that any recent CPAN distributions packaged with Module::Signature in mind will include a SIGNATURE file (an example!) GPG-signed by the author, listing the SHA256 hashes for every file in the distribution including all modules.

      Intercepting DynaLoader::dl_load_file(...) to verify the binary bits of XS modules would make this a lot more robust. I'd be more worried about a virus or something being written to inject code in a lib/dll than into a pure Perl module anyway.
      BEGIN { require DynaLoader; # no strict 'refs'; no warnings 'redefine'; my $keep = \&DynaLoader::dl_load_file; *DynaLoader::dl_load_file = sub { my ( $path, $flags ) = @_; warn "We should check '$path' here"; &$keep(@_); }; } # Random XS based core modules use Cwd; use Fcntl; use Digest::MD5;
      I guess you'll need to think about FFI loaded libraries eventually.
Re: [NTF] Nice Perl ideas I have no time for
by stevieb (Canon) on Oct 13, 2023 at 06:44 UTC

    When I started in Perl, sometime around 2000, I did so trying to automate system administration tasks. Shortly after, I started falling in love. Then I was on a hunt for things to build, just for fun. I'm sure I asked "what can I do" to the old time greats, where the response was often "find distributions, fix bugs, write patches" (where that advice was a Godsend actually in the long run).

    I think every budding developer goes through a period where they try to imagine what they want to dream up.

    Now, there's no hardware I can't conquer, no API I can't wrap, no challenge that's too big to put effort into. With that said...

    a Perl idea that I have no time for is one that can dream up new ideas for people like me in my earliest years while I was looking for them.

Re: [NTF] Nice Perl ideas I have no time for
by misc (Friar) on Oct 11, 2023 at 11:38 UTC
    There are mainly two ideas, it doesn't seem I'll have the time..
    Both imho good, but.

    Write a script to get completely selfbundled perl scripts.
    I do know par packer, and so on, but I think about having the perl interpreter installed at the target,
    and just put the script with all used modules included at the target.
    Yet I copied all used modules by hand into one file, what is .. tedious :(

    The other one, a text user interface library. Which is usable, and portable.
    Curses is incoherent, quite often the different curses interfaces are missing something, and so on.
    Perhaps starting with a really simple C interface, like tbox.
    Having all other things implemented in perl, in an object oriented manner.
    The things out there I found are either not finished, or not that nice to use.
    Would be wonderful, but I'm faced with other problems, so..

      Your first idea sounds very much like App::FatPacker. Perhaps that's all you need?


        .. :o perhaps. ;)
        Thanks a lot, somehow I didn't find this.

        About the second idea, here's the TO I did write some 16 years ago.
        console editor