Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Meditations

( [id://480]=superdoc: print w/replies, xml ) Need Help??

If you've discovered something amazing about Perl that you just need to share with everyone, this is the right place.

This section is also used for non-question discussions about Perl, and for any discussions that are not specifically programming related. For example, if you want to share or discuss opinions on hacker culture, the job market, or Perl 6 development, this is the place. (Note, however, that discussions about the PerlMonks web site belong in PerlMonks Discussion.)

Meditations is sometimes used as a sounding-board — a place to post initial drafts of perl tutorials, code modules, book reviews, articles, quizzes, etc. — so that the author can benefit from the collective insight of the monks before publishing the finished item to its proper place (be it Tutorials, Cool Uses for Perl, Reviews, or whatever). If you do this, it is generally considered appropriate to prefix your node title with "RFC:" (for "request for comments").

User Meditations
The intersection of M hyperplanes (Ndim)
3 direct replies — Read more / Contribute
by bliako
on Jul 18, 2024 at 09:41

    Here is my understanding of what the theory behind what I am trying to do is, correct me where I am wrong, also please check my questions at the end:

    The intersection of N hyperplanes (Ndim) is also known as the solution to "the system of N linear equations(=planes) with N variables". However, I am interested in the case when I have less equations. The result is not a single point in Ndim but a lower-dimensions plane. For example in 3D. The intersection of 3 planes (M=3, N=3), assuming any pair is not parallel, is a single 3D point and the intersection of 2 planes (M=2, N=3) is a line. In Ndim, the solution would be an Mdim hyperplane "living" in Ndim. Below, I call this a "hyperline".

    An example of the above problem is in this SO question. The result is the equation of the intersection line in parametric form. Arbitrary values of the parameter t yields a point on this line. Ideally this is what I want to achieve: finding points on the intersection of the planes.

    Solving the system of N linear equations with N variables (Ndim) can be done with, for example, Math::Matrix. Given N planes in the form a1x+b1y+c1z+...n1=0 etc., a matrix A is formed with each row being [a1, b1, c1, ..., n1] etc. then: Math::Matrix->new([[a1,b1,c1,...,n1],...])->solve->print;

    I have experimented with solve() for the case of M linear equations (Ndim) where M = N-1:

    use Math::Matrix; Math::Matrix->new( [1, 2, 1, -1], # ax+by+cz+n=0 [2, 3, -2, 2] )->solve->print; # says # -7.00000 7.00000 # 4.00000 -4.00000

    I interpret the result as: the 1st row refers to the 1st dim (aka x) and we have

    x -7*t + 7 = 0 => x = 7*t - 7
    The 2nd row yields
    y + 4*t -4 = 0 => y = -4*t + 4
    and the missing 3rd dim is the parameter t itself: z = t. And I can get a point on that line by setting t to an arbitrary value. I can verify that this point is on each plane by substituting the point's coordinates into each plane equation.

    I have also experimented with transforming the planes matrix to the Reduced Row Echelon Form. Which makes it convenient to work out the "hyperline" equation. This functionality is offered by Math::MatrixLUP :

    use Math::MatrixLUP; my @planes3D = ( [1, 2, 1, -1], # ax+by+cz+n=0 [2, 3, -2, 2] ); print Math::MatrixLUP->new(\@planes3D)->rref # says # [1, 0, -7, 7], # [0, 1, 4, -4]

    Which I interpret as: 1st row has 1 on 1st col, that's the 1st dim (aka x) :

    x -7 * t + 7 = 0 => x = 7*t-7
    etc.

    So that works well.

    Here is a demo I whipped up:

    I have some questions:

    • Two planes are parallel when their normal vectors (the coefficients of the dimensions: a, b, c, ... (but not the constant n) are multiples. E.g. nv1 = k * nv2. And this translates to checking if all the ratios of the coefficients : a1/a2 = b1/b2 = c1/c2 = ... are the same. My question is: what happens if any coefficient is zero (or actually both coefficients (e.g. a1 and a2) are zero?
    • How can I calculate the intersection when M < (N-1)? I.e. above I am always checking the intersection of M planes in Ndim where M = N-1. But what if there are even less planes? e.g.
      my @planes5D = ( [1, 2, 1, -2, 3, -1], # ax+by+cz+n=0 [2, 3, -2, 4, -4, 2], [3, -1, -2, -5, 6, 2], # only 3 planes (it was successful with 4) )
      In short, does the parametric equation of the intersecting "hyperline" contain 2 parameters now?
    • When I test test_in_beast_space which produces random planes in 666 dimensions, it fails to detect that a point on the intersection "hyperline" lies on all the planes. To do that I substitute the point in each plane equation and expect to have result as zero. However, it's not an exact zero. That's why I am doing a range test to check if it's close to zero. Well the closeness $SMALLNUMBER for 5 dimensions can be as small as 1E-09. But for these many dimensions it can be as low as 1E-02. Is the accuracy lost in summing 666 multiplications that much really?
    • I guess Math::Matrix::solve() is safe to be used for MxN matrices (e.g. M equations=planes in N unknowns (dimensions) where M<N)? From solve() of Math::Matrix :
      Solves a equation system given by the matrix. The number of colums mus +t be greater than the number of rows. If variables are dependent from + each other, the second and all further of the dependent coefficients + are 0. This means the method can handle such systems. The method ret +urns a matrix containing the solutions in its columns or undef in cas +e of error.
    • The edge cases where the 1st dimension is zero for all planes in 5D and when the 1st+2nd are zero, fail. How can I find the intersection in this case?

    Edits: updated the demo to correct the sub are_planes_parallel() as per the comments of hippo and LanX about what happens when plane-equation coefficients are zero.

    have fun in beast space! bw bliako

Who's still around?
17 direct replies — Read more / Contribute
by stevieb
on Jul 08, 2024 at 03:29

    In 1998, a friend gave me a computer. It was in pieces. I knew nothing about nothing. Within a year, I was communicating over dual telephone lines. Within two years of that, I was a sysadmin at an ISP. This was the year I found Perlmonks. I read a book, 'Perl in 21 days' or some such, and found Perl was what I wanted... a way to automate processes.

    Within months, I learned a dangerous amount about MySQL, CGI and Perl to allow any invader to break everything. Thankfully, during that time, everyone was out for themselves, and exploitation hadn't yet become a thing.

    By 2009, I'd grown a lot in many areas. No where near perfect in the security arena, but I was becoming proficient on how to interact with the open source world, and how to interact with the CPAN. It was this year that I joined Perlmonks as a member, and became a vocal person, not just a listener.

    Now, as some of the old timers will attest to, I always claimed "I'm not a programmer". With that said, I have done much work in fields so closely related to programming, that I have to bend and say that yeah, maybe I can classify as a hacker.

    Anyone else around who have claimed "I'm not a programmer", or who has been around since the very early days of Perlmonks who would just like to say "I'm still here!!!"?.

    -stevieb

Houston Perl Mongers Meeting Announcement Email List (new)
1 direct reply — Read more / Contribute
by oodler
on Jul 06, 2024 at 01:23
    Houston PM's private, self hosted email list (using Sendy/AWS SES - PHP sorry xD) - monthly meeting announcements will be from noreply@houstonperlmongers.org and reply-to is to brett.estrade@gmail.com. I'm working getting the link/form up on the Houston Perl Mongers site, https://houstonperlmongers.org (or houston.pm.org). Keeping up with all the outlets is too much, so going back to email and website announcements. I may still post here; July's meeting has not yet been set.

    Cheers
Generate random strings from regular expression
4 direct replies — Read more / Contribute
by bliako
on Jul 02, 2024 at 06:41

    I needed to generate random strings from a regular expression and I have found (*) a C++ library (regxstring C++ library by daidodo) which does just that and works fine for my use case. It claims that it supports most Perl 5 regexp syntax. It is quite fast too. So I have ported it into a Perl module, here : String::Random::Regexp::regxstring.

    perl -MString::Random::Regexp::regxstring -e 'print @{generate_random_ +strings(q{^\d{3}[-,.][A-V][a-z]{3}\d{2}})};'
    use String::Random::Regexp::regxstring; my $strings = generate_random_strings('^[a-c]{2}[-,.]\d{3}-[A-Z]{2}$', + 10); print "@$strings\n"

    The XS and C++ code bridging Perl to C++ library are very simple and can serve as an example for doing that for other libraries. Go forth and multiply.

    (*) Via this old thread I found Regexp::Genex and there is also String::Random. Neither worked for my use case.

    bw, bliako

[OT] Smartphone IO interface: how to control motors and read sensors
1 direct reply — Read more / Contribute
by bliako
on Jun 21, 2024 at 06:46

    I was thinking what my options are for controlling a remote farm remotely.

    I needed to do basic stuff like run the motor to push the feed to the animals, open a valve to let mains water fill their drinking buckets, check feed and water levels, check temperature. Even count the number of eggs.

    The farm has no electricity is off-grid and has no alternative power installed, so it must rely on solar energy. I want to minimise the use of batteries. There is no WIFI, but it is covered by the national 4,5G telephone network.

    A smartphone has all the communication modules available and ready: sms, voice, data, bluetooth, even rfid. It also has a good power supply and provided is not loaded with apps, it can last for 24 hours at least for the next sunshine for the solar charger to work. And it allows high-level programming.

    Unfortunately a smartphone lacks any connectivity to the world by means of IO ports. And so I am looking for an IO board to the smartphone via its USB or perhaps the bluetooth. The cheaper the better really and simpler too.

    I found IOIO-OTG which interfaces android-based smartphone or PC via usb to a lot of IO ports which can read sensors or run motors and actuators.

    I really like an IO board+Smartphone solution because I can sms to it commands like "feed animals" or "count eggs". And it can reply back via sms. I can also command it via internet ("data") and perhaps get the odd picture back. There is solar charging available and cheap. And with just a single app running perhaps it can manage 24hrs recharge cycles. From all of the above I love the control-by-sms idea.

    My first point of call was the raspberry but i would prefer that there was a plugin module to the pi to do that rather than buying cells+batteries and doing calculations. E.g. https://community.element14.com/products/raspberry-pi/f/forum/53345/power-raspberry-pi-using-solar-panel

    I would also like to communicate with the device. The most reliable way I see is via SMS. And the most practical is via 5G internet (data). The PI offers sending SMS but again, it's hands-on and a lot can go wrong.

    So, I am looking for recommendations on other IO boards for smartphones (android is just fine), and/or similar solutions for the PI just to be fair to the PI, if people feel that's a better environment.

    I have asked hacker news about this here

    thanks, bw, bliako

    edit:the IOIO-OTG board needs own power supply when plugged into smartphone. Not when plugged into PC

    Edit 27/06/2024: I have read that when connecting the board to smartphone (not PC) it must provide its own power supply. It can not be powered from Android. Android then asks you if you want to charge the phone with what it found on its USB port or transfer photos etc. This fits well with the design that the board has its own solar power+battery which then charges the phone as well, and also drives any external motors (TODO: noise from motors into the board). If external power runs out (no sun for days) then, firstly, the board stops and then the phone runs out of its own battery (sending an sms to me when in the critical zone) after a while. When solar power recharges on sun appearing, the board will be able to charge the phone too. Problem I see, how to turn the phone on when power comes back and how to tell it that what is on the USB port (our IO board) should be run on the specified mode (transfer files, charge, whatever) WITHOUT user intervention. Just by its own.

XS: How to call another Perl function/method with same arguments
1 direct reply — Read more / Contribute
by etj
on Jun 09, 2024 at 11:48
    Recently in PDL I was reimplementing often-used Perl functions in XS for raw speed gainz. A problem I dealt with was effectively dispatching to another Perl method in some circumstances with the same Perl arguments, as efficiently as possible.

    I'll leave out how it took a couple of days of experimenting to figure this out; that wasn't due to the dispatching technique being wrong, but a simple logic error a bit further up the function. If there's any value to mentioning that here, it's to check your assumptions (possibly with printf) when things don't go to expectations.

    The obvious way to do this, according to https://perldoc.perl.org/perlcall, is to pass the G_NOARGS flag. However, that is (somewhat obscurely) documented as not being a good idea to use with perl_call_method. The technique I settled on, permalinked at https://github.com/PDLPorters/pdl/blob/cd5ffff1b85ff81effefc2085067e9d71801efd6/Basic/Core/Core.xs#L199-L220:

    SP -= items; PUSHMARK(SP); SPAGAIN; /* these pass this set of ar +gs on */ int retvals = perl_call_method("new", G_SCALAR); SPAGAIN; if (retvals != 1) barf("new returned no values"); RETVAL = POPs;
    For completeness, this is the full text of the function:
    SV * topdl(klass, arg1, ...) SV *klass; SV *arg1; CODE: if (items > 2 || (!SvROK(arg1) && SvTYPE(arg1) < SVt_PVAV) || (SvROK(arg1) && SvTYPE(SvRV(arg1)) == SVt_PVAV) ) { SP -= items; PUSHMARK(SP); SPAGAIN; /* these pass this set of ar +gs on */ int retvals = perl_call_method("new", G_SCALAR); SPAGAIN; if (retvals != 1) barf("new returned no values"); RETVAL = POPs; } else if (SvROK(arg1) && SvOBJECT(SvRV(arg1))) { RETVAL = arg1; } else { barf("Can not convert a %s to a %s", sv_reftype(arg1, 1), SvPV_n +olen(klass)); } SvREFCNT_inc(RETVAL); OUTPUT: RETVAL
    EDIT to explain the magic a bit more: SP is the local copy (which Perl does for efficiency when you repeatedly push arguments) of the global current "Perl stack pointer" (as I am calling it here): the address of ST(items-1) for the next function called, i.e. the top of the current stack frame. When you PUSHMARK an address, that will be ST(0) for the next Perl function that gets called. items for that next function will be the global "Perl stack pointer" minus that next function's ST(0) (plus 1). SPAGAIN copies the global "Perl stack pointer" into SP - I am carefully not saying "copies back", because of the faintly tricky stuff done here.

    Breaking the code into individual statements, with comments for each:

    SP -= items; /* make the local SP point at our ST(0) */ PUSHMARK(SP); /* make the next function's ST(0) be that */ SPAGAIN; /* reset our local SP to the global "top of current stack" */ int retvals = perl_call_method("new", G_SCALAR); /* call method so has + right stack frame */ SPAGAIN; /* set our local SP to the global "top of current stack" */ if (retvals != 1) barf("new returned no values"); /* use croak in non- +PDL code */ RETVAL = POPs; /* set the SV* to the return value */
    You may think, as I have just realised in writing this, that the first 3 lines do unnecessary work, since PUSHMARK(SP - items) would valuably replace them. You would be right! And I have just done this. However, leaving this example as it is still seems valuable (and the generated machine code is likely to be extremely similar), so I am doing so.
a nifty utility script from chatgpt
4 direct replies — Read more / Contribute
by Aldebaran
on Jun 09, 2024 at 04:18

    Hey monks,

    I'm backing up files with chatgpt and perl. I got a pretty good utility program out of it (them, whatever). It's a utility to sort pics and convert .heic to .jpg. I happen to be looking for a particular picture I took in 2021, so this is really helpful. Typical output:

    Moved and renamed: /home/fritz/Pictures/6.pics/2022-12-08-220201009.mp +4 -> /media/hogan/175F-DC61/Organized_Pics/2022/12/2022_12_08_5.mp4 Skipping: Wallpapers (not a file) Processing file: Screenshot from 2022-10-16 16-58-24.png File path: /home/fritz/Pictures/Wallpapers/Screenshot from 2022-10-16 +16-58-24.png Failed to delete original file /home/fritz/Pictures/Wallpapers/Screens +hot from 2022-10-16 16-58-24.png: No such file or directory Moved and renamed: /home/fritz/Pictures/Wallpapers/Screenshot from 202 +2-10-16 16-58-24.png -> /media/hogan/175F-DC61/Organized_Pics/2022/10 +/2022_10_16_13.png Processing complete! Number of files created: 960 fritz@laptop:~/Documents$

    Source:

    #!/usr/bin/perl use strict; use warnings; use File::Find; use File::Path qw(make_path); use File::Copy qw(move); use POSIX qw(strftime); # Define the source and target directories my $source_dir = '/home/fritz/Pictures'; my $target_dir = '/media/hogan/175F-DC61/Organized_Pics'; # Check if the source directory exists unless (-d $source_dir) { die "Source directory $source_dir does not exist.\n"; } # Create the target directory if it doesn't exist unless (-d $target_dir) { make_path($target_dir) or die "Failed to create target directory $ +target_dir: $!\n"; print "Created target directory: $target_dir\n"; } # Hash to store file type counts my %file_types; my $file_count = 0; # Subroutine to process each file sub process_file { if (-f $_) { my ($ext) = $_ =~ /\.([^.]+)$/; $ext = lc $ext if defined $ext; # Convert to lowercase if (defined $ext && $ext =~ /^(jpg|jpeg|png|gif|bmp|tiff|heic| +mp4)$/) { print "Processing file: $_\n"; my $file_path = $File::Find::name; print "File path: $file_path\n"; my $mod_time = (stat($file_path))[9]; my $date = strftime "%Y_%m_%d", localtime($mod_time); my ($year, $month, $day) = split('_', $date); my $dest_dir = "$target_dir/$year/$month"; unless (-d $dest_dir) { make_path($dest_dir) or die "Failed to create destinat +ion directory $dest_dir: $!\n"; print "Created destination directory: $dest_dir\n"; } my $count = 1; my $new_file_name; if ($ext eq 'heic') { $new_file_name = "${year}_${month}_${day}_$count.jpg"; while (-e "$dest_dir/$new_file_name") { $count++; $new_file_name = "${year}_${month}_${day}_$count.j +pg"; } my $converted_file_path = "$dest_dir/$new_file_name"; my $convert_command = "heif-convert $file_path $conver +ted_file_path"; system($convert_command) == 0 or die "Failed to conver +t $file_path: $!\n"; unlink $file_path or warn "Failed to delete original f +ile $file_path: $!\n"; print "Converted and moved: $file_path -> $converted_f +ile_path\n"; } else { $new_file_name = "${year}_${month}_${day}_$count.$ext" +; while (-e "$dest_dir/$new_file_name") { $count++; $new_file_name = "${year}_${month}_${day}_$count.$ +ext"; } move($file_path, "$dest_dir/$new_file_name") or die "F +ailed to move file $file_path: $!\n"; unlink $file_path or warn "Failed to delete original f +ile $file_path: $!\n"; print "Moved and renamed: $file_path -> $dest_dir/$new +_file_name\n"; } $file_count++; } else { print "Skipping file: $_ (not an image)\n"; } } else { print "Skipping: $_ (not a file)\n"; } } # Find and process files in the source directory print "Starting to process files in $source_dir...\n"; find(\&process_file, $source_dir); print "Processing complete! Number of files created: $file_count\n";

    What bells and whistles might we coach it to add?

    Cheers from my own private Idaho,

SO and AI
8 direct replies — Read more / Contribute
by stevieb
on May 20, 2024 at 02:53

    Stack just monetized all of the data that we, as public advocates of free information, provided.

    Is Perlmonks going to do this? If it is, I want all of my content removed immediately. I do not agree to the knowledge I've learned from those before me and I've subsequently shared being sold to anyone without due attribution.

    If Perlmonks plans on selling its user data to anyone, I outright refuse to take part, and want my data to be excluded entirely.

    -stevieb

Bizarre copy of ARRAY etc - solved
No replies — Read more | Post response
by etj
on May 13, 2024 at 17:07
    A simple XS function in PDL, firstvals_nophys, was giving "panic: attempt to copy freed scalar", but only if called on a complex-valued ndarray. The aim of this post is to appear when a despairing XS programmer googles that message, and give them another thing to check. When constructing a test to capture this, another message that appeared was "Bizarre copy of ARRAY". This is the old text of the function:
    void firstvals_nophys(x) pdl *x PPCODE: if (!(x->state & PDL_ALLOCATED)) barf("firstvals_nophys called on +non-ALLOCATED %p", x); PDL_Indx i, maxvals = PDLMIN(10, x->nvals); EXTEND(SP, maxvals); for(i=0; i<maxvals; i++) { PDL_Anyval anyval = pdl_get_offs(x, i); if (anyval.type < 0) barf("Error getting value, type=%d", anyval +.type); SV *sv = sv_newmortal(); ANYVAL_TO_SV(sv, anyval); PUSHs(sv); }
    The problem was that the ANYVAL_TO_SV macro was, only for complex-valued data, calling a Perl function to create a Math::Complex object (well, a subclass thereof because the overloads were wrong). That obviously uses the top of the stack, including writing values into it, and reading values out of it, including mortal ones that then got freed because they were done with. Therefore, the function was returning with some garbage on the stack, but the last value was correct.

    The solution was simply to do a PUTBACK after the PUSH, which moves the top of the stack above data we care about. The new text of the function with that:

    void firstvals_nophys(x) pdl *x PPCODE: if (!(x->state & PDL_ALLOCATED)) barf("firstvals_nophys called on +non-ALLOCATED %p", x); PDL_Indx i, maxvals = PDLMIN(10, x->nvals); EXTEND(SP, maxvals); for(i=0; i<maxvals; i++) { PDL_Anyval anyval = pdl_get_offs(x, i); if (anyval.type < 0) barf("Error getting value, type=%d", anyval +.type); SV *sv = sv_newmortal(); ANYVAL_TO_SV(sv, anyval); PUSHs(sv); PUTBACK; }
    The commit that fixed this is at https://github.com/PDLPorters/pdl/commit/68389413537c6ea7ed85f121a580b3b008ba82a6. The docs on how to call a Perl function from C are at https://perldoc.perl.org/perlcall, with a full explanation of PUSHMARK, PUSH*, PUTBACK, and (after the call) SPAGAIN, and maybe POP* and maybe then PUTBACK.
Debugger issue solved (two years ago)
2 direct replies — Read more / Contribute
by talexb
on May 06, 2024 at 20:57

    I upgraded a machine to the latest Ubuntu and got Perl 5.34.0, which included a problem with the debugger:

    DB<4> v + Undefined subr +outine &DB::cmd_l called at /usr/share/perl/5.34/perl5db.pl line 6034 +. at /usr/share/perl/5.34/perl5db.pl line 6034. + DB::cm +d_v("v", "", 56) called at /usr/share/perl/5.34/perl5db.pl line 4798 DB::cmd_wrapper("v", "", 56) called at /usr/share/perl/5.34/pe +rl5db.pl line 4311 DB::Ob +j::_handle_cmd_wrapper_commands(DB::Obj=HASH(0x55e85838c150)) called +at /usr/share/perl/5.34/perl5db.pl line 32 00 + DB::DB + called at report_warnings.pl line 56 Debugged program terminated. Use q to quit or R to restart, + use o inhibit_ +exit to avoid stopping after program termination, h q, h R or h o to get additional info. DB<4>
    My friend Google brought me to this page that had links to the necessary patch to perl5db.pl on github, bringing the script from version 1.60 to 1.60_01.

    And, of course, the patch worked just fine. What a great community. Thanks for the patch!

    PS: Ugh, sorry -- the problem was that the v command that I use a lot (Where am I? Oh, there I am!) crashed the debugger. The patch solves that problem.

    Alex / talexb / Toronto

    Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

The Virtue of Laziness
4 direct replies — Read more / Contribute
by jo37
on May 04, 2024 at 17:57

    Dear Monks and Nuns,

    lately I came across an issue with dereferencing array refs. It looks like there is some hidden "lazy deref" when an array ref is used in a foreach loop, compared to the usage as a sub argument. Consider these two subs that do nothing but die. The main difference is the array dereference as an argument to foreach or map.

    use experimental 'signatures'; sub map_die ($ar) { eval {map {die} @$ar}; } sub for_die ($ar) { eval { for (@$ar) { die; } } }
    Benchmarking is amazing:
    use Benchmark 'cmpthese'; my @arr = ((0) x 1e6); cmpthese(0, { map => sub {map_die(\@arr)}, for => sub {for_die(\@arr)}, }); __DATA__ Rate map for map 1257/s -- -100% for 1664823/s 132352% --

    Then I remembered the "lazy generators" from List::Gen and gave it a try. There is some progress, but it cannot come up to foreach.

    use List::Gen 'array'; sub gen_die ($ar) { eval { &array($ar)->map(sub {die}); } } cmpthese(0, { map => sub {map_die(\@arr)}, for => sub {for_die(\@arr)}, gen => sub {gen_die(\@arr)}, }); __DATA__ Rate map gen for map 1316/s -- -93% -100% gen 18831/s 1330% -- -99% for 1662271/s 126174% 8727% --

    Wouldn't it be nice to have some kind of "explicit lazy dereferencing" in Perl?

    Update May 10, 2024: Added the signatures feature as suggested by Danny in Re: The Virtue of Laziness.

    Greetings,
    🐻

    $gryYup$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p$
Imager support for PNG, JPEG and GIF on macOS Sonoma
1 direct reply — Read more / Contribute
by Anonymous Monk
on Apr 20, 2024 at 22:51
    Installing our beloved Imager on macOS Sonoma with support for PNG, JPEG and GIF involves some pain. Here are the results of my struggle so others won't have to. Do this after installing Imager (requires Homebrew):
    brew install pkg-config
    
    brew install libpng
    pkg-config --cflags libpng
    cpan Imager::File::PNG
    
    brew install jpeg
    pkg-config --cflags libjpeg
    cpan Imager::File::JPEG
    
    brew install giflib
    cpan Imager::File::GIF
    
    GIF: Test code failed: Can't link/include 'gif_lib.h', 'stdio.h', 'errno.h', 'string.h', 'gif'...
    ! Configure failed for Imager-File-GIF-0.98. See /Users/you/.cpanm/work/1713652269.80239/build.log for details.
    
    cd /Users/you/.cpanm/work/1713652269.80239/Imager-File-GIF-0.98
    
    perl Makefile.PL -v --incpath=/opt/homebrew/include --libpath=/opt/homebrew/var/homebrew/linked/giflib/lib
    
    make
    make test
    make install
    
    For some reason the following didn't work with the cpan client:
    o conf makepl_arg "LIBS=-L/opt/homebrew/var/homebrew/linked/giflib/lib INC=-I/opt/homebrew/include"
    
    PS - pkg-config can't find giflib so the paths were found like this:
    sudo /usr/libexec/locate.updatedb
    
    locate giflib
    /opt/homebrew/var/homebrew/linked/giflib
    
    locate gif_lib.h        
    /opt/homebrew/Cellar/giflib/5.2.1/include/gif_lib.h
    /opt/homebrew/include/gif_lib.h
    
CPAN autobundle fail
2 direct replies — Read more / Contribute
by Anonymous Monk
on Apr 10, 2024 at 13:50
    I was trying to autobundle an old perl setup but cpan was just sitting there failing to contact mirrors (cpanm user so that mirror list was probably ancient). This inspired me to visit https://www.cpan.org/SITES.html where it says www.cpan.org don't do mirrors anymore. Created the autobundle like so:
    cpan -M https://www.cpan.org -a
UK tax system uses Perl
1 direct reply — Read more / Contribute
by Bod
on Apr 08, 2024 at 10:47

    I've just found out the HMRC (the UK's taxation department) uses Perl for at least some of its operations...

    The system is troubled today and I received this error revealing the language

    Ref: /home/ewf/MODULES/Common/PaymentApiService.pm Error Code 401 at line 30

    Update: corrected typo in title

[OT] The Long List is Long resurrected
2 direct replies — Read more / Contribute
by marioroy
on Apr 06, 2024 at 01:23

    Recently, I tried NVIDIA's nvc++ compiler and noticed a compile time regression (> 40 seconds). That's a long time. I reached out to NVIDIA. Though, no resolution from their side. Some time later, I saw another regression upgrading GCC from version 13 to 14. It seems that std::mutex regressed.

    That spawned a chain-reaction. I logged-appended the March and April 2024 events to my summary page.

    A spinlock mutex class resolved both issues, further improving performance. I reached out to Greg, author of the phmap C++ library. The LLiL challenge is interesting. He shared tips plus the string_cnt struct, accommodating fixed-length and variable-length keys.

    Clang performance regression due to GCC 14
    phmap: Ability to reset the inner sub map
    string_cnt struct commit by Greg

    eyepopslikeamosquito's llil3vec.cpp inspiration (scroll down the page), for making the vector version llil4vec fast (though gobbles memory for unlimited length strings), is the reason efforts to making the map variants catch up, while keeping memory consumption low. There are several map variants. The llil4hmap, llil4emh, and llil4umap demonstrations compute the key hash_value once only, and stores it with with the key.

    llil4map sub maps managed by the C++ library, using phmap::parallel_flat_hash_map
    llil4map2 memory efficient version, vector of pointers to the map key-value pairs
    llil4hmap sub maps managed by the application, using phmap::flat_hash_map
    llil4emh sub maps managed by the application, using emhash7::HashMap
    llil4umap sub maps managed by the application, using std::unordered_map

    Greg Popovitch added a clever new type string_cnt, supporting fixed-length and long strings without recompiling. See enhanced llil4map found in his examples folder, also memory efficient. Note: I beautified the include directives in my demonstrations, matching your version for consistency.

    I glanced through the long thread by eyepopslikeamosquito. At the time, we were thrilled completing in less than 10 seconds. More so below 6 seconds. Fast forward to early 2024, the llil4map demonstration completes in 0.3 seconds, due to linear scaling capabilities (all levels parallel). The llil4vec example may run faster, but stop improving beyond more CPU cores.

    We reached the sub-second territory, processing three input files.

    Limit 12 CPU Cores

    $ NUM_THREADS=12 ./llil4map big{1,2,3}.txt | cksum llil4map (fixed string length=12) start use OpenMP use boost sort get properties 0.311 secs map to vector 0.055 secs vector stable sort 0.095 secs write stdout 0.036 secs total time 0.500 secs count lines 10545600 count unique 10367603 2956888413 93308427 $ NUM_THREADS=12 ./llil4vec big{1,2,3}.txt | cksum llil4vec (fixed string length=12) start use OpenMP use boost sort get properties 0.170 secs sort properties 0.061 secs vector reduce 0.017 secs vector stable sort 0.065 secs write stdout 0.047 secs total time 0.362 secs count lines 10545600 count unique 10367603 2956888413 93308427

    No Limit

    $ ./llil4map big{1,2,3}.txt | cksum llil4map (fixed string length=12) start use OpenMP use boost sort get properties 0.101 secs map to vector 0.052 secs vector stable sort 0.115 secs write stdout 0.029 secs total time 0.298 secs count lines 10545600 count unique 10367603 2956888413 93308427 $ ./llil4vec big{1,2,3}.txt | cksum llil4vec (fixed string length=12) start use OpenMP use boost sort get properties 0.203 secs sort properties 0.088 secs vector reduce 0.024 secs vector stable sort 0.103 secs write stdout 0.029 secs total time 0.449 secs count lines 10545600 count unique 10367603 2956888413 93308427

    Many thanks eyepopslikeamosquito for being there. I promise no more messages for a while. You inspired C++ in me. Next is Taichi Lang.

    Greg Popovitch shared a tip for releasing memory immediately, during "map to vector". Thank you. That was helpful also, for llil4umap.

    Blessings and grace,


Add your Meditation
Title:
Meditation:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Domain Nodelet?
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this?Last hourOther CB clients
    Other Users?
    Others lurking in the Monastery: (1)
    As of 2024-07-23 02:23 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      No recent polls found

      Notices?
      erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.