more useful options PerlMonks

### Meditations

 Need Help??

If you've discovered something amazing about Perl that you just need to share with everyone, this is the right place.

This section is also used for non-question discussions about Perl, and for any discussions that are not specifically programming related. For example, if you want to share or discuss opinions on hacker culture, the job market, or Perl 6 development, this is the place. (Note, however, that discussions about the PerlMonks web site belong in PerlMonks Discussion.)

Meditations is sometimes used as a sounding-board — a place to post initial drafts of perl tutorials, code modules, book reviews, articles, quizzes, etc. — so that the author can benefit from the collective insight of the monks before publishing the finished item to its proper place (be it Tutorials, Cool Uses for Perl, Reviews, or whatever). If you do this, it is generally considered appropriate to prefix your node title with "RFC:" (for "request for comments").

User Meditations
 The intersection of M hyperplanes (Ndim) 3 direct replies — Read more / Contribute by bliako on Jul 18, 2024 at 09:41 Here is my understanding of what the theory behind what I am trying to do is, correct me where I am wrong, also please check my questions at the end: The intersection of N hyperplanes (Ndim) is also known as the solution to "the system of N linear equations(=planes) with N variables". However, I am interested in the case when I have less equations. The result is not a single point in Ndim but a lower-dimensions plane. For example in 3D. The intersection of 3 planes (M=3, N=3), assuming any pair is not parallel, is a single 3D point and the intersection of 2 planes (M=2, N=3) is a line. In Ndim, the solution would be an Mdim hyperplane "living" in Ndim. Below, I call this a "hyperline". An example of the above problem is in this SO question. The result is the equation of the intersection line in parametric form. Arbitrary values of the parameter t yields a point on this line. Ideally this is what I want to achieve: finding points on the intersection of the planes. Solving the system of N linear equations with N variables (Ndim) can be done with, for example, Math::Matrix. Given N planes in the form a1x+b1y+c1z+...n1=0 etc., a matrix A is formed with each row being [a1, b1, c1, ..., n1] etc. then: Math::Matrix->new([[a1,b1,c1,...,n1],...])->solve->print; I have experimented with solve() for the case of M linear equations (Ndim) where M = N-1: ```use Math::Matrix; Math::Matrix->new( [1, 2, 1, -1], # ax+by+cz+n=0 [2, 3, -2, 2] )->solve->print; # says # -7.00000 7.00000 # 4.00000 -4.00000 [download]``` I interpret the result as: the 1st row refers to the 1st dim (aka x) and we have ```x -7*t + 7 = 0 => x = 7*t - 7 [download]``` The 2nd row yields ```y + 4*t -4 = 0 => y = -4*t + 4 [download]``` and the missing 3rd dim is the parameter t itself: z = t. And I can get a point on that line by setting t to an arbitrary value. I can verify that this point is on each plane by substituting the point's coordinates into each plane equation. I have also experimented with transforming the planes matrix to the Reduced Row Echelon Form. Which makes it convenient to work out the "hyperline" equation. This functionality is offered by Math::MatrixLUP : ```use Math::MatrixLUP; my @planes3D = ( [1, 2, 1, -1], # ax+by+cz+n=0 [2, 3, -2, 2] ); print Math::MatrixLUP->new(\@planes3D)->rref # says # [1, 0, -7, 7], # [0, 1, 4, -4] [download]``` Which I interpret as: 1st row has 1 on 1st col, that's the 1st dim (aka x) : ```x -7 * t + 7 = 0 => x = 7*t-7 [download]``` etc. So that works well. Here is a demo I whipped up: Read more... (9 kB) I have some questions: Two planes are parallel when their normal vectors (the coefficients of the dimensions: a, b, c, ... (but not the constant n) are multiples. E.g. nv1 = k * nv2. And this translates to checking if all the ratios of the coefficients : a1/a2 = b1/b2 = c1/c2 = ... are the same. My question is: what happens if any coefficient is zero (or actually both coefficients (e.g. a1 and a2) are zero? How can I calculate the intersection when M < (N-1)? I.e. above I am always checking the intersection of M planes in Ndim where M = N-1. But what if there are even less planes? e.g. ```my @planes5D = ( [1, 2, 1, -2, 3, -1], # ax+by+cz+n=0 [2, 3, -2, 4, -4, 2], [3, -1, -2, -5, 6, 2], # only 3 planes (it was successful with 4) ) [download]``` In short, does the parametric equation of the intersecting "hyperline" contain 2 parameters now? When I test test_in_beast_space which produces random planes in 666 dimensions, it fails to detect that a point on the intersection "hyperline" lies on all the planes. To do that I substitute the point in each plane equation and expect to have result as zero. However, it's not an exact zero. That's why I am doing a range test to check if it's close to zero. Well the closeness \$SMALLNUMBER for 5 dimensions can be as small as 1E-09. But for these many dimensions it can be as low as 1E-02. Is the accuracy lost in summing 666 multiplications that much really? I guess Math::Matrix::solve() is safe to be used for MxN matrices (e.g. M equations=planes in N unknowns (dimensions) where M 2 || (!SvROK(arg1) && SvTYPE(arg1) < SVt_PVAV) || (SvROK(arg1) && SvTYPE(SvRV(arg1)) == SVt_PVAV) ) { SP -= items; PUSHMARK(SP); SPAGAIN; /* these pass this set of ar +gs on */ int retvals = perl_call_method("new", G_SCALAR); SPAGAIN; if (retvals != 1) barf("new returned no values"); RETVAL = POPs; } else if (SvROK(arg1) && SvOBJECT(SvRV(arg1))) { RETVAL = arg1; } else { barf("Can not convert a %s to a %s", sv_reftype(arg1, 1), SvPV_n +olen(klass)); } SvREFCNT_inc(RETVAL); OUTPUT: RETVAL [download]``` EDIT to explain the magic a bit more: SP is the local copy (which Perl does for efficiency when you repeatedly push arguments) of the global current "Perl stack pointer" (as I am calling it here): the address of ST(items-1) for the next function called, i.e. the top of the current stack frame. When you PUSHMARK an address, that will be ST(0) for the next Perl function that gets called. items for that next function will be the global "Perl stack pointer" minus that next function's ST(0) (plus 1). SPAGAIN copies the global "Perl stack pointer" into SP - I am carefully not saying "copies back", because of the faintly tricky stuff done here. Breaking the code into individual statements, with comments for each: ```SP -= items; /* make the local SP point at our ST(0) */ PUSHMARK(SP); /* make the next function's ST(0) be that */ SPAGAIN; /* reset our local SP to the global "top of current stack" */ int retvals = perl_call_method("new", G_SCALAR); /* call method so has + right stack frame */ SPAGAIN; /* set our local SP to the global "top of current stack" */ if (retvals != 1) barf("new returned no values"); /* use croak in non- +PDL code */ RETVAL = POPs; /* set the SV* to the return value */ [download]``` You may think, as I have just realised in writing this, that the first 3 lines do unnecessary work, since PUSHMARK(SP - items) would valuably replace them. You would be right! And I have just done this. However, leaving this example as it is still seems valuable (and the generated machine code is likely to be extremely similar), so I am doing so. a nifty utility script from chatgpt 4 direct replies — Read more / Contribute by Aldebaran on Jun 09, 2024 at 04:18 Hey monks, I'm backing up files with chatgpt and perl. I got a pretty good utility program out of it (them, whatever). It's a utility to sort pics and convert .heic to .jpg. I happen to be looking for a particular picture I took in 2021, so this is really helpful. Typical output: ``` Moved and renamed: /home/fritz/Pictures/6.pics/2022-12-08-220201009.mp +4 -> /media/hogan/175F-DC61/Organized_Pics/2022/12/2022_12_08_5.mp4 Skipping: Wallpapers (not a file) Processing file: Screenshot from 2022-10-16 16-58-24.png File path: /home/fritz/Pictures/Wallpapers/Screenshot from 2022-10-16 +16-58-24.png Failed to delete original file /home/fritz/Pictures/Wallpapers/Screens +hot from 2022-10-16 16-58-24.png: No such file or directory Moved and renamed: /home/fritz/Pictures/Wallpapers/Screenshot from 202 +2-10-16 16-58-24.png -> /media/hogan/175F-DC61/Organized_Pics/2022/10 +/2022_10_16_13.png Processing complete! Number of files created: 960 fritz@laptop:~/Documents\$ [download]``` Source: ```#!/usr/bin/perl use strict; use warnings; use File::Find; use File::Path qw(make_path); use File::Copy qw(move); use POSIX qw(strftime); # Define the source and target directories my \$source_dir = '/home/fritz/Pictures'; my \$target_dir = '/media/hogan/175F-DC61/Organized_Pics'; # Check if the source directory exists unless (-d \$source_dir) { die "Source directory \$source_dir does not exist.\n"; } # Create the target directory if it doesn't exist unless (-d \$target_dir) { make_path(\$target_dir) or die "Failed to create target directory \$ +target_dir: \$!\n"; print "Created target directory: \$target_dir\n"; } # Hash to store file type counts my %file_types; my \$file_count = 0; # Subroutine to process each file sub process_file { if (-f \$_) { my (\$ext) = \$_ =~ /\.([^.]+)\$/; \$ext = lc \$ext if defined \$ext; # Convert to lowercase if (defined \$ext && \$ext =~ /^(jpg|jpeg|png|gif|bmp|tiff|heic| +mp4)\$/) { print "Processing file: \$_\n"; my \$file_path = \$File::Find::name; print "File path: \$file_path\n"; my \$mod_time = (stat(\$file_path))[9]; my \$date = strftime "%Y_%m_%d", localtime(\$mod_time); my (\$year, \$month, \$day) = split('_', \$date); my \$dest_dir = "\$target_dir/\$year/\$month"; unless (-d \$dest_dir) { make_path(\$dest_dir) or die "Failed to create destinat +ion directory \$dest_dir: \$!\n"; print "Created destination directory: \$dest_dir\n"; } my \$count = 1; my \$new_file_name; if (\$ext eq 'heic') { \$new_file_name = "\${year}_\${month}_\${day}_\$count.jpg"; while (-e "\$dest_dir/\$new_file_name") { \$count++; \$new_file_name = "\${year}_\${month}_\${day}_\$count.j +pg"; } my \$converted_file_path = "\$dest_dir/\$new_file_name"; my \$convert_command = "heif-convert \$file_path \$conver +ted_file_path"; system(\$convert_command) == 0 or die "Failed to conver +t \$file_path: \$!\n"; unlink \$file_path or warn "Failed to delete original f +ile \$file_path: \$!\n"; print "Converted and moved: \$file_path -> \$converted_f +ile_path\n"; } else { \$new_file_name = "\${year}_\${month}_\${day}_\$count.\$ext" +; while (-e "\$dest_dir/\$new_file_name") { \$count++; \$new_file_name = "\${year}_\${month}_\${day}_\$count.\$ +ext"; } move(\$file_path, "\$dest_dir/\$new_file_name") or die "F +ailed to move file \$file_path: \$!\n"; unlink \$file_path or warn "Failed to delete original f +ile \$file_path: \$!\n"; print "Moved and renamed: \$file_path -> \$dest_dir/\$new +_file_name\n"; } \$file_count++; } else { print "Skipping file: \$_ (not an image)\n"; } } else { print "Skipping: \$_ (not a file)\n"; } } # Find and process files in the source directory print "Starting to process files in \$source_dir...\n"; find(\&process_file, \$source_dir); print "Processing complete! Number of files created: \$file_count\n"; [download]``` What bells and whistles might we coach it to add? Cheers from my own private Idaho, SO and AI 8 direct replies — Read more / Contribute by stevieb on May 20, 2024 at 02:53 Stack just monetized all of the data that we, as public advocates of free information, provided. Is Perlmonks going to do this? If it is, I want all of my content removed immediately. I do not agree to the knowledge I've learned from those before me and I've subsequently shared being sold to anyone without due attribution. If Perlmonks plans on selling its user data to anyone, I outright refuse to take part, and want my data to be excluded entirely. -stevieb Bizarre copy of ARRAY etc - solved No replies — Read more | Post response by etj on May 13, 2024 at 17:07 A simple XS function in PDL, firstvals_nophys, was giving "panic: attempt to copy freed scalar", but only if called on a complex-valued ndarray. The aim of this post is to appear when a despairing XS programmer googles that message, and give them another thing to check. When constructing a test to capture this, another message that appeared was "Bizarre copy of ARRAY". This is the old text of the function: ```void firstvals_nophys(x) pdl *x PPCODE: if (!(x->state & PDL_ALLOCATED)) barf("firstvals_nophys called on +non-ALLOCATED %p", x); PDL_Indx i, maxvals = PDLMIN(10, x->nvals); EXTEND(SP, maxvals); for(i=0; istate & PDL_ALLOCATED)) barf("firstvals_nophys called on +non-ALLOCATED %p", x); PDL_Indx i, maxvals = PDLMIN(10, x->nvals); EXTEND(SP, maxvals); for(i=0; i v + Undefined subr +outine &DB::cmd_l called at /usr/share/perl/5.34/perl5db.pl line 6034 +. at /usr/share/perl/5.34/perl5db.pl line 6034. + DB::cm +d_v("v", "", 56) called at /usr/share/perl/5.34/perl5db.pl line 4798 DB::cmd_wrapper("v", "", 56) called at /usr/share/perl/5.34/pe +rl5db.pl line 4311 DB::Ob +j::_handle_cmd_wrapper_commands(DB::Obj=HASH(0x55e85838c150)) called +at /usr/share/perl/5.34/perl5db.pl line 32 00 + DB::DB + called at report_warnings.pl line 56 Debugged program terminated. Use q to quit or R to restart, + use o inhibit_ +exit to avoid stopping after program termination, h q, h R or h o to get additional info. DB<4> [download]``` My friend Google brought me to this page that had links to the necessary patch to perl5db.pl on github, bringing the script from version 1.60 to 1.60_01. And, of course, the patch worked just fine. What a great community. Thanks for the patch! PS: Ugh, sorry -- the problem was that the v command that I use a lot (Where am I? Oh, there I am!) crashed the debugger. The patch solves that problem. Alex / talexb / Toronto Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013. The Virtue of Laziness 4 direct replies — Read more / Contribute by jo37 on May 04, 2024 at 17:57 Dear Monks and Nuns, lately I came across an issue with dereferencing array refs. It looks like there is some hidden "lazy deref" when an array ref is used in a foreach loop, compared to the usage as a sub argument. Consider these two subs that do nothing but die. The main difference is the array dereference as an argument to foreach or map. ```use experimental 'signatures'; sub map_die (\$ar) { eval {map {die} @\$ar}; } sub for_die (\$ar) { eval { for (@\$ar) { die; } } } [download]``` Benchmarking is amazing: ```use Benchmark 'cmpthese'; my @arr = ((0) x 1e6); cmpthese(0, { map => sub {map_die(\@arr)}, for => sub {for_die(\@arr)}, }); __DATA__ Rate map for map 1257/s -- -100% for 1664823/s 132352% -- [download]``` Then I remembered the "lazy generators" from List::Gen and gave it a try. There is some progress, but it cannot come up to foreach. ```use List::Gen 'array'; sub gen_die (\$ar) { eval { &array(\$ar)->map(sub {die}); } } cmpthese(0, { map => sub {map_die(\@arr)}, for => sub {for_die(\@arr)}, gen => sub {gen_die(\@arr)}, }); __DATA__ Rate map gen for map 1316/s -- -93% -100% gen 18831/s 1330% -- -99% for 1662271/s 126174% 8727% -- [download]``` Wouldn't it be nice to have some kind of "explicit lazy dereferencing" in Perl? Update May 10, 2024: Added the signatures feature as suggested by Danny in Re: The Virtue of Laziness. Greetings,🐻 \$gryYup\$d0ylprbpriprrYpkJl2xyl~rzg??P~5lp2hyl0p\$ Imager support for PNG, JPEG and GIF on macOS Sonoma 1 direct reply — Read more / Contribute by Anonymous Monk on Apr 20, 2024 at 22:51 Installing our beloved Imager on macOS Sonoma with support for PNG, JPEG and GIF involves some pain. Here are the results of my struggle so others won't have to. Do this after installing Imager (requires Homebrew): ```brew install pkg-config brew install libpng pkg-config --cflags libpng cpan Imager::File::PNG brew install jpeg pkg-config --cflags libjpeg cpan Imager::File::JPEG brew install giflib cpan Imager::File::GIF GIF: Test code failed: Can't link/include 'gif_lib.h', 'stdio.h', 'errno.h', 'string.h', 'gif'... ! Configure failed for Imager-File-GIF-0.98. See /Users/you/.cpanm/work/1713652269.80239/build.log for details. cd /Users/you/.cpanm/work/1713652269.80239/Imager-File-GIF-0.98 perl Makefile.PL -v --incpath=/opt/homebrew/include --libpath=/opt/homebrew/var/homebrew/linked/giflib/lib make make test make install ``` For some reason the following didn't work with the cpan client: ```o conf makepl_arg "LIBS=-L/opt/homebrew/var/homebrew/linked/giflib/lib INC=-I/opt/homebrew/include" ``` PS - pkg-config can't find giflib so the paths were found like this: ```sudo /usr/libexec/locate.updatedb locate giflib /opt/homebrew/var/homebrew/linked/giflib locate gif_lib.h /opt/homebrew/Cellar/giflib/5.2.1/include/gif_lib.h /opt/homebrew/include/gif_lib.h ``` CPAN autobundle fail 2 direct replies — Read more / Contribute by Anonymous Monk on Apr 10, 2024 at 13:50 I was trying to autobundle an old perl setup but cpan was just sitting there failing to contact mirrors (cpanm user so that mirror list was probably ancient). This inspired me to visit https://www.cpan.org/SITES.html where it says www.cpan.org don't do mirrors anymore. Created the autobundle like so: ```cpan -M https://www.cpan.org -a [download]``` UK tax system uses Perl 1 direct reply — Read more / Contribute by Bod on Apr 08, 2024 at 10:47 I've just found out the HMRC (the UK's taxation department) uses Perl for at least some of its operations... The system is troubled today and I received this error revealing the language ```Ref: /home/ewf/MODULES/Common/PaymentApiService.pm Error Code 401 at line 30 [download]``` Update: corrected typo in title [OT] The Long List is Long resurrected 2 direct replies — Read more / Contribute by marioroy on Apr 06, 2024 at 01:23 Recently, I tried NVIDIA's nvc++ compiler and noticed a compile time regression (> 40 seconds). That's a long time. I reached out to NVIDIA. Though, no resolution from their side. Some time later, I saw another regression upgrading GCC from version 13 to 14. It seems that std::mutex regressed. That spawned a chain-reaction. I logged-appended the March and April 2024 events to my summary page. A spinlock mutex class resolved both issues, further improving performance. I reached out to Greg, author of the phmap C++ library. The LLiL challenge is interesting. He shared tips plus the string_cnt struct, accommodating fixed-length and variable-length keys. eyepopslikeamosquito's llil3vec.cpp inspiration (scroll down the page), for making the vector version llil4vec fast (though gobbles memory for unlimited length strings), is the reason efforts to making the map variants catch up, while keeping memory consumption low. There are several map variants. The llil4hmap, llil4emh, and llil4umap demonstrations compute the key hash_value once only, and stores it with with the key. llil4map sub maps managed by the C++ library, using phmap::parallel_flat_hash_map llil4map2 memory efficient version, vector of pointers to the map key-value pairs llil4hmap sub maps managed by the application, using phmap::flat_hash_map llil4emh sub maps managed by the application, using emhash7::HashMap llil4umap sub maps managed by the application, using std::unordered_map Greg Popovitch added a clever new type string_cnt, supporting fixed-length and long strings without recompiling. See enhanced llil4map found in his examples folder, also memory efficient. Note: I beautified the include directives in my demonstrations, matching your version for consistency. I glanced through the long thread by eyepopslikeamosquito. At the time, we were thrilled completing in less than 10 seconds. More so below 6 seconds. Fast forward to early 2024, the llil4map demonstration completes in 0.3 seconds, due to linear scaling capabilities (all levels parallel). The llil4vec example may run faster, but stop improving beyond more CPU cores. We reached the sub-second territory, processing three input files. Limit 12 CPU Cores ```\$ NUM_THREADS=12 ./llil4map big{1,2,3}.txt | cksum llil4map (fixed string length=12) start use OpenMP use boost sort get properties 0.311 secs map to vector 0.055 secs vector stable sort 0.095 secs write stdout 0.036 secs total time 0.500 secs count lines 10545600 count unique 10367603 2956888413 93308427 \$ NUM_THREADS=12 ./llil4vec big{1,2,3}.txt | cksum llil4vec (fixed string length=12) start use OpenMP use boost sort get properties 0.170 secs sort properties 0.061 secs vector reduce 0.017 secs vector stable sort 0.065 secs write stdout 0.047 secs total time 0.362 secs count lines 10545600 count unique 10367603 2956888413 93308427 [download]``` No Limit ```\$ ./llil4map big{1,2,3}.txt | cksum llil4map (fixed string length=12) start use OpenMP use boost sort get properties 0.101 secs map to vector 0.052 secs vector stable sort 0.115 secs write stdout 0.029 secs total time 0.298 secs count lines 10545600 count unique 10367603 2956888413 93308427 \$ ./llil4vec big{1,2,3}.txt | cksum llil4vec (fixed string length=12) start use OpenMP use boost sort get properties 0.203 secs sort properties 0.088 secs vector reduce 0.024 secs vector stable sort 0.103 secs write stdout 0.029 secs total time 0.449 secs count lines 10545600 count unique 10367603 2956888413 93308427 [download]``` Many thanks eyepopslikeamosquito for being there. I promise no more messages for a while. You inspired C++ in me. Next is Taichi Lang. Greg Popovitch shared a tip for releasing memory immediately, during "map to vector". Thank you. That was helpful also, for llil4umap. Blessings and grace,

text here (a paragraph)

and:  code here to format your post, it's "PerlMonks-approved HTML":

• Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
• Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
• Read Where should I post X? if you're not absolutely sure you're posting in the right place.
• Please read these before you post! —
• Posts may use any of the Perl Monks Approved HTML tags:
a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
• You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
 For: Use: & & < < > > [ [ ] ]
• Link using PerlMonks shortcuts! What shortcuts can I use for linking?
• See Writeup Formatting Tips and other pages linked from there for more info.

Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others lurking in the Monastery: (1)
As of 2024-07-23 02:23 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?

No recent polls found

Notices?
 • erzuuli ‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.