Interesting. I guess they can't blame me for this, as I haven't released my Thai module yet! (However, I have no idea why or how such an error could occur simply with the use of \p{...} characters in one's regex--how can a unicode property definition be illegal?)

For what it's worth, I discovered that my nomenclature for the \p{...} characters was illegal--but they never produced an error message like that...they just didn't work, as if they had not been defined or imported. I learned that there are only two possibilities with names of unicode properties:

\p{InProperty}
\p{IsProperty}
[download]

Using something like '\p{Property}' is illegal--it must be prefixed by either "In" or by "Is" or Perl will not accept it or recognize it.

But the error shown here in the OP doesn't appear to have much to do with this, leaving me to wonder what it is for.

Its because perl regexp itself is coded in C. In this case the code for /p was stripping 'utf8::' from the string, but not adjusting the position/length of the buffer properly to account for that stripping along all code paths, allowing you to craft a buffer overflow exploit.

!!! Important Update !!!

It appears all three of the new versions were broken: I don't have details. Each has been superseded by a newer version. I've collated the information in the following table.

5.38.1 installed normally for me on 10+ installations and works fine with the 100+ modules i installed.

From what i can tell, the main change in the new release is Module::CoreList, which should be easily fixable by installing the new version on the existing 5.38.1 installation.

But of course, since the new Perl release also comes with a new B::Op_private, ye olde cpan upgrade will fail with an error in the middle of the process :-(

To be clear, i'm not complaining that perlporters had to make an emergency bugfix release; in fact i'm glad that everyone reacted quickly and efficiently. I'm just frustrated that all the work i did monday was now made worse-than-useless, and i have to run through that whole process again. If this problem had happened on a normal feature relase, i wouldn't have started the upgrade process for a couple of weeks. But since it was an important patch for some security problems, i started right away...

Oh well, back to building pyramids.

PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP

G'day cavac,

I posted "Updated releases!" as soon as I became aware of the newer versions. I did actually think of you, as you'd written:

"Guess i'll be spending most of today reinstalling Perl+Modules on my 15+ systems i administrate..."

Hoping 🏗️◭◮ is not too onerous.

— Ken

Having been burnt before, I had the luxury of being able to wait a few days (unlike cavac).

With no new PM questions posted today, I decided to build perl v5.38.2 from source, using the same steps as last time (but installed into $HOME/my/p5382 this time, i.e. $ type perl - perl is hashed $HOME/my/p5382/bin/perl) and it seems to be working nicely so far, at least kcott's and Tux's superb Unicode utilities are both working flawlessly. :)

Of course, having just completed that chore, I'm half expecting an urgent new perl v5.38.3 to be rushed out tomorrow (hope saying that doesn't jinx cavac's pyramid building :).

See Also

• Perl Download (perl.org) - getting started quickly (notes the latest stable version)
• CPAN Perl source code (cpan.org) - how to build/install perl from source (notes the latest stable version)

Extra Modules Installed Later (Update)

Got errors using kcott's perlu alias (described at Re^2: Another Unicode/emoji question) ... to remedy, installed IPC::System::Simple:

$ alias perlu='perl -Mstrict -Mwarnings -Mautodie=:all -Mutf8 -C -E'
$ type perlu
perlu is aliased to `perl -Mstrict -Mwarnings -Mautodie=:all -Mutf8 -C
+ -E'
$ perlu 'say chr 0x1f436; say chr 128054;'
IPC::System::Simple required for Fatalised/autodying system() at -e li
+ne 0.
        main::BEGIN() called at -e line 0
        eval {...} called at -e line 0
BEGIN failed--compilation aborted.
[download]

As noted at SO question, it seems strange that a core functionality requires a non-core module, and quietly too (doesn't complain at installation). See also: perl-autodie missing dependency for IPC::System::Simple (RedHat bugzilla).

This fixed my perlu issue:

$ type perl
perl is hashed ($HOME/my/p5382/bin/perl)
$ cd $HOME/my/p5382
$ cpanm --from https://www.cpan.org/ --verify IPC::System::Simple 2>&1
+ | tee IPC-System-Simple.tmp
--> Working on IPC::System::Simple
Fetching https://www.cpan.org/authors/id/J/JK/JKEENAN/IPC-System-Simpl
+e-1.30.tar.gz ... OK
Fetching https://www.cpan.org/authors/id/J/JK/JKEENAN/CHECKSUMS ... OK
Configuring IPC-System-Simple-1.30 ... OK
Building and testing IPC-System-Simple-1.30 ... OK
Successfully installed IPC-System-Simple-1.30
1 distribution installed
[download]

$ perlu 'say chr(0x1f436); say chr(128054);'
🐶
🐶
$ perlu 'say "\x{1f436}"'
🐶
$ perlu 'say "\N{DOG FACE}"'
🐶
$ echo -e '\U1f436'
🐶

For more detail on perl character escapes see: perlrebackslash (perldoc)

Later, installed Data::Dump and List::MoreUtils and Module::Starter:

$ cd $HOME/my/p5382
$ cpanm --from https://www.cpan.org --verify Data::Dump 2>&1 | tee Dat
+a-Dump.tmp
$ cpanm --from https://www.cpan.org --verify List::MoreUtils 2>&1 | te
+e List-MoreUtils.tmp
$ cpanm --from https://www.cpan.org --verify Module::Starter 2>&1 | te
+e Module-Starter.tmp
[download]

TODO: Install more CPAN modules (e.g. List::AllUtils and stuff by PEVANS e.g. List::UtilsBy, List::Keywords, ...). Further analysis of perlrun and utf8.

👁️🍾👍🦟

G'day Tux,

++ Thanks for this notification and the CVE information.

Is there any official (or at least detailed) write-ups of these vulnerabilities? I typically just look in "Perl: Security Vulnerabilities, CVEs" but they weren't listed there. I also looked in quite a few other places; many just had "unknown", "not found", or a very minimal description; here's an incomplete list:

• The perldelta pages for the three newly released versions:
  • 5.34.2 perldelta
  • 5.36.2 perldelta
  • 5.38.1 perldelta
• Perl: Products and vulnerabilities
• https://www.cve.org/
• NIST - National Vulnerability Database:
  • https://nvd.nist.gov/vuln/detail/CVE-2023-47038
  • https://nvd.nist.gov/vuln/detail/CVE-2023-47039
• cPanel: Perl CVE-2023-47038 and CVE-2023-47039
  • Some basic information; mostly how it relates to cPanel
• securityonline.info:CVE-2023-47038 & CVE-2023-47039: Two flaws found in Perl programming language
  • The most detailed of what I found; but still only high level descriptions.
  • As an example, CVE-2023-47038 had this sort of level of information: "This vulnerability emerges when a specially crafted regular expression is compiled, leading to a buffer overflow in a heap-allocated buffer"

By way of comparison, these CVEs have, or link to, detailed discussions, workarounds, and such like: "CVE-2023-31484 (CPAN)" and "CVE-2023-31486 (HTTP::Tiny)".

Test::CVE

Somewhat tangentially, as part of my searching I came across the extremely new — its timestamp had "AN HOUR AGO" — Test::CVE authored by your good self.

This looks like something I may add to the list of Standardised Author Tests that I've written for $work.

I do note from the INCENTIVE section:

"The functionality explicitly limits to passive analysis: the is no active scanning of source code to find security vulnerabilities."

Would I be correct in assuming that this would pick up issues with CPAN and HTTP::Tiny, but not a problematic \p{...}?

An affirmative answer would not preclude its use as one of my Standardised Author Tests; I would just want to document this limitation. As an example, another of these types of tests uses Test::MinimumVersion and its limitations are noted.

Hah, changing the subject :)

Test::CVE uses a database with known CPAN vulnerabilities and the versions in where these were fixed. By scanning <cpanfile</c>, Makefile.PL and possible other sources, the module looks for required and use modules/releases and possible declared versions. It will report if the declared version is open to CVE's. The advice from the security group would be to either require the version that fixed the CVE(s) or to make that version a recommendation and document that when using the older version, you are on your own.

As you stated, \p{...} would *not* be picked up by this module.


Enjoy, Have FUN! H.Merijn

"Fix read/write past buffer end: perl-security#140" appears to be a reasonably detailed description of CVE-2023-47038.

I'm still hunting around for a similar description of CVE-2023-47039.

CVE-2023-47038 is only relevant during the use of \p in regexes.

While im certain i don't use that construct in any of my codebases, i don't really have the week or so it takes to audit *all* my dependencies. Guess i'll be spending most of today reinstalling Perl+Modules on my 15+ systems i administrate...