Re^7: AI in the workplace (... in the Monastery)

(a longer meditation)

Many search engines nowadays come up with an AI generated preface summing up the results before showing the hits.

Of course we are not a search engine nor a code writing service, but a community tool helping others becoming better programmers.

I initially said it's a horrible idea, bc the way harangzsolt is proposing would lead to too many problems, (apart from implementation problems)

But let's try:

So I took the liberty to feed a current question into Duck.AI using ChatGPT 4o

> Why is "any" slow in this case?

Answer:

## 🧩 Benchmark Analysis of Perl Code

The benchmarks you've provided show the performance of different methods for filtering out "uninteresting" lines from a string of 2D data. Let's break down the results and analyze why certain methods perform better or worse than others.

### 📊 Benchmark Results Overview

Here's a summary of the benchmark results:

| Method | Rate (lines/sec) | Comparison to Others | |------------|------------------|----------------------| | any_cr | 331/s | -- | | any | 724/s | -54% | | ugly | 930/s | -22% | | ugly_cr | 1265/s | -43% |

### 🔍 Performance Analysis

#### 1. *Understanding the Methods*

**ugly**: Directly uses `$1` and `$2` from the regex match for comparisons.
**ugly_cr**: Assigns `$1` and `$2` to lexicals `$c` and `$r` before comparisons.
**any**: Uses the `any` function from `List::Util` to check if the values are in the `@skip` array.
**any_cr**: Similar to `any`, but assigns the regex matches to lexicals first.

#### 2. *Why "ugly" is Slower*

The performance drop in "ugly" compared to "ugly_cr" can be attributed to the fact that `$1` and `$2` are read-only and are re-evaluated each time they are used in a comparison. This means that every time you check `$1` or `$2`, Perl has to numify them again, which adds overhead. In contrast, "ugly_cr" assigns these values to lexicals, which are faster to access.

#### 3. *Why "any_cr" is Slower*

The "any_cr" method is slower than "any" because of the additional overhead of assigning values to lexicals before performing the checks. While lexicals are generally faster to access, the overhead of the assignment in this case outweighs the benefits when combined with the `any` function. The `any` function itself is optimized for performance, but the added step of assigning to lexicals introduces unnecessary complexity in this context.

#### 4. *Asymmetry Between "ugly vs. ugly_cr" and "any vs. any_cr"*

**ugly vs. ugly_cr**: The performance difference is primarily due to the repeated numification of `$1` and `$2` in "ugly". In "ugly_cr", the values are stored in lexicals, which reduces the overhead of repeated access.
**any vs. any_cr**: The performance drop in "any_cr" is due to the overhead of assigning values to lexicals before checking against the `@skip` array. The `any` function is efficient enough that the overhead of assignment negates its benefits in this case.

### 🧠 Conclusion

In summary, the performance differences can be attributed to:

*Repeated numification* in "ugly" leading to slower performance.
*Overhead of assignments* in "any_cr" outweighing the benefits of using lexicals.
The *efficiency of the `any` function* making it faster than the lexically assigned version.

If you're looking to optimize further, consider sticking with the "ugly_cr" method for its balance of readability and performance, especially if you plan to expand the list of values to skip. The `any` function is great for larger lists but may not be necessary for just a few values.

It reads interesting - like always with AI - but requires deeper analysis for disguised non-sense. (Disclaimer: I didn't)

Definitely nothing an amateur could handle. But an expert can draw inspiration from this.

For instance I was intrigued by the idea of $1 being slow because it can't optimize numification.(point 2)

So I asked for clarification, guessing this being about dual-values.

> why is numification of read-only slower

Answer:

Again interesting, but I'm not convinced. The claim (Point 2) that "read-only variables do not benefit from caching" can't be reproduced. Because $1 is indeed also a dual-value.

:~$ perl -MDevel::Peek -E'"42"=~/(\d+)/; say $1; Dump $1; $a=$1+1; Dum
+p $1'
42
SV = PVMG(0x5d60b28468a0) at 0x5d60b2868a50
  REFCNT = 1
  FLAGS = (GMG,SMG,POK,pPOK)
  IV = 0
  NV = 0
  PV = 0x5d60b283f5f0 "42"\0
  CUR = 2
  LEN = 16
  MAGIC = 0x5d60b286f0e0
    MG_VIRTUAL = &PL_vtbl_sv
    MG_TYPE = PERL_MAGIC_sv(\0)
    MG_OBJ = 0x5d60b2868a38
    MG_LEN = 1
SV = PVMG(0x5d60b28468a0) at 0x5d60b2868a50
  REFCNT = 1
  FLAGS = (GMG,SMG,IOK,POK,pIOK,pPOK)
  IV = 42                                             #<--- Caching
  NV = 0
  PV = 0x5d60b283f5f0 "42"\0
  CUR = 2
  LEN = 16
  MAGIC = 0x5d60b286f0e0
    MG_VIRTUAL = &PL_vtbl_sv
    MG_TYPE = PERL_MAGIC_sv(\0)
    MG_OBJ = 0x5d60b2868a38
    MG_LEN = 1
:~$
[download]

Probably another caching mechanism is meant??? Nope ChatGPT confirms that dual-values are meant:

MY CONCLUSION (so far):

LLM output can inspire good ideas in our context but require an expert to deal with. The wordy answers are often full of hidden pitfalls and contrary to a human being the LLM doesn't even try to cross-check what it (hear-)says.

Is there is a meaningful way to integrate an LLM into our question/answer process?
I don't know. But using normal questions as prompts is obviously a horrible idea.
But can we stop amateurs posting AI generated texts here, in order to pose as an expert?
Hardly. This is most likely already happening, at least with AnoMonk posts.
Will we be forced to increasingly deal with AI here?
Very likely, at least in requiring us to increase the quality standards of answers or demanding POCs.

Cheers Rolf
_{(addicted to the Perl Programming Language :)

see Wikisyntax for the Monastery}

IN HINDSIGHT

This should probably better be a meditation in it's own thread.

Disclaimer

This test is far from methodical. I ran this conversation with Duck.AI after the original question already got many answers. Hence the replies might have already been training and influencing the LLM.

Comment on Re^7: AI in the workplace (... in the Monastery) Select or Download Code

Replies are listed 'Best First'.
Re^8: AI in the workplace (... in the Monastery) by ikegami (Patriarch) on Jul 28, 2025 at 18:30 UTC
The performance drop in "ugly" compared to "ugly_cr" can be attributed to the fact that `$1` and `$2` are read-only and are re-evaluated each time they are used in a comparison. This means that every time you check `$1` or `$2`, Perl has to numify them again, which adds overhead. In contrast, "ugly_cr" assigns these values to lexicals, which are faster to access. Perl does have to numify them again, but not for the reason given. `no warnings qw( void ); use Devel::Peek qw( Dump ); "a" =~ /(.)/s or die; 0+$1; # Fetch and numify Dump($1); my $x = $1; # Fetch Dump($1);` [download] `... FLAGS = (GMG,SMG,POK,pIOK,pNOK,pPOK) ... FLAGS = (GMG,SMG,POK,pPOK) ...` [download] As you can see, `$1` gets numified. But every time you read from it, it gets repopulated since it's a magic variable. This wipes the previous values. In the context of analyzing the AI's answer, it's worth noting that I missed the repeated numification in my answer. I stopped too soon. The "any_cr" method is slower than "any" because of the additional overhead of assigning values to lexicals before performing the checks. That can't be true since `ugly_cr` is way faster than `ugly`. The actual culprit is the overhead from the addition of capturing. In the context of analyzing the AI's answer, it's worth noting the response is self-contradicting. According to the AI, assigning to the lexicals makes the cr version faster by only doing numification once, but it makes the cr version slower because of the addition of an assignment.	[reply] [d/l] [select]
Re^9: AI in the workplace (... in the Monastery) by LanX (Saint) on Jul 28, 2025 at 20:20 UTC
In the example I've shown, the IV slot is set after the first numification. In fact it doesn't look to me like repeated numification (or magic fetching) was happening. `:~$ perl -MDevel::Peek -E'"42"=~/(\d+)/; say $1; Dump $1; $a=$1+1; Dum +p $1; say $1; Dump $1' 2>&1 \|grep IV IV = 0 IV = 42 IV = 42 :~$` [download] Edit hm ... probably the IV values don't matter if the appropriate `I` flags aren't set. `~$ perl -MDevel::Peek -E'"42"=~/(\d+)/; say $1; Dump $1; $a=$1+1; Dump + $1; say $1; Dump $1' 2>&1 \|grep FLAGS FLAGS = (GMG,SMG,POK,pPOK) FLAGS = (GMG,SMG,IOK,POK,pIOK,pPOK) FLAGS = (GMG,SMG,POK,pPOK)` [download] Cheers Rolf _{(addicted to the Perl Programming Language :) see Wikisyntax for the Monastery}	[reply] [d/l] [select]
Re^10: AI in the workplace (... in the Monastery) by ikegami (Patriarch) on Jul 28, 2025 at 20:25 UTC
Correct, you should be looking at the flags.	[reply]

MY CONCLUSION (so far):

IN HINDSIGHT

Disclaimer

Edit