in reply to Re: RE on lines read from in-memory scalar is very slow
in thread RE on lines read from in-memory scalar is very slow

32 gigs of RAM. The file on disk is only 34M. My perl is 5.36.3 on cygwin.

I just ran it on another system (Linux) on the same file and got: 
0.457707 read lines from disk and do RE.
0.675398 read lines from in-memory file and do RE.

[download]
So it seems to be specific to my system. It doesn't seem to be memory as it only uses around 34M RAM when running on that file and my memory is nowhere near exhausted.

Replies are listed 'Best First'.
Re^3: RE on lines read from in-memory scalar is very slow
by eyepopslikeamosquito (Archbishop) on Jan 23, 2024 at 06:09 UTC

    > My perl is 5.36.3 on cygwin ... I just ran it on another system (Linux) on the same file ...
    > and got 0.675398 read lines from in-memory file and do RE (vs 48.796606 on cygwin)

    Wow, that is such a massive difference, I suspect something's seriously wrong with your cygwin perl v5.36.3 environment ... so I suggest you run your test program on cygwin with various tools to try to understand why on earth it's taking so long. Some tools you might try:

    Other suggestions welcome. I suspect a system level tool (such as VTune) is the best bet in this case, hopefully it will reveal something glaringly wrong (as it did for me here).

    For future reference, posting a test program using Benchmark, that anyone can run with zero effort, is the ideal way to get help on these sorts of performance problems (see Fastest way to lookup a point in a set (update: and Re: Confused by RegEx count by choroba) for examples).

    See also: Code Optimization and Performance References (and Memory Tools References) 

    👁️🍾👍🦟
[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom