in reply to RE on lines read from in-memory scalar is very slow

G'day Danny,

I can confirm and reproduce the types of results you've presented. I have Cygwin running on Win10 with the latest updates as of a few days ago.

$ uname -a CYGWIN_NT-10.0-19045 titan 3.4.10-1.x86_64 2023-11-29 12:12 UTC x86_64 + Cygwin

I have a variety of Perl versions installed using Perlbrew:

$ perlbrew list * perl-5.39.3 perl-5.38.0 perl-5.36.0 perl-5.34.0 perl-5.33.5 perl-5.32.0 perl-5.30.0

I used perl-5.39.3, perl-5.36.0 and perl-5.30.0 for the tests below.

I didn't know too much about your test data. I created these:

I made a verbatim copy of your posted code: "op_code.pl". Here's the test results:

v5.39.3

$ perl -v | head -2 | tail -1 This is perl 5, version 39, subversion 3 (v5.39.3) built for cygwin-th +read-multi $ ./op_code.pl test_data 0.111368 read lines from disk and do RE. 0.096441 read lines from in-memory file and do RE. $ ./op_code.pl test_data_Q 0.356852 read lines from disk and do RE. 398.759730 read lines from in-memory file and do RE.

v5.36.0

$ perl -v | head -2 | tail -1 This is perl 5, version 36, subversion 0 (v5.36.0) built for cygwin-th +read-multi $ ./op_code.pl test_data 0.118364 read lines from disk and do RE. 0.094379 read lines from in-memory file and do RE. $ ./op_code.pl test_data_Q 0.362871 read lines from disk and do RE. 399.115542 read lines from in-memory file and do RE.

v5.30.0

$ perl -v | head -2 | tail -1 This is perl 5, version 30, subversion 0 (v5.30.0) built for cygwin-th +read-multi $ ./op_code.pl test_data 0.112352 read lines from disk and do RE. 0.096379 read lines from in-memory file and do RE. $ ./op_code.pl test_data_Q 0.327187 read lines from disk and do RE. 398.730810 read lines from in-memory file and do RE.

I have ~32GB memory available; at no time during the running of these tests was more than 6.5GB used.

I see throughout this thread that both you and others have not encountered this anomaly on other operating systems: I can't readily discern a reason for it.

— Ken

Replies are listed 'Best First'.
Re^2: RE on lines read from in-memory scalar is very slow
by Danny (Chaplain) on Jan 23, 2024 at 09:59 UTC
        >I can confirm and reproduce the types of results you've presented

    awesome! thanks for checking.