Re^2: Problem with regex is a bug? or my regex (updated)

in reply to Re: Problem with regex is a bug? or my regex (updated)
in thread Problem with regex is a bug? or my regex

Thank you for your guide,

I managed to create a self contained example that reproduces the problem.

Now, I ran your test and it works on both perl versions, as you said.

But running the regex inside Expect.pm fails.

So the bug is in the expect package?

chain.txt

######################################################################
+########\r\n# This system is a restricted access system.             
+                  #\r\n# If collected security information reveals po
+ssible criminal activity that  #\r\n# exceeds privileges, evidence of
+ such activity may be provided to the rele- #\r\n# vantauthorities fo
+r further action. By continuing past this point, you #\r\n# expressly
+ consent to this security monitoring.              #\r\n#############
+#################################################################\r\n
+\r\nhostname: ~#
[download]

test.pl

use strict;
use Expect;

my $re1 = '(([#%:>~\$\] ])(?!\2)){3,4}|([\w\-\.]*)\$ *$|(\w[@\/]\w|sft
+p).*?[#%>~\$\]]|^[#%\$>\:]~] *$';
my $test;

open($test,"<","chain.txt");

my $exp = Expect->exp_init($test);

$exp->expect(1,
    [ $re1 => sub {
        my $exp = shift;
        print "Match before : ",$exp->before(),"\n";
        print "Match        : ",$exp->match(),"\n";
        print "Match after  : ",$exp->after(),"\n";
    }]
);

close $test;
[download]

hans@hans-desktop  ~  perl -v
This is perl 5, version 30, subversion 0 (v5.30.0) built for x86_64-li
+nux-gnu-thread-multi
hans@hans-desktop ~ perl -MExpect -e 'print $Expect::VERSION ."\n";'
1.21

perl test.pl
Match before : #######################################################
+#######################\r\n# This system is a restricted access syste
+m.                               #\r\n# If collected security informa
+tion reveals possible criminal activity that  #\r\n# exceeds privileg
+es, evidence of such activity may be provided to the rele- #\r\n# van
+tauthorities for further action. By continuing past this point, you #
+\r\n# expressly consent to this security monitoring.              #\r
+\n###################################################################
+###########\r\n\r\nhostname
Match        : : ~#
Match after  :
[download]

[hans@fedora ~]$ perl -v
This is perl 5, version 34, subversion 0 (v5.34.0) built for x86_64-li
+nux-thread-multi
[hans@fedora ~]$ perl -MExpect -e 'print $Expect::VERSION ."\n";'
1.35

[hans@fedora ~]$ perl test.pl 
Match before : #######################################################
+#######################\r\n# This system is a restricted access syste
+m.                             
Match        :   #
Match after  : \r\n# If collected security information reveals possibl
+e criminal activity that  #\r\n# exceeds privileges, evidence of such
+ activity may be provided to the rele- #\r\n# vantauthorities for fur
+ther action. By continuing past this point, you #\r\n# expressly cons
+ent to this security monitoring.              #\r\n##################
+############################################################\r\n\r\nh
+ostname: ~#
[download]

Comment on Re^2: Problem with regex is a bug? or my regex (updated) Select or Download Code

Replies are listed 'Best First'.
Re^3: Problem with regex is a bug? or my regex (updated) by haukex (Archbishop) on Nov 20, 2021 at 14:45 UTC
Thanks for posting the details. Expect 1.21 is about ten years older than 1.35 (2007 vs 2017). Since I'm unable to reproduce your issue with Expect 1.35 on both versions of Perl, I am guessing that the issue lies with one of the bugs that was fixed in Expect over those 10 years. I'd say your best course of action is to upgrade the module. Update: Sorry, I see now that you're getting your expected behavior on the older version of the module instead of the newer version. The Changelog does mention "Eliminate $` and $' from the code. part of (RT #61395) This fix might break some existing code n some extreme cases when the regex being matched has a lookbehind or a lookahead at the edges." which could potentially be a hint, but finding out if this actually is the issue will take a bit more digging.	[reply]
Re^4: Problem with regex is a bug? or my regex (updated) by hanspr (Sexton) on Nov 20, 2021 at 15:05 UTC
Strange, I can reproduce in my machine, I upgraded to 1.35 and now its broken. hans@hans-desktop ~ perl -MExpect -e 'print $Expect::VERSION ."\n";' 1.35 hans@hans-desktop ~ perl test.pl Match before : ####################################################### +#######################\r\n# This system is a restricted access syste +m. Match : # Match after : \r\n# If collected security information reveals possibl +e criminal activity that #\r\n# exceeds privileges, evidence of such + activity may be provided to the rele- #\r\n# vantauthorities for fur +ther action. By continuing past this point, you #\r\n# expressly cons +ent to this security monitoring. #\r\n################## +############################################################\r\n\r\nh +ostname: ~# [download]	[reply] [d/l]
Re^5: Problem with regex is a bug? or my regex by haukex (Archbishop) on Nov 20, 2021 at 15:47 UTC
I checked and can now confirm that the version where the behavior of your test script changes is Expect v1.26, which is where I quoted the Changelog entry from above. Specifically, it's commits `7ee4816` and `d9e0365` that each change the behavior of your script. The problem actually turns out to be fairly simple: The position of a capturing group was changed. Previously, the module did `/$pattern->[2]()/` where it now does `/($pattern->[2])/`, which changes the ordering of the capture groups and which breaks your `\2` backreference. Change the backreference to `\g-1` and your regex should work even on the newest version of Expect (at least my testing shows that it does).	[reply] [d/l] [select]
Re^6: Problem with regex is a bug? or my regex by hanspr (Sexton) on Nov 21, 2021 at 04:14 UTC
Re^3: Problem with regex is a bug? or my regex (updated) by duelafn (Parson) on Nov 20, 2021 at 15:01 UTC
For reference: $ perl -v This is perl 5, version 32, subversion 1 (v5.32.1) built for x86_64-li +nux-gnu-thread-multi $ perl -MExpect -e 'print $Expect::VERSION ."\n";' 1.21 $ perl 11138979.pl Match before : ####################################################### +#######################\r\n# This system is a restricted access syste +m. #\r\n# If collected security informa +tion reveals possible criminal activity that #\r\n# exceeds privileg +es, evidence of such activity may be provided to the rele- #\r\n# van +tauthorities for further action. By continuing past this point, you # +\r\n# expressly consent to this security monitoring. #\r +\n################################################################### +###########\r\n\r\nhostname Match : : ~# Match after : [download] Good Day, Dean	[reply] [d/l]

In Section Seekers of Perl Wisdom