http://qs1969.pair.com?node_id=11144479


in reply to How to remove everything after last occurrence of a string?

G'day ovedpo15,

I think your biggest problem here is that you've immediately reached for a regex solution. When I first saw the question title, before even looking at the content, my first thought was rindex and substr.

Perl's string handling functions (and operators) are, in my experience, substantially faster than achieving the same functionality with regexes. There are times when a regex is appropriate; however, often it's not the best solution.

So, rather than making guesses and assumptions, let's Benchmark. 

#!/usr/bin/env perl

use 5.010;
use strict;
use warnings;

use constant {
    PATH => 0,
    VERSION => 1,
    WANT => 2,
};

use Benchmark 'cmpthese';
use Test::More;

my @tests = (
    [qw{
        /a/b/version1/c/d
        version1
        /a/b/version1
    }],
    [qw{
        /some/path/3.5.2+tcl-tk-8.5.18+sqlite-3.10.0/a/b/c
        3.5.2+tcl-tk-8.5.18+sqlite-3.10.0
        /some/path/3.5.2+tcl-tk-8.5.18+sqlite-3.10.0
    }],
);

plan tests => 2*@tests;

for my $test (@tests) {
    is _regex(@$test[PATH, VERSION]), $test->[WANT];
    is _rindex(@$test[PATH, VERSION]), $test->[WANT];
}

cmpthese 0 => {
    regex0 => sub { _regex(@{$tests[0]}[PATH, VERSION]); },
    regex1 => sub { _regex(@{$tests[1]}[PATH, VERSION]); },
    rindex0 => sub { _rindex(@{$tests[0]}[PATH, VERSION]); },
    rindex1 => sub { _rindex(@{$tests[1]}[PATH, VERSION]); },
};

sub _regex {
    my ($path, $version) = @_;

    $path =~ s/.*\Q$version\E\K.*//s;

    return $path;
}

sub _rindex {
    my ($path, $version) = @_;

    $path = substr $path, 0, length($version) + rindex $path, $version
+;

    return $path;
}

[download]

The Test::More code is just to ensure the functions are producing correct results; which they are. The output from that is identical in all runs, so I'll just post it once: 

1..4
ok 1
ok 2
ok 3
ok 4

[download]

I ran the benchmark five times. Here's the sections of output that relate to that: 

             Rate  regex1  regex0 rindex0 rindex1
regex1   809214/s      --     -7%    -69%    -75%
regex0   866627/s      7%      --    -67%    -73%
rindex0 2632175/s    225%    204%      --    -19%
rindex1 3257343/s    303%    276%     24%      --

             Rate  regex1  regex0 rindex0 rindex1
regex1   825777/s      --     -5%    -68%    -75%
regex0   870952/s      5%      --    -66%    -73%
rindex0 2579956/s    212%    196%      --    -21%
rindex1 3261289/s    295%    274%     26%      --

             Rate  regex1  regex0 rindex0 rindex1
regex1   807841/s      --     -8%    -69%    -75%
regex0   880657/s      9%      --    -66%    -73%
rindex0 2625422/s    225%    198%      --    -20%
rindex1 3265104/s    304%    271%     24%      --

             Rate  regex1  regex0 rindex0 rindex1
regex1   807626/s      --     -7%    -69%    -75%
regex0   873101/s      8%      --    -66%    -73%
rindex0 2567429/s    218%    194%      --    -21%
rindex1 3255180/s    303%    273%     27%      --

             Rate  regex1  regex0 rindex0 rindex1
regex1   827447/s      --     -6%    -68%    -75%
regex0   877972/s      6%      --    -66%    -73%
rindex0 2579110/s    212%    194%      --    -21%
rindex1 3260240/s    294%    271%     26%      --

[download]

Do you still want to go with a regex solution? :-)

— Ken

Replies are listed 'Best First'.
Re^2: How to remove everything after last occurrence of a string?
by GrandFather (Saint) on Jun 07, 2022 at 21:29 UTC
    Do you still want to go with a regex solution? :-)

    No, because the rindex approach is likely to be more maintainable than the regex solution.

    While execution time can sometimes be critical, mostly it doesn't matter at all. It is generally much more important for code to be correct and maintainable than fast. If fast is a side effect of correct and maintainable code (often it is) then so much the better, but fast comes way down the list during the first stages of designing a coding solution.

    Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond
[reply]

      G'day GrandFather,

      ++ Thanks for your input.

      I did possibly end up (implicitly) suggesting that speed was the all-important factor.

      I have seen innumerable cases where regexes have been used to test exact matches (/^some_string$/), test if strings start with some token (/^some_token/), and so on. As I said originally, "Perl's string handling functions (and operators) are, in my experience, substantially faster than achieving the same functionality with regexes."; as such, reaching for a regex first has become something of an annoyance for me.

      The OP had asked "How it can be acheived without escapeing the special chars?" and I rather thought that was implicit in my "rindex" code. Perhaps I should have highlighted that.

      I also answered the OP's title question, "How to remove everything after last occurrence of a string?". Again, I didn't highlight that.

      I hadn't really considered the maintainability aspect but, I agree, the "rindex" code is easily understandable and works in all versions of Perl5; that's not to say that I shy away from regexes (see "Syntax-highlight Non-Perl Code for HTML"). Furthermore, if those maintaining the code are expected to have a solid grounding in regexes, then I'd say that neither solution is particularly complex and both are equally maintainable (a YMMV situation).

      The OP may have a very specific reason for choosing a regex solution; however, if not, why not choose an alternative that's three times faster.

      — Ken

[reply]
[d/l]
[select]
      Do you still want to go with a regex solution? :-)

      No | Yes, because (in this case at least) the rindex solution is wrong.

      How can one say any result is incorrect if the OPer has specified no clear set of requirements for results? I admit this is tricky, but one can say the use of s/// in the OPed example code implies that the string operand should be unchanged if no version substring match is found. _rindex() in the code here fails to do this.

      It's easy enough to define an rindex-based function that handles the no-match case (and it might even be a bit faster). But the argument seems to be that one should avoid using and learning about regexes because they are a bit arcane (indeed, regexes are the most counter-intuitive programming construct I know) and may vary a bit from language to language. This argument can be extended to languages themselves: We should not use Perl because it's not Python; not use Python because it's not C++; not use C++ because it's not...

      To answer a use-case such as that described in the OP, I tend to reach first for a regex solution because it most clearly represents and achieves the required operation, not because it is the fastest (although sometimes it is). Implementing the required operation in terms of index/rindex, substr, etc., is possible, but may have its own pitfalls and drawbacks in terms of basic correctness, readability and maintainability.

      These are all my own very personal preferences; others may differ.

      Read more... (3 kB)

      Update: Rats... Trashed the thrust of the entire post by getting the very first word wrong. Oh, well...

      Give a man a fish:  <%-{-{-{-<

[reply]
[d/l]
[select]

        It comes down to choosing the right tool for the task at hand. That is an art in itself. Often a regex is the right tool to solve a tricky matching problem. Sometimes something like index is the tool to use for a simple matching problem or where speed is critical. My main point is that raw execution speed should not be the first consideration when choosing tools to solve a problem. Demonstrable correctness and maintainability should come first and often go hand in hand. At this point the discussion doesn't have much to do with the OP or the specific implementation of any reply.

        My reply was addressing what could be seen as a "speed first" approach to coding being advocated by kcott's reply. Ken is an experienced coder so he can skip a few iterations ahead to get a solution that is both clean and quick. That is something that comes with experience, often bitter, so I thought a heads up for the less experienced was in order.

        Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond
[reply]

In Section Seekers of Perl Wisdom