in reply to counting words in string

There Is More Than One Way To Do It :-)

The method you showed is fine. If the input string starts getting long, you might have to watch out for memory usage. Here's a version that uses a regex and a while loop to scan the string, without building an intermediate @names array. I'm using the regex \S+ (one or more non-whitespace characters) to match "names", so that this code should produce the same output as yours regardless of the input string. If you need to match only certain characters in the names, you'd have to tell us more about that (Re: How to ask better questions using Test::More and sample data).

use warnings; use strict; use Test::More tests=>1; my $str = "iowq john stepy andy anne alic bert stepy anne bert andy st +ep alic andy"; my %names; pos($str)=undef; while ($str=~/\G\s*(\S+)(?:\s+|\z)/gc) { $names{$1}++; } die "failed to parse \$str" unless pos($str)==length($str); is_deeply \%names, { alic => 2, andy => 3, anne => 2, bert => 2, iowq => 1, john => 1, step => 1, stepy => 2 };

Replies are listed 'Best First'.
Re^2: counting words in string
by frazap (Monk) on Aug 08, 2018 at 14:26 UTC

    Since I'm a not an expert with perl regex, I start digging in the code of haukex with commenting it's original code and with searching a simpler loop.

    Here we go:

    use warnings; use strict; use Test::More tests=>2; my $str = "iowq john stepy andy anne alic bert stepy anne bert andy st +ep alic andy"; my %names; =for comment pos Returns the offset of where the last m//g search left off for the vari +able in question ($_ is used when the variable is not specified). Note that 0 is a valid match offset. undef indicates that the search position is reset (usually due to matc +h failure, but can also be because no match has yet been run on the s +calar). =cut pos($str)=undef; =for comment https://www.regular-expressions.info/continue.html The position where the last match ended is a "magical" value that is r +emembered separately for each string variable. The position is not associated with any regular expression. This means that you can use \G to make a regex continue in a subject s +tring where another regex left off. If a match attempt fails, the stored position for \G is reset to the s +tart of the string. To avoid this, specify the continuation modifier +/c. =cut while ($str=~/\G #start where the last match ended \s* #match 0 to n space char (\S+) #remember any non space char after that and followed by (?: #start clustering of \s+|\z #1 to n spaces or the end of the string ) #end clustering /gcx) { $names{$1}++; } die "failed to parse \$str" unless pos($str)==length($str); test_it (\%names); %names = (); #Takes a new variable #my $str2 = "iowq john stepy andy anne alic bert stepy anne bert andy +step alic andy"; #or reset pos for the original var pos($str)=undef; my $last; while ($str=~/(\w+)/g) { #print $1, " ", pos $str, "\n"; $names{$1}++; $last = pos $str; } die "failed to parse \$str" unless $last ==length($str); test_it(\%names); sub test_it { my $hr_names = shift; is_deeply $hr_names, { alic => 2, andy => 3, anne => 2, bert => 2, iowq => 1, john => 1, step => 1, stepy => 2 }; }
    I have 3 questions
    • Where is the /c modifier documented in the perldoc ? It is given at the end of pos but I couldn't find no other description
    • How comes that I have to remember pos $str in the second loop ? It's undef after the second loop but not after the first one
    • Are the two loops equivalent or will the second one failed in some situation ?

    Cheers

    François

      pos($str)=undef;

      Note: This may seem like overkill in the simple example I showed, but it might become important if the code is copy and pasted into a larger script, where it's possible that a previous regex operation on $str left its pos set (but we want to start matching at the beginning of the string).

      Where is the /c modifier documented in the perldoc ?

      Hm, you're right that the doc seems a little hard to find. There's a pretty good explanation in perlretut's "Global Matching", and a more complex example of the usage of m/\G.../gc in perlop under "\G assertion".

      How comes that I have to remember pos $str in the second loop ?

      That would be the effect of the /c modifier: your second loop ends when the regex fails to match. The regex failing to match also resets pos, except when /c is used.

      Are the two loops equivalent or will the second one failed in some situation ?

      The way you've written it, no, because \w+ is not the same as \S+: the latter will match any non-word characters too, anything except \s characters.

      If I change your loop's regex to /(\S+)/g, you can get the "failed to parse $str" error, namely when there's whitespace at the end of $str. This is because the last match doesn't reach all the way to the end of the string, while the regex I showed does, because it includes a \s+ after the \S.

      The pos==length check makes sense if you want to make sure your loop went all the way to the end of the string and there's no garbage left at the end that we didn't match, but in this case, the only thing that could possibly be left after matching \S+ is whitespace, which we're not interested in.

      searching a simpler loop

      I showed the m/\G.../gc technique because I wanted to demonstrate it - it's a great tool for certain types of parsing (one example in the first paragraph of this post, or this), but as long as the OP just wants to split on whitespace, it's admittedly overkill. If you wanted to simplify the loop, you could say this (others have shown variations of this):

      my %names; while ($str=~/(\S+)/g) { $names{$1}++; }

      But as soon as the matching rule isn't as simple as \S+, things can get tricky. For example, if the rule is \w+, and the input string is " foo bar *", anything that looks for \w+ only will miss the junk at the end of the string.