Re^5: Splitting multiline string into words, the stuff between words, and newlines

Well, specifically for the apostrophe, \b{wb} doesn't seem to take initial ones as part of words. It breaks your sample sentence as follows: {Not} {_} {sure} {_'} {cause} {_} {that's} {_'} {bout} {_} {words} {_} {also} {_} {including}

But I agree with you that there are probably other cases of words (as defined by \b{wb}) that don't start by a character matching \w.

At the end, my conclusion is that the only way to handle the OP problem in a way fully consistent with \w{wb} semantics is to just split using it, and maybe repack non word fragments afterwards:

my $book = "Not sure 'cause that's 'bout words also including ...\n.\n
+_\n\n...";

my @fragments;
my $last_was_symbol;

for (split /\b{wb}/, $book) {
    if (/[\w\n]/) {
        $last_was_symbol = 0;
        push @fragments, $_;
    }
    else {
        if ($last_was_symbol) {
            $fragments[-1] .= $_;
        }
        else {
            push @fragments, $_;
            $last_was_symbol = 1;
        }
    }
}

sub show {
    my $str = shift;
    $str =~ tr/\n/$/;
    $str =~ tr/ /_/;
    print "{$str} ";
}

show $_ for @fragments;
print "\n";
[download]

Comment on Re^5: Splitting multiline string into words, the stuff between words, and newlines Select or Download Code

Replies are listed 'Best First'.
Re^6: Splitting multiline string into words, the stuff between words, and newlines by LanX (Saint) on Feb 25, 2022 at 11:11 UTC
> \b{wb} doesn't seem to take initial ones as part of words good catch! > my conclusion is that the only way to handle the OP problem in a way fully consistent with \w{wb} semantics is to just split using it, and maybe repack non word fragments afterwards My intuition says split on non-words like whitespace, reject "words" without `\w` or equivalent characters and repack the rest afterwards. I doubt it's possible to cover all desirable edge cases by `\b{wb}` this will depend on the user's perspective, especially when considering multi-language environments and unicode. Cheers Rolf _{(addicted to the Perl Programming Language :) Wikisyntax for the Monastery}	[reply] [d/l] [select]
Re^7: Splitting multiline string into words, the stuff between words, and newlines by salva (Canon) on Feb 25, 2022 at 11:19 UTC
This seems to work too: `my @fragments = $book =~ /\G(?:[^\n\w]+?\b{wb})+\|.+?\b{wb}/sg;` [download]	[reply] [d/l]


"be consistent"
	PerlMonks