in reply to Splitting a Sentence

Quick note -- it's not actually working so well yet, even disregarding abbreviations. Consider the following input:

This is a sentence. This is another sentence. This is a third sentence, which also happens to be spanning a line. This is a sentence as well... I think. This is an abbreviation, cf. the list posted on Perlmonks. This is a sentence; this is the last sentence.

This produces:

This is a sentence. This is another sentence. This is a thirdsentence, which also happens to be spanning a line. This is asentence as well. I think. This is an abbreviation, cf. Perlmonks. This is a sentence;

As opposed to:

This is a sentence. This is another sentence. This is a third sentence, which also happens to be spanning a line. This is a sentence as well... I think. This is an abbreviation, cf. the list posted on Perlmonks. This is a sentence; this is the last sentence.

Note how the third and fourth one are missing a space and how lowercase characters following semicolons or periods aren't handled correctly.

Try the following -- add a space in your loop, and use a lookahead assertion to take a peek at what's following a period or colon:

#!/usr/bin/perl use feature qw/say/; use strict; use warnings; my $s; my @arr; while(<>) { chomp $_; $s .= $_ . " "; } @arr = $s =~ m/[A-Z].+?[.;](?=[^.;][A-Z]|\s*$)/g; foreach (@arr) { say; }

This produces:

This is a sentence. This is another sentence. This is a third sentence, which also happens to be spanning a line. This is a sentence as well... I think. This is an abbreviation, cf. the list posted on Perlmonks.

As you can see, it's not perfect -- it still splits if you have e.g. a run-on sentence followed by an "I", or in fact any uppercase word, e.g. a proper noun --, but it mostly handles abbreviations (arbitrary ones, even). What I'd do to fix the remaining edge cases is add another processing step after the regex where you loop over @arr, check if each element ends with a known abbreviation, and join that element with the next one if so.

Replies are listed 'Best First'.
Re^2: Splitting a Sentence
by SimonPratt (Friar) on Jul 03, 2014 at 16:00 UTC

    Try this one on for size:

    [A-Z].+?[.;](?<!\.{3})(?=[^.;][A-Z]|\s*$)

      Works like a charm. BTW, I was gonna say that

      m/[A-Z].+?[.;](?<!\.{2,})(?=[^.;][A-Z]|\s*$)/g

      would perhaps be even better, so as to not hardcode any specific number of periods for run-on sentences, but it turns out that:

      Variable length lookbehind not implemented in regex m/[A-Z].+?[.;](?<! +\.{2,})(?=[^.;][A-Z]|\s*$)/

      Sigh. (Oh well, at least there's workarounds; see Why is variable length lookahead implemented while lookbehind is not?.)

Re^2: Splitting a Sentence
by ardibehest (Novice) on Jul 03, 2014 at 14:50 UTC
    Many thanks, I tried it and it seems to work pretty well, The loop over @arr seemed to work for a couple of abbreviations. I will need to add all of them and I am sure it will work pretty decently.