in reply to Re^3: How to match regex over multiline file
in thread How to match regex over multiline file

I tried using your code with a couple of changes, primarily substituting in the regex I had for the my sentences (the data that I'm using has a ton of punctuation, such as ... and "" and so on, so the regex you had in your example wasn't what I needed and was cutting out a lot of the data). However, when I put my regex in, it went back to the same issue I had before, which was that it would print out the first paragraph with the sentence brackets around the sentences but then print out the same paragraph again but with the paragraph brackets around the whole paragraph...

Just so you can have an idea of what I changed, I've included the code below, with an excerpt of the text file I'm using.

local $/ = ""; open $fh, $ARGV[0] or die "File $ARGV[0] not found!\n"; $scount = 0; $pcount=0; @paragraphs; while ($paragraph = <$fh>){ @sentences; while($paragraph =~ /\s*(([A-Z][A-Za-z]*)(((([A-Za-z]|[0-9])*((\'* +|\-*)[A-Za-z]*))\s*(\.{3})*\!*\"*\(*\)*\,*\:*\s*)*(([A-Za-z]|[0-9])*) +)(\.|\?|\!))/g){ push @sentences, "<s>$1</s>"; $scount++; } push @paragraphs, "<p>\n\t" . join("\n\t", @sentences) . "\n</p>\n +"; $pcount++; } print for @paragraphs; print "\n Total Lines: $scount\n"; print "\n Total Paragraphs: $pcount\n";

Data:

But the truth is that in the short run, markets can occasionally be pushed, especially when so many decisions to buy or sell are keyed off what everyone else in the market is doing. Chain reactions are not much harder to start (in fact, given how quickly price moves get noticed, they may be easier) than they were 70 years ago.

All that notwithstanding, the interesting thing about the Greenspan resignation rumor was that it raised an obvious question: Would it really matter? As Jacob Weisberg just pointed out in " Ballot Box," Steve Forbes is apparently the only American who doesn't think Greenspan has done a terrific job as Fed chairman. And most of us would be happy to have Greenspan stay in office even after his current term expires in the middle of next year. But it's interesting to note that in the past couple of months there have been more than a few voices--including those of economists Greg Mankiw and Robert Barr--suggesting that Greenspan has been more the beneficiary of good economic fundamentals than the creator of them.

That position may be a bit overstated, particularly since Greenspan has shown an unusual ability to let his thinking on inflation, productivity, and the economy's possible growth rate evolve in response to changing data. But the essential point, that the soundness of this economy does not depend on Greenspan's presence at the head of the Fed, is right. That might not be the case if Greenspan's successor were either an inflation dove like William Greider or a perma-bear like Jim Grant. But whoever would succeed Greenspan would be nothing of the sort. He or she would be, in a word, Greenspanian, still concerned about the possibility of an overheating economy but also convinced that important technological changes have allowed this economy to grow faster than in the past without sparking inflation.

If anything, in fact, the bond market should have rallied on news that Greenspan might be stepping down, since he has long since stopped being paranoid enough for bondholders, who seem perpetually convinced that the United States is about to become Brazil. There are certainly Fed governors out there who would be far more likely to raise interest rates aggressively at the first hint of price pressures than Greenspan.

Replies are listed 'Best First'.
Re^5: How to match regex over multiline file
by aaron_baugher (Curate) on Oct 12, 2013 at 12:49 UTC

    I don't know about your regex, but since you don't limit @sentences to the context of your while loop with my, it's a global variable. It doesn't go out of context at the end of the loop, so each time through the loop it retains the elements it already had, and then you push the next paragraph's sentences onto it. Like this:

    for my $l ('a'..'d'){ @list; push @list, $l; } print @list: # prints qw( a a b a b c a b c d );

    You should learn to localize variables within a block with my, and use strict to tell you when you forgot to do that. Failing that, you should empty @sentences at the start of each loop with @sentences=(); . But really, learn strict and my.

    Aaron B.
    Available for small or large Perl jobs; see my home node.

Re^5: How to match regex over multiline file
by Athanasius (Archbishop) on Oct 13, 2013 at 03:01 UTC

    aaron_baugher++ has solved the problem. But as to the regex, why reinvent the wheel when has modules to identify English sentences? For example, Lingua::EN::Sentence contains a get_sentences function which seems to do nicely:

    #! perl use strict; use warnings; use Lingua::EN::Sentence 'get_sentences'; local $/ = ''; # Paragraph mode my $sentence_count = 0; my $paragraph_count = 0; my @paragraphs; while (my $paragraph = <DATA>) { my $sentences = get_sentences($paragraph); @$sentences = map { '<s>' . $_ . '</s>' } @$sentences; push @paragraphs, "<p>\n\t" . join("\n\t", @$sentences) . "\n</p>\ +n"; $sentence_count += scalar @$sentences; ++$paragraph_count; } print "\nTotal sentences: $sentence_count\n"; print "Total paragraphs: $paragraph_count\n"; print for @paragraphs; __DATA__ But the truth is that in the short run, markets can occasionally be pu +shed, especially when so many decisions to buy or sell are keyed off +what everyone else in the market is doing. Chain reactions are not mu +ch harder to start (in fact, given how quickly price moves get notice +d, they may be easier) than they were 70 years ago. This is a sentence containing ... an ellipsis. "Well, OK then," he sai +d, "but let's not get ahead of ourselves." And so to bed... And in this sentence, dialogue is delimited with single quotes. 'Well, + OK then,' he said, 'but let's not get ahead of ourselves.' And the h +euristic still works!

    Output:

    12:59 >perl 741a_SoPW.pl Total sentences: 8 Total paragraphs: 3 <p> <s>But the truth is that in the short run, markets can occasio +nally be pushed, especially when so many decisions to buy or sell are + keyed off what every one else in the market is doing.</s> <s>Chain reactions are not much harder to start (in fact, give +n how quickly price moves get noticed, they may be easier) than they +were 70 years ago.</s> </p> <p> <s>This is a sentence containing ... an ellipsis.</s> <s>"Well, OK then," he said, "but let's not get ahead of ourse +lves."</s> <s>And so to bed...</s> </p> <p> <s>And in this sentence, dialogue is delimited with single quo +tes.</s> <s>'Well, OK then,' he said, 'but let's not get ahead of ourse +lves.'</s> <s>And the heuristic still works!</s> </p> 12:59 >

    Hope that helps,

    Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,