cyocum has asked for the wisdom of the Perl Monks concerning the following question:

Oh great and wise monks,

I have been working on a couple of example uses for perl in the humanities. See this node for some background information. I have run into a problem however that I hope you can help me solve.

I was going to show them how to parse up some of the information in the Annals of Ulster. For example, 

U448.1\n
\n
Several walls of the imperial city of Constantinople which had been fr
+eshly\n
rebuilt with masonry, and fifty-seven towers collapsed as a result of 
+a\n
violent earthquake which prevailed in various places.\n
\n
U483.2\n
\n
Or, this year the battle of Ocha according to others, won by Muirchert
+ach\n
Mac Erca and by Fergus Cerball son of Conall of Cremthann, and by Fiac
+hra\n
Lon son of the king of Dal nAraide.\n
\n

[download]

Now as you can see, it always starts with the letter 'U', which is a good thing to anchor the regex. Then it has two \n another good solid thing. Then I run into problems because the lines of text for a paragraph are broken up by single \n then it ends with two \n. So I come up with a regex something like this: 

/U(\d+\.\d+)\n{2}(.+)\n{2}/

[download]

The problem is that if I use the regex modifier /s it is completely greedy and matches everything rather than just things between the two \n\n delimiters. What I would like is a regex that matches multiple lines including single \n but stops when it gets to \n\n. I am sure that I am missing something simple but I cannot think of it right now and humbily beseech your wisdom on the topic.

Replies are listed 'Best First'.
Re: Regex Question
by tachyon (Chancellor) on Apr 11, 2004 at 12:23 UTC

    Assuming you want the best solution to the problem then the way to parse a data file that contains records separated by some delimiter ie two newlines (aka a blank line) is to set the input record separator to that delimiter and then read the data in one record at a time. For example in your case you could parse the data into a hash like this: 

    #!/usr/bin/perl
local $/ = "\n\n";
while(<DATA>) {
    next unless m/U\d+/;
    chomp;
    chomp(my $text = <DATA>);
    $h{$_} = $text;
}
use Data::Dumper;
print Dumper \%h;
__DATA__
U448.1

Several walls of the imperial city of Constantinople which had been fr
+eshly
rebuilt with masonry, and fifty-seven towers collapsed as a result of 
+a
violent earthquake which prevailed in various places.

U483.2

Or, this year the battle of Ocha according to others, won by Muirchert
+ach
Mac Erca and by Fergus Cerball son of Conall of Cremthann, and by Fiac
+hra
Lon son of the king of Dal nAraide.

    [download]

    An inefficient regex solution is: 

    my $str = join '', <DATA>;
%bits = $str =~ m/^(U[\d\.]+)\n\n(.*?)\n\n/msg;
use Data::Dumper;
print Dumper \%bits;
__DATA__
[snip]

    [download]

    You need .*? /m /s and /g to make it work and it is not efficient due to backtracking. See perlre

    cheers

    tachyon

[reply]
[d/l]
[select]
Re: Regex Question
by borisz (Canon) on Apr 11, 2004 at 12:23 UTC
    I think you ask for ? to make it non greedy and s to make . include newlines. 
    /U(\d+\.\d+)\n{2}(.+?)\n{2}/s;

    [download]
    update:tachyon point me to a typeo in the regex the ? was after the )
    Boris
[reply]
[d/l]

      The ? needs to be inside the capturing parenths like  (.+?). In your example  (.+)? you have code that effectively reads the least ammount of greedy matches and as a result fails to DWIM.

[reply]
[d/l]
[select]
Re: Regex Question
by Aragorn (Curate) on Apr 11, 2004 at 22:50 UTC
    A solution in which the whole file is read into memory and processed in chunks: 
    #!/usr/bin/perl

use strict;
use warnings;

$/ = undef;

my $string = <DATA>;

while ($string =~ /(            # Capture into $1
                     ^U\d+\.\d  # A 'U' at the start of a line, and 2
                                # dot-separated numbers
                   )
                   \n\n         # 2 newlines
                   (            # Capture into $2
                     (?:
                       [^\n]    # Match anything but a newline
                       |        # or
                       \n(?!\n) # A newline, but only if not followed
                                # by another newline
                     )+
                   )
                   /mgx) {
    print "\$1: <$1>:\n\$2: \n<$2>\n\n";
}

__DATA__
U448.1

Several walls of the imperial city of Constantinople which had been fr
+eshly
rebuilt with masonry, and fifty-seven towers collapsed as a result of 
+a
violent earthquake which prevailed in various places.

U483.2

Or, this year the battle of Ocha according to others, won by Muirchert
+ach
Mac Erca and by Fergus Cerball son of Conall of Cremthann, and by Fiac
+hra
Lon son of the king of Dal Araide.

    [download]
    Arjen
[reply]
[d/l]
Re: Regex Question
by cyocum (Curate) on Apr 11, 2004 at 12:46 UTC

    Thanks for your answers. I knew it was something as simple as '?'. tachyon, I will look into your solution there are other complications that I did not present here but your solution looks good. Thanks again!

[reply]

Back to Seekers of Perl Wisdom