Eshan_k has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks, I am seeking perl wisdom again. I have a big file with 5000 lines. I want to print lines between three patterns(basically a block of code). I tried making multi-line regex as follows but seems no help. Till now I tried,

my $beg = qr/^\s*APP>\sSTART_PARTU_8X8\n\s*APP>\sPARTITION_SPLIT/; my $end = qr/^\s*APP>\sEND_PARTU_8X8/; if (/$beg/../$end/){ print "$line\n"; }

Input data:

APP> START_PARTU_8X8 APP> PARTITION_NONE START_PARTU_16X16 APP> END_PARTU_8X8 APP> START_PARTU_8X8 APP> PARTITION_SPLIT START_PARTU_4X4 ........ START_PARTU_4X4 ........ START_PARTU_4X4 ........ START_PARTU_4X4 ........ APP> END_PARTU_8X8 PARTU_16X4 16X16 4X16 APP> END_PARTU_8X8

I want to print lines between pattern (these will be two consecutive lines thats why I am trying multi line regex)

APP> START_PARTU_8X8 APP> PARTITION_SPLIT

and 1st occurrence of

APP> END_PARTU_8X8
Output: START_PARTU_4X4 START_PARTU_4X4 START_PARTU_4X4 START_PARTU_4X4

Can anybody please help/suggest something ?

Replies are listed 'Best First'.
Re: Print lines between multi-line regex pattern
by haukex (Archbishop) on Jun 11, 2018 at 09:56 UTC

    When dealing with multi-line data, there are two basic possibilities: Either (1) read the entire file into memory (5000 lines is not that big, especially if the lines are only as short as you've shown) and apply a regex over the whole thing, or (2) read the file line-by-line and keep some kind of state, keeping in mind that you're only looking at one line at a time (Update: or, if the state you are keeping is a buffer, like a sliding window, a few lines). This second point is why your code isn't working: You're expecting your regex $beg to match two lines since you've included a \n in your regex, but if you're reading the file line-by-line, you'll only ever have one line in $_ at a time (or $line, that's unclear from your code).

    The downside of the first approach is that it uses more RAM and will start being problematic if your files grow. Here's an example:

    use warnings; use strict; my $data = do { open my $fh, '<', 'sample.txt' or die $!; local $/; <$fh> }; # slurp whole file into memory my $regex = qr{ ^ \s*APP>\sSTART_PARTU_8X8 \n ^ \s*APP>\sPARTITION_SPLIT \n (.*?) ^ \s*APP>\sEND_PARTU_8X8 $ }msx; if ($data=~$regex) { print $1; }

    The downside of the second approach is that it makes the logic a bit more complex, but I tend to prefer it since it allows for any size input file, and it can be (IMO) more easily customized and extended for more complex state changes. Here's an example:

    use warnings; use strict; my $beg1 = qr/^\s*APP>\sSTART_PARTU_8X8$/; my $beg2 = qr/^\s*APP>\sPARTITION_SPLIT$/; my $end = qr/^\s*APP>\sEND_PARTU_8X8$/; open my $fh, '<', 'sample.txt' or die $!; my $prev_line; my $state = 'not_in_block'; while ( my $line = <$fh> ) { if ( $state eq 'not_in_block' ) { if ( $prev_line && $prev_line=~$beg1 && $line=~$beg2 ) { $state = 'am_in_block'; } } elsif ( $state eq 'am_in_block' ) { if ( $line=~$end ) { $state = 'not_in_block'; } else { print $line; } } } continue { $prev_line = $line } close $fh;

    Using the sample input data from your post as sample.txt, both pieces of code output:

    START_PARTU_4X4 ........ START_PARTU_4X4 ........ START_PARTU_4X4 ........ START_PARTU_4X4 ........

    Of course, this being Perl, there are other ways to approach this, but these are two pretty common ones.

      Thank you haukex for your suggestion. I really appreciate your help.
Re: Print lines between multi-line regex pattern
by tybalt89 (Monsignor) on Jun 12, 2018 at 01:58 UTC

    Here's another option - read in chunks that end with your ending text.

    #!/usr/bin/perl # https://perlmonks.org/?node_id=1216375 use strict; use warnings; $/ = 'APP> END_PARTU_8X8'; while( <DATA> ) { s/.*APP> START_PARTU_8X8\s+APP> PARTITION_SPLIT\n//s or next; /(.*\n)\s*APP> END_PARTU_8X8/s and print $1; } __DATA__ APP> START_PARTU_8X8 APP> PARTITION_NONE START_PARTU_16X16 APP> END_PARTU_8X8 APP> START_PARTU_8X8 APP> PARTITION_SPLIT START_PARTU_4X4 ........ START_PARTU_4X4 ........ START_PARTU_4X4 ........ START_PARTU_4X4 ........ APP> END_PARTU_8X8 PARTU_16X4 16X16 4X16 APP> END_PARTU_8X8

    Outputs:

    START_PARTU_4X4 ........ START_PARTU_4X4 ........ START_PARTU_4X4 ........ START_PARTU_4X4 ........

    There is a conflict between your text description of what you want printed and your output. I went by the text.

      Thank you tybalt89 I chose this solution. I really appreciate your help.