Re: Print lines based on matching words

Your description of the problem makes almost no sense at all and your code makes even less sense. However, maybe what you want is to skip lines containing the word "remark"? If so then:

#!/usr/bin/perl
use strict;
use warnings;

# print the next word followed by object-group
while (<DATA>) {
    print if ! /\bremark\b/;
}
[download]

should get you started. However, that doesn't jell with the output you are looking for so maybe the access-list lines are key lines containing key words used to extract blocks of data. So lets play with that idea:

#!/usr/bin/perl
use strict;
use warnings;
use 5.010;

my @lists;
my %group;
my $lastGroup;

while (defined(my $line = <DATA>)) {
    next if $line =~ /^access-list .*\bremark\b/;

    given ($line) {
        when (/^access-list/) {
            my @keys = $line =~ /([A-Z]+[-_][A-Z]+)/g;
            push @lists, {groups => \@keys, line => $line};
        }
        when (/object-group\s+(\S+)\s+(\S+)/) {
            $group{$2}{line} = $line;
            $group{$2}{data} = [];
            $lastGroup       = $2;
        }
        default {
            push @{$group{$lastGroup}{data}},   $line;
            push @{$group{$lastGroup}{groups}}, $1
                if $line =~ /^\s+\S+\s+(\S+)$/;
        }
    }
}

for my $list (@lists) {
    print $list->{line};
    printGroups($list->{groups}, %group);
    print "\n";
}


sub printGroups {
    my ($groups, %groupData) = @_;

    for my $group (sort @$groups) {
        next if !exists $groupData{$group};

        print $groupData{$group}{line};
        print @{$groupData{$group}{data}} if $groupData{$group}{data};
        printGroups($groupData{$group}{groups}, %groupData)
            if exists $groupData{$group}{groups};
    }
}

__DATA__
access-list INSIDE_IN remark Web Users To Web Server
access-list INSIDE_IN extended permit tcp object-group WEB-CLIENT obje
+ct-group WEB-SERVER object-group WEB_TCP
access-list INSIDE_IN remark EMAIL To EMAIL Server
access-list INSIDE_IN extended permit tcp object-group EMAIL-CLIENT ob
+ject-group EMAIL-SERVER object-group SMTP_TCP
object-group network PC1_1st
 network-object host 10.1.1.11
object-group network PC2_1st
 network-object host 10.1.1.12
object-group network WEB-CLIENT
 group-object PC1_1st
 group-object PC2_1st
object-group network WEB-SERVER
 network-object host 10.1.1.5
object-group service WEB_TCP tcp
 port-object eq www
 port-object eq https
object-group network EMAIL-CLIENT
 group-object PC1_1st
 group-object PC2_1st
object-group network EMAIL-SERVER
 network-object host 10.1.1.6
object-group service SMTP_TCP tcp
 port-object eq SMTP
[download]

prints:

access-list INSIDE_IN extended permit tcp object-group WEB-CLIENT obje
+ct-group WEB-SERVER object-group WEB_TCP
object-group network WEB-CLIENT
 group-object PC1_1st
 group-object PC2_1st
object-group network PC1_1st
 network-object host 10.1.1.11
object-group network PC2_1st
 network-object host 10.1.1.12
object-group network WEB-SERVER
 network-object host 10.1.1.5
object-group service WEB_TCP tcp
 port-object eq www
 port-object eq https

access-list INSIDE_IN extended permit tcp object-group EMAIL-CLIENT ob
+ject-group EMAIL-SERVER object-group SMTP_TCP
object-group network EMAIL-CLIENT
 group-object PC1_1st
 group-object PC2_1st
object-group network PC1_1st
 network-object host 10.1.1.11
object-group network PC2_1st
 network-object host 10.1.1.12
object-group network EMAIL-SERVER
 network-object host 10.1.1.6
object-group service SMTP_TCP tcp
 port-object eq SMTP
[download]

which looks pretty much like your required output.

Perl is the programming world's equivalent of English

Comment on Re: Print lines based on matching words Select or Download Code

Replies are listed 'Best First'.
Re^2: Print lines based on matching words by ArifS (Beadle) on Dec 23, 2014 at 19:01 UTC
oops never mind. I had to add in the following- `/([A-Z\d]+[-_][A-Z\d]+)/ig;` Working as expected. Thanks again!	[reply] [d/l]
Re^2: Print lines based on matching words by ArifS (Beadle) on Dec 23, 2014 at 13:53 UTC
That's exactly what I was looking for. Thank you very much!	[reply]
Re^2: Print lines based on matching words by ArifS (Beadle) on Dec 23, 2014 at 14:32 UTC
I tried the script with the following for something different & seems like it's not giving the desired output- __DATA__ access-list INSIDE_IN remark Web Users To Web Server access-list INSIDE_IN extended permit tcp object-group WEB-CLIENT obje +ct-group WEB-SERVER object-group WEB_TCP access-list INSIDE_IN remark EMAIL To EMAIL Server access-list INSIDE_IN extended permit tcp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_TCP access-list INSIDE_IN remark SRVR to Client access-list INSIDE_IN extended permit tcp object-group MYSRVR_2nd obje +ct-group MYCLIENTS_1st object-group WEB_TCP object-group network PC1_1st network-object host 10.1.1.11 object-group network PC2_1st network-object host 10.1.1.12 object-group network WEB-CLIENT group-object PC1_1st group-object PC2_1st object-group network WEB-SERVER network-object host 10.1.1.5 object-group service WEB_TCP tcp port-object eq www port-object eq https object-group network EMAIL-CLIENT group-object PC1_1st group-object PC2_1st object-group network EMAIL-SERVER network-object host 10.1.1.6 object-group service SMTP_TCP tcp port-object eq SMTP object-group network MYCLIENTS_1st network-object host 10.1.2.1 network-object host 10.1.2.2 network-object host 10.1.2.3 network-object host 10.1.2.4 object-group network MYSRVR_2nd network-object host 10.1.1.4 network-object host 10.1.1.3 network-object host 10.1.1.31 object-group service WEB_TCP tcp port-object eq www port-object eq https [download] PRINTS: access-list INSIDE_IN extended permit tcp object-group WEB-CLIENT obje +ct-group WEB-SERVER object-group WEB_TCP object-group network WEB-CLIENT group-object PC1_1st group-object PC2_1st object-group network PC1_1st network-object host 10.1.1.11 object-group network PC2_1st network-object host 10.1.1.12 object-group network WEB-SERVER network-object host 10.1.1.5 object-group service WEB_TCP tcp port-object eq www port-object eq https print "\n"; access-list INSIDE_IN extended permit tcp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_TCP object-group network EMAIL-CLIENT group-object PC1_1st group-object PC2_1st object-group network PC1_1st network-object host 10.1.1.11 object-group network PC2_1st network-object host 10.1.1.12 object-group network EMAIL-SERVER network-object host 10.1.1.6 object-group service SMTP_TCP tcp port-object eq SMTP access-list INSIDE_IN extended permit tcp object-group MYSRVR_2nd obje +ct-group MYCLIENTS_1st object-group WEB_TCP object-group service WEB_TCP tcp port-object eq www port-object eq https print "\n"; [download] INSTEAD OF- access-list INSIDE_IN extended permit tcp object-group WEB-CLIENT obje +ct-group WEB-SERVER object-group WEB_TCP object-group network WEB-CLIENT group-object PC1_1st group-object PC2_1st object-group network PC1_1st network-object host 10.1.1.11 object-group network PC2_1st network-object host 10.1.1.12 object-group network WEB-SERVER network-object host 10.1.1.5 object-group service WEB_TCP tcp port-object eq www port-object eq https access-list INSIDE_IN extended permit tcp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_TCP object-group network EMAIL-CLIENT group-object PC1_1st group-object PC2_1st object-group network PC1_1st network-object host 10.1.1.11 object-group network PC2_1st network-object host 10.1.1.12 object-group network EMAIL-SERVER network-object host 10.1.1.6 object-group service SMTP_TCP tcp port-object eq SMTP access-list INSIDE_IN extended permit tcp object-group MYSRVR_2nd obje +ct-group MYCLIENTS_1st object-group WEB_TCP object-group network MYSRVR_2nd network-object host 10.1.1.4 network-object host 10.1.1.3 network-object host 10.1.1.31 object-group network MYCLIENTS_1st network-object host 10.1.2.1 network-object host 10.1.2.2 network-object host 10.1.2.3 network-object host 10.1.2.4 object-group service WEB_TCP tcp port-object eq www port-object eq https [download] Please let me know.	[reply] [d/l] [select]
Re^2: Print lines based on matching words by ArifS (Beadle) on Dec 24, 2014 at 15:25 UTC
From the script above - `my @keys = $line =~ /([A-Z\d]+[-_][A-Z\d]+)/ig;` [download] Seems to match - WEB-CLIENT, SMTP_TCP etc. But, how can I also match - MYCLIENTS-IP_1st, MYSRVR_IP-S_2nd etc. __DATA__ access-list INSIDE_IN remark Web Users To Web Server access-list INSIDE_IN extended permit tcp object-group WEB-CLIENT obje +ct-group WEB-SERVER object-group WEB_TCP access-list INSIDE_IN remark EMAIL To EMAIL Server access-list INSIDE_IN extended permit tcp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_TCP access-list INSIDE_IN remark SRVR to Client access-list INSIDE_IN extended permit tcp object-group MYSRVR_IP-S_2nd + object-group MYCLIENTS-IP_1st object-group WEB_TCP object-group network PC1_1st network-object host 10.1.1.11 object-group network PC2_1st network-object host 10.1.1.12 object-group network WEB-CLIENT group-object PC1_1st group-object PC2_1st object-group network WEB-SERVER network-object host 10.1.1.5 object-group service WEB_TCP tcp port-object eq www port-object eq https object-group network EMAIL-CLIENT group-object PC1_1st group-object PC2_1st object-group network EMAIL-SERVER network-object host 10.1.1.6 object-group service SMTP_TCP tcp port-object eq SMTP object-group network MYCLIENTS-IP_1st network-object host 10.1.2.1 network-object host 10.1.2.2 network-object host 10.1.2.3 network-object host 10.1.2.4 object-group network MYSRVR_IP-S_2nd network-object host 10.1.1.4 network-object host 10.1.1.3 network-object host 10.1.1.31 object-group service WEB_TCP tcp port-object eq www port-object eq https [download] Please let me know.	[reply] [d/l] [select]
Re^3: Print lines based on matching words by AnomalousMonk (Archbishop) on Dec 26, 2014 at 03:12 UTC
Here's a way to match/extract ordinal substrings. Note this won't match an improper ordinal like `'1th'` or `'4rd'`. (Update: It's also case-sensitive.) Tested on Windows 7 under ActiveState 5.8.9 and Strawberry 5.14.4. Read more... (2 kB)	[reply] [d/l] [select]
Re^4: Print lines based on matching words by ArifS (Beadle) on Dec 26, 2014 at 13:43 UTC
Thank you for your reply. That's a nice way to match the rest. I however, using the following as it seems to fit my needs- `while (defined(my $line = <$fh>)) { next if $line =~ /^access-list .*\bremark\b/; given ($line) { when (/^access-list/) { # take the next word followed by object-group /object-group\s/; my @after = split(/\s/,$'); my @keys = @after; push @lists, {groups => \@keys, line => $line}; } ............... } }` [download]	[reply] [d/l]
Re^2: Print lines based on matching words by ArifS (Beadle) on Jul 07, 2016 at 11:52 UTC
I am trying to skip a Section if it already Exists (above). However, the following script add a new line "Exists", but doesn't skip the Section. use strict; use warnings; use 5.010; my @lists; my %group; my $lastGroup; while (defined(my $line = <DATA>)) { next if $line =~ /^access-list .*\bremark\b/; given ($line) { when (/^access-list/) { my @keys = $line =~ /([A-Z]+[-_][A-Z]+)/g; push @lists, {groups => \@keys, line => $line}; } when (/object-group\s+(\S+)\s+(\S+)/) { $group{$2}{line} = $line; $group{$2}{data} = []; $lastGroup = $2; } default { push @{$group{$lastGroup}{data}}, $line; push @{$group{$lastGroup}{groups}}, $1 if $line =~ /^\s+\S+\s+(\S+)$/; } } } for my $list (@lists) { print $list->{line}; printGroups($list->{groups}, %group); print "\n"; } sub printGroups { my ($groups, %groupData) = @_; for my $group (sort @$groups) { ############ Skip if Exists above ##### print "Exists above\n" if exists $groupData{$group}; ############ end Skip ##### next if !exists $groupData{$group}; print $groupData{$group}{line}; print @{$groupData{$group}{data}} if $groupData{$group}{data}; printGroups($groupData{$group}{groups}, %groupData) if Existss $groupData{$group}{groups}; } } __DATA__ access-list INSIDE_IN remark EMAIL To EMAIL Server access-list INSIDE_IN extended permit tcp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_TCP access-list INSIDE_IN extended permit udp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_UDP object-group network PC1_1st network-object host 10.1.1.11 object-group network PC2_1st network-object host 10.1.1.12 object-group network EMAIL-CLIENT group-object PC1_1st group-object PC2_1st object-group network EMAIL-SERVER network-object host 10.1.1.6 object-group service SMTP_TCP tcp port-object eq SMTP object-group service SMTP_UDP udp port-object eq SMTP [download] Current OUTPUT access-list INSIDE_IN extended permit tcp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_TCP Exists above object-group network EMAIL-CLIENT group-object PC1_1st group-object PC2_1st Exists above object-group network PC1_1st network-object host 10.1.1.11 Exists above object-group network PC2_1st network-object host 10.1.1.12 Exists above object-group network EMAIL-SERVER network-object host 10.1.1.6 Exists above object-group service SMTP_TCP tcp port-object eq SMTP access-list INSIDE_IN extended permit udp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_UDP Exists above object-group network EMAIL-CLIENT group-object PC1_1st group-object PC2_1st Exists above object-group network PC1_1st network-object host 10.1.1.11 Exists above object-group network PC2_1st network-object host 10.1.1.12 Exists above object-group network EMAIL-SERVER network-object host 10.1.1.6 Exists above object-group service SMTP_UDP udp port-object eq SMTP [download] OUTPUT needed access-list INSIDE_IN extended permit tcp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_TCP object-group network EMAIL-CLIENT group-object PC1_1st group-object PC2_1st object-group network PC1_1st network-object host 10.1.1.11 object-group network PC2_1st network-object host 10.1.1.12 object-group network EMAIL-SERVER network-object host 10.1.1.6 object-group service SMTP_TCP tcp port-object eq SMTP access-list INSIDE_IN extended permit udp object-group EMAIL-CLIENT ob +ject-group EMAIL-SERVER object-group SMTP_UDP Exists above Exists above Exists above Exists above object-group service SMTP_UDP udp port-object eq SMTP [download] Please let me know.	[reply] [d/l] [select]
Re^2: Print lines based on matching words by ArifS (Beadle) on Jul 11, 2016 at 13:10 UTC
GrandFather, Your script is working just fine. I however am trying to take out the duplicate lines if exists above. I tried as following but giving me a hash error- Original- `for my $list (@lists) { print $list->{line}; printGroups($list->{groups}, %group); print "\n"; }` [download] with `sub uniq { my %seen; grep !$seen{$_}++, @_; } for my $list (@lists) { my @filtered = uniq(@lists); printGroups(@filtered->{groups}, %group); # prints acl lines AFTER object-groups print $list->{line}; print "line 54: ACL line\n\n"; }` [download] Please let me know.	[reply] [d/l] [select]
Re^3: Print lines based on matching words by GrandFather (Saint) on Jul 11, 2016 at 21:09 UTC
Add a minimum number of lines of code to initialize @lines with data built into the test code and ask again. I have no way of telling what the contents of @lines is, but it seems most likely it is an array of hash refs. Premature optimization is the root of all job security	[reply]