Frits has asked for the wisdom of the Perl Monks concerning the following question:

Dear All, I'm not a very strong perl programmer but need to grep data from a large file where I have multiple outputs. Used the following example in ggrep to get my correct output:

 cat file.large | ggrep country | ggrep city1 | ggrep street1 | ggrep -v name1

Horrilbe, I know.... but need to get the value of name2 only as output. File text looks sort of:

Country City Street1 number1 name1 Country City1 Street1 number1 name2 Country City1 Street1 number2 name3 Country City1 Street2 number1 name4 Country City1 Street2 number1 name5

The output of the info in the 2nd line and going into $myoutput1 as:

Country City Street1 number1 name2

The last output required is "$thenameis" will just be name2 from the file. But this can be done with

$thenameis = $myoutput1[4]; ##This should be correct..

PS this grep will be in a foreach loop. Any ideas.. thank you

Replies are listed 'Best First'.
Re: Need to multi grep and grep exclude in perl.
by Discipulus (Canon) on Feb 23, 2016 at 08:53 UTC
    Hello Frits

    I'm not sure to have understood your question and there is not perl code to check, also the data and the output and what i understood of requirements does not collimates (name2 and name3 meet both the expression) anyway i'd start with something like (i skip name3 for what said above)

    use strict; use warnings; while (<DATA>){ print if $_ =~/country\s+city1\s+street1/i and $_ !~/name3/ ; } __DATA__ Country City Street1 number1 name1 Country City1 Street1 number1 name2 Country City1 Street1 number2 name3 Country City1 Street2 number1 name4 Country City1 Street2 number1 name5 #OUTPUT Country City1 Street1 number1 name2

    PS if you need to have just the name you can or split the string and check every item or capture the name in the first regex and print just this like in:

    print "$1\n" if $_ =~/country\s+city1\s+street1\s+\S+\s+(\S+)$/i and $_ !~/name3$/ ;

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
Re: Need to multi grep and grep exclude in perl.
by kcott (Archbishop) on Feb 23, 2016 at 15:45 UTC

    G'day Frits,

    I'm familiar with grep, egrep and fgrep but not ggrep. Please provide a link to documentation or a brief description of how it differs from grep.

    Here's how you can do the "multi grep and grep exclude" part:

    grep { /Country/ && /City1/ && /Street1/ && ! /name1/ } ...
    "this grep will be in a foreach loop"

    Why? Please provide some code to show what you're doing.

    As already stated by others, your requirements are unclear. Here's my best guess as to the type of thing you want.

    #!/usr/bin/env perl -l use strict; use warnings; while (<DATA>) { if ((my $result) = grep { /Country/ && /City1/ && /Street1/ && ! / +name1/ } $_) { print +(split ' ', $result)[4]; last; } } __DATA__ Country City Street1 number1 name1 Country City1 Street1 number1 name2 Country City1 Street1 number2 name3 Country City1 Street2 number1 name4 Country City1 Street2 number1 name5

    Output:

    name2

    — Ken

Re: Need to multi grep and grep exclude in perl.
by Random_Walk (Prior) on Feb 23, 2016 at 08:55 UTC

    cat file.large | ggrep country | ggrep city1 | ggrep street1 | ggrep -v name1

    Horrilbe, I know.... but need to get the value of name2 only as output. File text looks sort of:

    So do you know the value of name1 in advance to be removing it from the output, or do you want to discard the first name and display the second, for each country/city/street/number match?

    Update

    Here is some code that prints the second name at an address, ignoring the first name, and any third, fourth,...

    use strict; use warnings; my %seen; # here we note addresses we saw already while (<DATA>){ my ($country, $city, $street, $number, $name) = split; next unless ++$seen{$country}{$city}{$street}{$number} == 2; print "Second name at $city, $country, $street, $number is $name\n +"; } __DATA__ Country City1 Street1 number1 name1 Country City1 Street1 number1 name2 Country City1 Street1 number2 name3 Country City1 Street2 number1 name4 Country City1 Street2 number1 name5

    Cheers,
    R.

    Pereant, qui ante nos nostra dixerunt!
Re: Need to multi grep and grep exclude in perl.
by perlfan (Parson) on Feb 23, 2016 at 17:10 UTC
    You appear to be doing a series of set operations. If you break down what your sets are and what operations you're needing, there are a number of modules that will allow you to easily, extensibly, and clearly operate over these sets

    However, I'd recommend you use DBD::SQLite to throw the data into SQLite, then use it to query the db for the information you want.

    An interesting reference I always seem to turn to in times of set operations in shell is, http://www.catonmat.net/blog/set-operations-in-unix-shell.

      very useful link. Thanks!
Re: Need to multi grep and grep exclude in perl.
by FreeBeerReekingMonk (Deacon) on Feb 23, 2016 at 18:39 UTC

    Maybe this code can give you ideas for your problem:

    #!/usr/bin/perl use strict; use warnings; my @GREPLIST = grep {!/^[\+\-\~]/} @ARGV; my @SKIPLIST = grep {s/^~//} @ARGV; my $DEBUG = grep {/^-d/} @ARGV; my $IGNORECASE = grep {/^-i/} @ARGV; (my @POSITIONS) = grep {s/^\+//} @ARGV; # add this to regexp with ignore case if($IGNORECASE){ grep { s/^/(?i)/ } @GREPLIST; } print "# POSITIONS=@POSITIONS;IGNORECASE=$IGNORECASE;GREPLIST=@GREPLIS +T;SKIPLIST=@SKIPLIST\n" if $DEBUG; # arrays start with zero, so decrease field numbers by 1 --$_ for @POSITIONS; LINE: while(<STDIN>){ for my $re (@GREPLIST){ print "# $re =~ $_" if ($DEBUG); next LINE unless m/$re/; } for my $re (@SKIPLIST){ print "# $re !~ $_" if ($DEBUG); next LINE if m/$re/; } chomp; if(@POSITIONS){ # print all fields in the requested order print join(" ", +(split /\s+/)[@POSITIONS]); }else{ # just print the whole line print } print "\n"; }



    cat ~/a | ./gggrep.pl country city1 street1 ~name1 -i +5 name2 name3 cat ~/a | ./gggrep.pl -i country city1 street1 ~name1 +2 +5 City1 name2 City1 name3

Re: Need to multi grep and grep exclude in perl.
by Anonymous Monk on Feb 24, 2016 at 11:12 UTC

    Got it working now.

     grep { /Country/ && /City1/ && /Street1/ && ! /name1/ }

    Still wondering why my first grep (all most similar as above) in perl was not working the first time. But with the provided grep all was working to an sertain extend. Detected during the loop output that not all in _DATA_ was always correct (massive flat file) ...lol..., causing in some occations the array to be empty with the exclude. Therefore added an additional step in the grep options within the foreach loop.

    @dataset4 = grep ( !/$name/, @dataset3); if (@dataset4 eq '' || @dataset4 eq '0') { $outputmove = 'NONE'; last; } @dataset5 = split ( / +/, $dataset4[0] ); $outputmove = $dataset5[0];

    Will move my data coming weekend to MySQL... thx to all whom replied.

      PS:..extra thx to FreeBeerReekingMonk & kcott this was helpful for me to also find the part what was going wrong with my output.