in reply to Grep asterisk sign in input file

Hi all, i am currently thinking of improving a script, that this script will read through an input file and a waiver.csv. Input file is a timing violations report and waiver.csv is to state which ports/pins timing violations can be waived. Basically, i am able to write the script out. But now, in the waiver file, now the pins/ports name will have wildcard * which the user no need to key in so many data inside, with the strings before the wildcard * match will do. Input file example: (first column is scenario, second is pins/ports name, third is target threshold, fourth is the result threshold, the result threshold must be lower or equal to the target threshold)

f1 c4a_123 350.00 3251.94 -2901.94 (VIOLATED) f1 c4b_123 350.00 2419.08 -2069.08 (VIOLATED) f2 c4emib_2060 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2061 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2062 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2063 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2064 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2065 250.00 2000.00 -1750.00 (VIOLATED) and so on

Waiver.csv, first column is pins/ports name, second is the target threshold value, third is adjusted target threshold value

c4a_123,350.00,3300.00,"Justification","Waived,by","Waived,Date","Appr +oved,by","Approved,date" c4emib_*,250.00,2000.00,"Justification","Waived,by","Waived,Date","App +roved,by","Approved,date"

So the expected outcome result file is like this:

f1 c4a_123 350.00 3251.94 -2901.94 (WAIVED) f1 c4b_123 350.00 2419.08 -2069.08 (VIOLATED) f2 c4emib_2060 250.00 2000.00 -1750.00 (WAIVED) f2 c4emib_2061 250.00 2000.00 -1750.00 (WAIVED) f2 c4emib_2062 250.00 2000.00 -1750.00 (WAIVED) f2 c4emib_2063 250.00 2000.00 -1750.00 (WAIVED) f2 c4emib_2064 250.00 2000.00 -1750.00 (WAIVED) f2 c4emib_2065 250.00 2000.00 -1750.00 (WAIVED)

my previous code is able to filter it out the expected output. But now since the wildcard is inserted in waiver file so the user no need to key in so many data if the pins/ports name are similar in the characters in front, so i no have the idea of (1) How my script can read the * wildcard symbol (2) and apply in the hash, to check line by line in input file for te strings that matched My code is like this:

#! /tools/perl/5.8.8/linux/bin/perl use strict; use warnings; use Data::Dumper; # Source script my $report = $ARGV[1] ; my $waiver = $ARGV[3] ; my $result = $ARGV[5] ; # Set up a hash to receive the information my %identifier = (); # Read the violations file into the hash open my $filter, '<', $waiver or die; while (my $vline = <$filter>) { next unless $vline =~ /\S/; #skip blank lines $vline =~ s/^\S+//; #trim leading space my ($pins2, $threshold2, $newthreshold2) = split /,/, $vline; $identifier{$pins2}{'threshold2'} = $threshold2; $identifier{$pins2}{'newthreshold2'} = $newthreshold2; } print Dumper \%identifier; # Read input file line by line and compare 2 files open my $input, '<', $report or die; open my $output, ">", $result or die; while (my $wline = <$input>){ my (undef,$scenario, $pins1, $threshold1, $newthreshold1, $diff, $ +status) = split /\s+/, $wline; # overwrite values if match if ($_ =~ $identifier{$pins1.}) { if ( ($threshold1 == $identifier{$pins1}{'threshold2'}) && ($newth +reshold1 <= $identifier{$pins1}{'newthreshold2'}) ) { $status = '(WAIVED)'; } } printf $output "%-44s %-24s %-8s %-8s %-8s %-10 +s\n", $scenario, $pins1, $threshold1, $newthreshold1, + $diff, $status; } close $filter; close $input; close $output;

Replies are listed 'Best First'.
Re: Wildcard in a hash
by shmem (Chancellor) on Jul 27, 2017 at 06:49 UTC
    if ($_ =~ $identifier{$pins1.}) {

    This doesn't work. {$pins1.} is a syntax error. There is no wildcard lookup in a hash.

    $vline =~ s/^\S+//; #trim leading space

    Nope. \S is non-blank.

    my (undef,$scenario, $pins1, $threshold1, $newthreshold1, $diff, $stat +us) = split /\s+/, $wline;

    Your key is ending up in $scenario here. Skip the undef.

    If you encounter an identifier with a wildcard in the waiver file, you need to rewrite that identifier as a valid perl regular expression and later match each key of the waiver hash against the current $pins1:

    #! /tools/perl/5.8.8/linux/bin/perl use strict; use warnings; use Data::Dumper; # Source script my $report = $ARGV[1] ; my $waiver = $ARGV[3] ; my $result = $ARGV[5] ; # Set up a hash to receive the information my %identifier = (); # Read the violations file into the hash open my $filter, '<', $waiver or die; while (my $vline = <$filter>) { next unless $vline =~ /\S/; #skip blank lines print $vline; $vline =~ s/^\s+//; #trim leading space my ($pins2, $threshold2, $newthreshold2) = split /,/, $vline; $pins2 =~ s/\*$/.*/; # make key into regex string if trailing * $identifier{$pins2}{'threshold2'} = $threshold2; $identifier{$pins2}{'newthreshold2'} = $newthreshold2; } close $filter; print Dumper \%identifier; # Read input file line by line and compare 2 files open my $input, '<', $report or die; open my $output, ">", $result or die; while (my $wline = <$input>){ my ($scenario, $pins1, $threshold1, $newthreshold1, $diff, $status +) = split /\s+/, $wline; # overwrite values if match for my $key(keys %identifier) { if ($pins1 =~ /^$key$/) { if ( ($threshold1 == $identifier{$key}{'threshold2'}) && ( +$newthreshold1 <= $identifier{$key}{'newthreshold2'}) ) { $status = '(WAIVED)'; } last; # skip remaining keys of for()-loop } } printf $output "%-44s %-24s %-8s %-8s %-8s %-10 +s\n", $scenario, $pins1, $threshold1, $newthreshold1, + $diff, $status; } close $input; close $output;

    Close your filehandles as soon as you are done with them.

    perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: Wildcard in a hash
by tybalt89 (Monsignor) on Jul 27, 2017 at 15:09 UTC

    It works for your test case :)

    #!/usr/bin/perl # http://perlmonks.org/?node_id=1196126 use strict; use warnings; my (%waiverhash, @waiverstar); open my $w, '<', \<<END or die; c4a_123,350.00,3300.00,"Justification","Waived,by","Waived,Date","Appr +oved,by","Approved,date" c4emib_*,250.00,2000.00,"Justification","Waived,by","Waived,Date","App +roved,by","Approved,date" END while(<$w>) { my ($name, $target, $adhust) = split /,/; if( $name =~ s/\*/.*/g ) { push @waiverstar, [ qr/^$name$/, $target, $adhust ]; } else { $waiverhash{$name} = [ $target, $adhust ]; } } close $w; while(<DATA>) { my ($name, $target, $adjust) = (split)[1 .. 3]; if( $waiverhash{$name} ) { $target == $waiverhash{$name}[0] and $adjust <= $waiverhash{$name} +[1] and s/VIOLATED/WAIVED/; } else { for my $star ( @waiverstar ) { if( $name =~ $star->[0] ) { $target == $star->[1] and $adjust <= $star->[2] and s/VIOLATED/WAIVED/; last; } } } print; } __DATA__ f1 c4a_123 350.00 3251.94 -2901.94 (VIOLATED) f1 c4b_123 350.00 2419.08 -2069.08 (VIOLATED) f2 c4emib_2060 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2061 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2062 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2063 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2064 250.00 2000.00 -1750.00 (VIOLATED) f2 c4emib_2065 250.00 2000.00 -1750.00 (VIOLATED)
Re: Wildcard in a hash
by poj (Abbot) on Jul 27, 2017 at 06:14 UTC
      Previous may be a bit messy, this one more details and complete
Re: Wildcard in a hash
by Anonymous Monk on Jul 27, 2017 at 02:56 UTC
    How would you do it if you had to use pencil and paper and no computer? Please describe the steps
      Steps (1) type command to source the script, perl <path of the script> -input <path of the input> -waiver <path of the waiver> -output <path of the output> (2) Script will be activated, and read through input file and waiver.csv, to generate a filtered input to output path (3) Data in waiver.csv will be stored in hash, with the ports/pins name and the threshold target are the both of the strings which can be used to check if any matched in input file, if yes, then check if the adjusted threshold value lower or equal to the target threshold in input file, if yes, status change from violated to waived (4) output file will be the same as input file, unless the conditions mentioned in (3) are fulfilled.

        Steps (1) type command to source the script, perl <path of the script> -input <path of the input> -waiver <path of the waiver> -output <path of the output> (2) Script will be activated, and read through input file and waiver.csv, to generate a filtered input to output path (3) Data in waiver.csv will be stored in hash, with the ports/pins name and the threshold target are the both of the strings which can be used to check if any matched in input file, if yes, then check if the adjusted threshold value lower or equal to the target threshold in input file, if yes, status change from violated to waived (4) output file will be the same as input file, unless the conditions mentioned in (3) are fulfilled.

        I think i wasn't clear enough.

        If I give you a piece of paper with three four lines of input on it, what would you do to create the final list on another piece of paper? What is every single step you would take?

         

        Grab some pen/paper and actually physically perform this task, then write down every action, every step, leave nothing out, and post those steps here

Re: Wildcard in a hash
by Anonymous Monk on Jul 27, 2017 at 14:19 UTC

    Your question is very vague. It's not even clear what domain you're working in. Just for the interest of the other Monks, I'm going to guess that "emi" in your example file stands for electromagnetic interference, and that "ports/pins" are locations where you're applying and/or measuring some kind of test signal in a piece of hardware.

    You talk about asterisks in your question, but your example doesn't contain any. Your other post was more understandable in that regard. But what if your waiver file contains these two lines:

    c4*_123
    c4a_*

    If the input is c4a_123, which pattern should match? The first one? In that case, you need to remember the order of the waiver file, so you don't want to use a hash. See choroba's response to your original post.

      > but your example doesn't contain any.

      It does:

      c4emib_*,250.00,2000.00,"Justification","Waived,by","Waived,Date","App +roved,by","Approved,date" ^ | here

      ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,
      The wildcard can be anywhere, Let say if meet case like c4*_123, how i going to write?

        See perlre. In perl regular expression syntax, the asterisc * is a quantifier which means "zero or more times" .

        So, in c4*_123 the asterisc would mean "zero or more times 4". To make the * into a wildcard, preceed it with a period, which means "any character". In my previous answer in this thread, there was the line

        $pins2 =~ s/\*$/.*/; # make key into regex string if trailing *

        The dollar sign after the escaped asterisc is an end anchor, so this matches only an asterisc at the end of the string. To match the asterisc anywhere, just remove the end anchor:

        $pins2 =~ s/\*/.*/; # make key into regex string if containing *

        If your keys can contain more than one asterisc, add the /g modifier (global) to the s///:

        $pins2 =~ s/\*/.*/g; # make key into regex string if containing *
        perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'