Filter out an input file with a given waiver file, and output to a specific file 2.0

DespacitoPerl has asked for the wisdom of the Perl Monks concerning the following question:

Hi, i am currently writing a script to filter out an input file with a waiver file, to an output file. Input file format:

abcd123 klmn123 100.00 1000.00 -900.00 (VIOLATED)
abcd124 klmn124 100.00 1000.00 -900.00 (VIOLATED)
.
.and so on
[download]

Waiver fileformat:

klmn124,100.00,2500.00,"justifiedbyChan","Date:14/4"
.
.and so on
[download]

Ps: waiver file is a filter, menas if watever inside it, the similiar content in input file should change to output format like this:

abcd123 klmn123 100.00 1000.00 -900.00 (VIOLATED)
abcd124 klmn124 100.00 2500.00 -2400.00 (WAIVED)
[download]

my code is like this:

#! /tools/perl/5.8.8/linux/bin/perl

use strict;
use warnings;

# Set up a hash to receive the information
my %identifier = ();

# Read the violations file into the hash
open my $filter, '<', $waiver or die;
while (my $vline = <$filter>) {
    
    my ($pins2, $threshold2, $newthreshold2, $note1, $note2, $note3, $
+note4, $note5) = split /\s+/, $vline;
    #$identifier{$pins2}{threshold2} = $threshold2;
    $identifier{$pins2}{newthreshold2} = $newthreshold2;
}

#Read input file line by line
open my $input, '<', $report or die;
open my $output, '>', $result or die;
while (my $wline = <$input>){
    my ($scenario, $pins1, $threshold1, $newthreshold1, $diff, $status
+) = split /\s+/, $wline;
    print $output "$scenario    $pins1    $threshold1    $newthreshold
+1    $diff    $status\n";
    foreach my $okey (keys %identifier) {
    foreach my $skey (keys %{$identifier{$okey}}) {
    if (exists $identifier{$okey}) {
        #my $diff2 = $skey - $threshold1;
        print $output "$scenario    $pins1    $threshold1    $skey    
+$diff    (WAIVED)\n";
            }
    }
    }
}
close $filter;
close $input;
close $output;
[download]

Anyone has any ideas?

Comment on Filter out an input file with a given waiver file, and output to a specific file 2.0 Select or Download Code

Replies are listed 'Best First'.
Re: Filter out an input file with a given waiver file, and output to a specific file 2.0 by GrandFather (Saint) on Jul 13, 2017 at 09:05 UTC
What happens when you run the code and how is that different than you expect? Premature optimization is the root of all job security	[reply]
Re: Filter out an input file with a given waiver file, and output to a specific file 2.0 by choroba (Cardinal) on Jul 13, 2017 at 09:58 UTC
Why are you creating a new thread? You haven't replied to my requirements in Re: Filter out an input file with a given waiver file, and output to a specific file, and you used the same vague words again: > similiar content What do you mean by "similar"? ($q=q:Sq=~/;[c](.)(.)/;chr(-\|\|-\|5+lengthSq)`"S\|oS2"`map{chr \|+ord }map{substrSq`S_+\|`\|}3E\|-\|`7**2-3:)=~y+S\|`+$1,++print+eval$q,q,a, [download]	[reply] [d/l]
Re^2: Filter out an input file with a given waiver file, and output to a specific file 2.0 by DespacitoPerl (Acolyte) on Jul 14, 2017 at 01:43 UTC
sry, for more completed information, it should be, the output should just change the value of "new data", which is fourth column, the "difference", which is the fifth column, and the status, which is the sixth column, if the code, which is the second column, in the input file, is mentioned in waiver file, in first column. But anyway, thanks you guys enthusiatic helping, i have found the solution.	[reply]
Re: Filter out an input file with a given waiver file, and output to a specific file 2.0 (Update2) by thanos1983 (Parson) on Jul 13, 2017 at 11:23 UTC
Hello DespacitoPerl, This is one possible solution. Read more... (4 kB) What I have implemented here is a combination of eof FILEHANDLE, Access and Printing of a HASH OF HASHES and at the end to print the data join. The script is simple, you provide as input any number of files just like the input data that you provide us and any number of filter files like the filter file data that you provided, Syntax `perl test.pl in1.txt filter1.txt filter2.txt in2.txt etc...`. The order of the files or number is irrelevant. I created this script in order to make your problem more generic. As a next step I sort files and then I compare the hashes with each other. So far so good, but you are wondering why the lines are not in order although the hashes are in order. Well this is a bit tricky so I will leave this with you to solve ;). If you have more trouble let us know and I will assist more. Update: Well I read the other monks reply after answering the question, so I did not wanted to delete the answer. So at this point I will just provide you with the final solution. Read more... (4 kB) Update2: It came to my mind that the filter file is smaller in comparison to the input file so I changed the algorithm to iterate each line of the file once and compare with the input file instead of the opposite. It should save some resources in big files. Find code bellow: Read more... (4 kB) Hope this helps, BR. Seeking for Perl wisdom...on the process of learning...not there...yet!	[reply] [d/l] [select]