Comparing a Hash key with a variable (if statement)

dkhalfe has asked for the wisdom of the Perl Monks concerning the following question:

Hi all, I have asked numerous questions regarding my code but have pasted them in bits and pieces. I have not found a solution to my problem, so this time around I am going to post the entirety of my code. Please do not critique it too harshly, I am simply looking for a fix to my dilemna (in other words, I do not want to change a lot of my code).

Gameplan: I am going to read in a filter_file and an input file.

Filter file will be tab-delim formatted as such:

column relationship value num_or_string filter_or_append order
a      <=           0.3   num           filter           1
b      eq           abc   string        append           3
c      <=           0.3   num           filter           2
[download]

Input file will be tab-delim formatted as such:

a     b     c
0.2   abc   0.3
0.1   abd   0.3
0.4   abe   0.2
0.1   abc   0.5
0.7   abt   0.7
0.1   abd   0.8
[download]

Here is my code that reads in the filter file, creates an array of the line by splitting it by \t, makes a reference of it and pushes the reference back to the filter array (Also sorts filter file based on 'order' column):

#!/usr/bin/perl
use warnings;
use strict;
use Data::Dumper;

#link to input file from command line - 2 arguments or error displayed
+; Set to respective variables
@ARGV == 2 or die "Invalid number or arguments. Please re-run program 
+and supply as arguments: \n1) the filter file and \n2) the input file
+.";
my ($filter_file, $input_file) = @ARGV;

open (FILTER,"$filter_file");
my @filter;

<FILTER>; # read one line from the file [HEADERS]

while (<FILTER>)
{ # read other lines
    chomp; # remove "\n" from the end of the line
    push @filter,[(split /\t/,$_)]; # create an array of the line by s
+plitting it by <TAB>, make a reference of it and push the reference t
+o the @filter array
    close $filter_file if eof;
}


@filter = sort { $a->[5] <=> $b->[5] } @filter; # sort the array 

#PRINTS REFERENCES TO ARRAY FROM LEAST TO GREATEST REGARDING 'ORDER' C
+OLUMN
print Dumper \@filter; #### For Debugging Purposes
my $num_elements = (@filter-1); #Number of elements in array - will be
+ used to control loop when comparisons take place later in code
[download]

the next part of my code focuses on reading in the input file. Column headers are stored as keys; each key can be called to retrieve the data in the specified column, hashes are put into an array and each hash is an array row.

open (IN, "$input_file") or die "Cannot open file: $!";

#variables
my $x=0;
my @keys;
my @holder;
my @array_hash;
my $blank = "-";
my $numerical_value = "num";
my $string_value = "string";
my $filter_data = "filter";
my $append = "append";

open (OUTFILE, ">>OUTPUT_$input_file") or die "Cannot create an output
+ file: $!";

while(my $line = <IN>)
{
        chomp($line);
        $x++;
        if ($x==1)
        {
                print OUTFILE $line, "\n";
                (@keys)=split(/\t/, $line);
        }
        else
        {
                my $y=0;
                (@holder) = split(/\t/, $line);
                my %hash;
                for my $column (@holder)
                {
                        $hash{$keys[$y]}=$column;
                        $y++;
                }
[download]

Ok, so the above pasted code works. I can manually make comparisons with using the eval function as such: (the output I want is printed to an output file, perfect)

if ((eval "$hash{$filter[0]->[0]} $filter[0]->[1] $filter[0]->[2]") &&
+ (eval "'$hash{$filter[1]->[0]}' $filter[1]->[1] '$filter[1]->[2]'") 
+&& (eval "$hash{$filter[2]->[0]} $filter[2]->[1] $filter[2]->[2]"))
{
    print OUTFILE $line, "\n";
}
[download]

Let me explain what is going on in this step: If the hash of column a evaluates to less than or equal to the value specified in the filter file (.03), continue. This part works. Then, if the hash of column b evaluates to 'equal' the value specified in the filter file (abc), continue, etc...

However, I want to format this comparison step in a for loop. For example

    for(my $c = 0; $c <= $num_elements; $c++)
    {
        if($hash{$filter[$c]->[4]} eq $filter_data)
        {
            if (exists($hash{$filter[$c]->[0]}))
            {
                if($hash{$filter[$c]->[3]} eq $numerical_value)
                {
#                    #process filtering for numerical value
                    if ((eval "$hash{$filter[$c]->[0]} $filter[$c]->[1
+] $filter[$c]->[2]"))
                    {
                        print OUTFILE $line, "\n";
                    }
                }
            }
            else
            {
                print "COLUMN NAME DOES NOT EXIST. CHECK FILTER FILE F
+OR ANY POSSIBLE ERRORS AND RE-RUN PROGRAM. PROGRAM WILL NOW TERMINATE
+.", "\n";
                exit 0;
            }
        }
    }
[download]

When I run the above for loop and comment out my previously stated eval step, only columnd headers are printed to the output file. Where am I going wrong? Thank you for any help!

EDIT: Is the format of my loops/hash keys/references to arrays correct in the code above?

Comment on Comparing a Hash key with a variable (if statement) Select or Download Code

Replies are listed 'Best First'.
Re: Comparing a Hash key with a variable (if statement) by Corion (Patriarch) on Jul 26, 2012 at 16:35 UTC
Have you thought of adding `else` statements to all your `if` statements? If you do that, and print there that some element does not match some condition, then maybe you will find why your loop gets skipped. Also consider printing the strings that you hand to `eval` and also check that your `eval` statement does not raise any errors. See perlvar for `$@`. Of course, if you provided a short, self-contained valid program instead of pasted-together snippets that provide a key-hole view of your program, it would be far easier for me to try and replicate your problem. I recommend hard-coding values instead of reading those values from external files. Please try to make your code 20 or 25 lines maximum.	[reply] [d/l] [select]
Re: Comparing a Hash key with a variable (if statement) by ww (Archbishop) on Jul 26, 2012 at 19:40 UTC
I empathize with your remark that "Please do not critique it too harshly, I am simply looking for a fix to my dilemna (in other words, I do not want to change a lot of my code). " Truly. Been there; done that. But, it's not necessarily an attitude that serves your best interests. At worst, if someone suggests an alternate to some or all of your script, you can take a look at how that works, and -- at least sometimes -- use your new knowledge to see what you need to do to make your approach work. Sure, some of the tips will wander far off the path you're beating, but if none of the replies you get, here in the Monastery, comes close to your approach, you may be getting an indication that your code if flawed, not just in detail, but fundamentally. And, at best, as suggested above, you'll probably learn something, if you pay attention to the advice from the Monks, no matter how unpalatable it may be.	[reply]
Re: Comparing a Hash key with a variable (if statement) by GrandFather (Saint) on Jul 26, 2012 at 23:22 UTC
There are a raft of techniques that clean your code up. The first is to throw the CSV file parsing over to a tool built for the job (Text::CSV). After that tossing the eval stuff aside and using a match function cleans up the rest of the code something wonderful. Consider: #!/usr/bin/perl use warnings; use strict; use Text::CSV; use 5.010; my $ops = <<OPS; column relationship value num_or_string filter_or_append + order a <= 0.3 num filter 1 b eq abc string append 3 c <= 0.3 num filter 2 OPS my $csv = Text::CSV->new ({sep_char => "\t"}); open my $opsIn, '<', \$ops or die "Cannot open file: $!"; $csv->column_names($csv->getline($opsIn)); my @ops = @{$csv->getline_hr_all($opsIn)}; close $opsIn; my %filtColumns = map {$_->{column} => 1} @ops; my @inColumns = @{$csv->getline(DATA)}; if (grep {! exists $filtColumns{$_}} @inColumns) { die <<DIE; COLUMN NAME DOES NOT EXIST. CHECK FILTER FILE FOR ANY POSSIBLE ERRORS +AND RE-RUN PROGRAM. PROGRAM WILL NOW TERMINATE. DIE exit 0; } $csv->column_names(@inColumns); $csv->combine(@inColumns); print $csv->string (), "\n"; while (my $line = $csv->getline_hr(DATA)) { next if grep {! match($line, $_)} @ops; $csv->combine(@{$line}{@inColumns}); print $csv->string (), "\n"; } sub match { my ($line, $filter) = @_; my %fields = %$filter; my $value = $line->{$fields{column}}; given($fields{relationship}) { when ('<=') {return $value <= $fields{value}} when ('eq') {return $value eq $fields{value}} default { die "match can't handle $fields{relationship} operator\n"; } } } __DATA__ a b c 0.2 abc 0.3 0.1 abd 0.3 0.4 abe 0.2 0.1 abc 0.5 0.7 abt 0.7 0.1 abd 0.8 [download] Prints: `a b c 0.2 abc 0.3` [download] No doubt I've missed the significance of some of your code (I've not followed along on your whole journey to this point), but the sample code above should give you a good starting point for further development. True laziness is hard work	[reply] [d/l] [select]
Re: Comparing a Hash key with a variable (if statement) by aitap (Curate) on Jul 26, 2012 at 17:37 UTC
Try running your code in debugger (`perl -d`), setting a breakpoint (`b line_number`) on the line which contains your `eval`, running to this line (`c`), making one step (`n`) and checking the value of `$@`. It is possible that some data from your input file form a syntaxically incorrect Perl expression, which causes `eval` to fail. Perhaps it will be safer to write a `sub` which checks the value of `$filter[$c]->[1]` and makes comparsions based on some `if (...) { ... } elsif (...) {...} ...` code. Sorry if my advice was wrong.	[reply] [d/l] [select]
Re: Comparing a Hash key with a variable (if statement) by choroba (Cardinal) on Jul 27, 2012 at 00:03 UTC
Another attempt to get you started. Your specification is not detailed enough, so you might need to tweak it. For example, I assumed 'filter' means do not output the line if the condition is met, while 'append' means do output. Also, I did not use the 'num_or_string', because the relation seems to indicate the types. Read more... (2 kB) Nothing gets printed for your sample input (I did not know what to do if no filter nor append matched). I had to add the following line to it to get any output: `0.7 abc 0.9` [download] Both a and c are big enough not to be filtered and b equals the string.	[reply] [d/l] [select]