Create a Hash Dynamically and add the values of same key

rahulruns has asked for the wisdom of the Perl Monks concerning the following question:

I am writing MAP REDUCE in PERL SCRIPT. I have written a MAP that outputs Key Value pair and I am writung a REDUCE to work on that Key Value Pair. The MAP command output is fed to the REDUCER and I am trying to create a hash of Key Value pair in reducer. But the hash is empty when I feed the data to it. The MAP is giving a proper output. I am facing a problem while creating the hash. Also I need to add the values of hash key that are same. How do I do that


MAP

#!/usr/bin/perl

open (MYFILE, "$ARGV[0]")or die $!;
while ( <MYFILE> ) {

chomp;
(my $txnid,my $date,my $custid,my $amount, my $productc, my $subc, my 
+$city, my $state, my $mode) = split(",");
my $key_to_reduce = join ".", "$subc", "$state";

print "$key_to_reduce\t$amount \n";

}
close(MYFILE);

REDUCE

#!/usr/bin/perl

my %worth = ();

while (<>) {
chomp;
(my $combined, my $value) = split ("\t");
print "Key to Reducer is:$combined \n";
print "Value of the key:$value \n";
$worth{$combined} = $value;
}
print "The hash is: %worth";
foreach my $key ( keys %worth ) {
print "key: $combined, value: $worth{$combined}\n";
}

REDUCE OUTPUT

Key to Reducer is:Fencing.Missouri
Value of the key:169.68
Key to Reducer is:Cheerleading.California
Value of the key:078.61
Key to Reducer is:Field Hockey.Texas
Value of the key:085.64
Key to Reducer is:Weightlifting Belts.Florida
Value of the key:030.48

............

key: , value:
key: , value:
key: , value:
key: , value:
key: , value:
key: , value:
key: , value:
key: , value:
key: , value:

MAP OUTPUT

notroot@ubuntu:~/lab/programs$ perl transaction_map.pl /home/notroot/l
+ab/data/txns | more
Cardio Machine Accessories.Tennessee    040.33
Weightlifting Gloves.California 198.44
Weightlifting Machine Accessories.California    005.58
Gymnastics Rings.Wisconsin      198.19
Field Hockey.Tennessee  098.81

DATA

00000000,06-26-2011,4007024,040.33,Exercise & Fitness,Cardio Machine A
+ccessories,Clarksville,Tennessee,credit
00000001,05-26-2011,4006742,198.44,Exercise & Fitness,Weightlifting Gl
+oves,Long Beach,California,credit
00000002,06-01-2011,4009775,005.58,Exercise & Fitness,Weightlifting Ma
+chine Accessories,Anaheim,California,credit
00000003,06-05-2011,4002199,198.19,Gymnastics,Gymnastics Rings,Milwauk
+ee,Wisconsin,credit
00000004,12-17-2011,4002613,098.81,Team Sports,Field Hockey,Nashville 
+ ,Tennessee,credit
00000005,02-14-2011,4007591,193.63,Outdoor Recreation,Camping & Backpa
+cking & Hiking,Chicago,Illinois,credit
00000006,10-28-2011,4002190,027.89,Puzzles,Jigsaw Puzzles,Charleston,S
+outh Carolina,credit
00000007,07-14-2011,4002964,096.01,Outdoor Play Equipment,Sandboxes,Co
+lumbus,Ohio,credit
00000008,01-17-2011,4007361,010.44,Winter Sports,Snowmobiling,Des Moin
+es,Iowa,credit

COMMAND

perl transaction_map.pl /home/notroot/lab/data/txns | perl transaction
+_reduce.pl
[download]

Comment on Create a Hash Dynamically and add the values of same key Download Code

Replies are listed 'Best First'.
Re: Create a Hash Dynamically and add the values of same key by poj (Abbot) on Jun 18, 2013 at 10:32 UTC
The error is here ; `foreach my $key ( keys %worth ) { # print "key: $combined, value: $worth{$combined}\n"; print "key: $key, value: $worth{$key}\n" }` [download] poj	[reply] [d/l]
Re^2: Create a Hash Dynamically and add the values of same key by rahulruns (Scribe) on Jun 19, 2013 at 04:05 UTC
Thank You for pointing the mistake. It is working now. I am able to get key and value	[reply]
Re: Create a Hash Dynamically and add the values of same key by hdb (Monsignor) on Jun 18, 2013 at 10:42 UTC
For the adding you would do `$worth{$combined} += $value;` [download] instead of `$worth{$combined} = $value;`	[reply] [d/l] [select]
Re^2: Create a Hash Dynamically and add the values of same key by rahulruns (Scribe) on Jun 19, 2013 at 04:07 UTC
It is not adding the values related to one key. Is is possible to make the key unique and let all the values for that key be associated to the key and then sum all the values related to that key	[reply]
Re^3: Create a Hash Dynamically and add the values of same key by hdb (Monsignor) on Jun 19, 2013 at 06:11 UTC
In the data you provided there is no duplicate keys, so no adding will happen. The key is the concatenation of $subc and $state and this combination creates unique keys for your data. See below where I integrated you MAP and REDUCE into on loop. use strict; use warnings; my %worth; while ( <DATA> ) { chomp; my( $txnid, $date, $custid, $amount, $productc, $subc, $city, +$state, $mode ) = split /,/; $worth{"$subc.$state"} += $amount; # REDUCE } while( my ( $key, $value) = each %worth ) { print "key: $key, value: $value\n"; } __DATA__ 00000000,06-26-2011,4007024,040.33,Exercise & Fitness,Cardio Machine A +ccessories,Clarksville,Tennessee,credit 00000001,05-26-2011,4006742,198.44,Exercise & Fitness,Weightlifting Gl +oves,Long Beach,California,credit 00000002,06-01-2011,4009775,005.58,Exercise & Fitness,Weightlifting Ma +chine Accessories,Anaheim,California,credit 00000003,06-05-2011,4002199,198.19,Gymnastics,Gymnastics Rings,Milwauk +ee,Wisconsin,credit 00000004,12-17-2011,4002613,098.81,Team Sports,Field Hockey,Nashville + ,Tennessee,credit 00000005,02-14-2011,4007591,193.63,Outdoor Recreation,Camping & Backpa +cking & Hiking,Chicago,Illinois,credit 00000006,10-28-2011,4002190,027.89,Puzzles,Jigsaw Puzzles,Charleston,S +outh Carolina,credit 00000007,07-14-2011,4002964,096.01,Outdoor Play Equipment,Sandboxes,Co +lumbus,Ohio,credit 00000008,01-17-2011,4007361,010.44,Winter Sports,Snowmobiling,Des Moin +es,Iowa,credit [download]	[reply] [d/l]
Re: Create a Hash Dynamically and add the values of same key by Anonymous Monk on Jun 18, 2013 at 12:16 UTC
To store more than one value with a particular hash key, let the bucket for that key be an arrayref, or another hashref.	[reply]
Re: Create a Hash Dynamically and add the values of same key by hbm (Hermit) on Jun 18, 2013 at 14:08 UTC
Suggestions for your MAP loop - rather than: `while ( <MYFILE> ) { chomp; (my $txnid,my $date,my $custid,my $amount, my $productc, my $subc, +my $city, my $state, my $mode) = split(","); my $key_to_reduce = join ".", "$subc", "$state"; print "$key_to_reduce\t$amount\n"; }` [download] This? `while ( <MYFILE> ) { # not needed if you don't use last field # chomp; # one 'my' for a list; grab only the needed fields my ($amount, $subc, $state) = (split/,/)[3,5,7]; # no need for join: my $key_to_reduce = "$subc.$state"; print "$key_to_reduce\t$amount\n"; }` [download]	[reply] [d/l] [select]