push @AoH, %data

This flattens the hash %data, which is not what you want. Instead push @AoH, \%data, which pushes a (not flat) reference.

Please read perlreftut on how you can use references to build complex data structures.

Ah. That makes sense. I was wondering what happened. So I have two Arrays, @get_array and @post_array which are array of hashes. What is a more efficient way that doing nested foreach loops for when I do my comparing? That seems like a bunch of uneeded processing, and there might be some kind of seek, or....?

Some quick observations about your code so far:

• Unless you are using a very old perl, you don't need the & before function calls: logToHash($_) will do just fine.
• You will find your debugging life much easier if you add use strict; use warnings; immediately after the shabang line.

As to your question about data structures and comparisons , perhaps you could be more specific about how you want to compare fields - are you counting the GET/POST requests for each user agent? each IP? each data? some combination of the three?

Your plans will affect your choice of data structure. For instance, if you want to count all HTTP requests with the same user agent, you will need an HoAoH hash where each key is a user agent and each value is an array containing all of request hashes sharing the same user agent.

You can build this hash most efficiently while you are reading in the data. Depending on your goals, you may be able to eliminate the AoH hash entirely and rely solely on the HoAoH hashes.

Best, beth

For instance, I want to compare IP's in the get and post arrays to see if there are any matches.

Then I will look at the User Agent, to see if those are matching, and finally will also check the time stamps to see if they fall within a certain alloted time.

Basically for a few days something wasn't being tracked, so I would like to go back and get the numerical data out of the GET request and associate it to the correct post request (which will have some user data in it). So I need to track timestamps, user agent, and the IP address to get as close as possible.

If I am understanding you correctly, you could store each record in a HoHoHoA. For each record you read in, you would need the following psuedo code:

   my $IP = #extract from %data;
   my $userAgent = #extract from %data;
   my $date = #extract from %data;
   my $aRequests = $hRequests{$IP}{$userAgent}{$date};
   push @$aRequests, \%data;
[download]

Then after you have read in all the requests, loop through %hRequests using the hash keys to select the requests that interest you. The pseudo code would look something like this:

   while (my ($IP, $hUserAgents) = each(%hRequests)) {
      next if #IP is boring;
      while (my ($userAgent, $hDates = each(%$hUserAgents)) {
          next if #user agent is boring;
          while (my ($date, $aRequests) = each(%$hDates)) {
             #do something if date is in range
             #wanted for $IP, $userAgent
          }
      }
   }
[download]

There's a lot of work navigating references to hashes and arrays here. As moritz said previously, studying perldata, perlref (or perlreftut) and perldsc might be well worth your time.

Best, beth

For instance, I want to compare IP's in the get and post arrays to see if there are any matches

Then it would make sense to store it in a hash of hashes, with the IP as the key. Then the searching for common IPs is as simple as iterating over the keys of the first hash, and look them op in the second hash (which doesn't require another iteration). See for example perlfaq4, "How can I get the unique keys from two hashes?" for inspiration.

I'm not entirely sure, but I have the feeling you could do better with fewer lists and more hashes. In my experience, the only time a list/array is better than a hash is if the order of the items is important and you really don't have a usable key OR you're making a hash where multiple items have the same key, in which case you might want to use a hash of lists. Certainly, if you're going to search for items in the data structure, use a a hash. If you want to make a list of things and later check if something is in that list, don't make it with an array, make it with a hash where all the values are just 1, or something like that. It's easier and faster. I'm sure that's a massive overgeneralization.

I don't know how many times I've said "Okay, this should really be a list" and then later turned it into a hash.

Instead of making a list of all your hashes, make a hash and use the value you want to compare as the key. Then you can just do something like if ( $hash_1{$foo} eq $hash_2{$foo} )....

Thanks for the tips. I believe I am close, but not there yet. This section of code doesn't seem to be operating as I expect?


 if (%data) 
      {
          # We have data to process
          while( my ($key, $value) = each(%data) ) 
          {
              if($key =~ '%{User-Agent}i\""')
              {
                  $userAgent = $value;
              }
              if($key =~ '%t')
              {
                  $date = $value;
              }
          }
          $aRequests = $hRequests{$date}{$userAgent};
             push @$aRequests, \%data;
           }
[download]



I am using eclipse and even though I breakpoint and see the hash keys %{User-Agent}i\"" & %t, those if statements are never satisfied. It loops through the entire hash, but the $key never changes from %{Referer}i.

What am I doing wrong?

Are you sure it's not that the pattern doesn't match?

In addition to the great references (pun intended) above, you may find the following useful:

• How can I visualize my complex data structure?
• How do I compare/match deeply nested (complex) data structures?