Smitty has asked for the wisdom of the Perl Monks concerning the following question:

I would like to solve the following problem, which can be easily solved without strict refs, but don't want to go that route.

I capture a series of fields, as an example: 

    my($ip,$name,$proto,$email) =~ /^(\S*) (\S*) (\S*) (\S*)$/;

[download]
and then use an auto-vivifying hash to act as a counter of the unique data set: 
    $userdata{$name}{$ip}{$proto}{$email}++;

[download]
Clearly this allows tracking the number of times a given NAME was used, along with the various IPs, and for each of those PROTOs, and finally EMAILs.

But this has a built-in order. I'd like to be able to allow a user-configurable ordering of that data, for example: 

    @order = qw(ip name email proto)

[download]
such that I could capture essentially as: 
    $userdata{$order[0]}{$order[1]}{$order[2]}{$[order[3]}++;
    # which I want to result in ...
    $userdata{$ip}{$name}{$email}{$proto}++;

[download]
essentially changing the primary, secondary, etc. key orders. Creating the many combinatorics to suit all cases isn't an option of course.

Can anyone suggest a mechanism to accomplish this that is a) fast, and b) not terribly memory intensive. The amount of data can be quite large, and is coming from a plain text file and not a DB.

Thanks in advance.

Replies are listed 'Best First'.
Re: Configurable key orders
by Fletch (Bishop) on Jun 22, 2008 at 16:41 UTC

    Sounds like you're reaching the point where you do want to introduce a DB. Something like DBD::SQLite would let you have the underlying SQL engine do the bucketing for you quickly without keeping multiple views resident in memory.

    Update: And as to your actual question you could store the original split values into a hash instead perhaps: 

    my %vals;
@vals{ qw( ip name proto email ) } = /^.../;

    [download]

    Then you just use $vals{ $order[0] } to get what was $ip etc.

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

[reply]
[d/l]
[select]
Re: Configurable key orders
by samtregar (Abbot) on Jun 22, 2008 at 17:55 UTC 
      # the order the data appears in the file
  my %pos = (ip => 0, name => 1, email => 2, proto => 3);

  # the order requested by the user
  my @req = qw(name email proto ip);

  # the ordering used to build the hash
  my @order = map { $pos{$_} } @req;

  # get the data
  my @data =~ /^(\S*) (\S*) (\S*) (\S*)$/;
 
  # file in the hash
  $userdata{$data[$order[0]]}{$data[$order[1]]}{$data[$order[2]]}{$dat
+a[$order[3]]}++;

    [download]

    -sam

[reply]
[d/l]
Re: Configurable key orders
by roboticus (Chancellor) on Jun 22, 2008 at 17:10 UTC
    Smitty:

    A similar question was asked several months ago, and no doubt super search could help find it. But I have no idea what keywords to use. However, if I recall correctly, one interesting solution was to sort the keys, so all combinations would resolve to the same element.

    ...roboticus
[reply]

Back to Seekers of Perl Wisdom