# The @AllPatterns array is probably fed elsewhere, this is just for t
+he test
@AllPatterns = ([qw(1 2 1 1 8)],
        [qw(1 6 5 12 12)],
        [qw(1 1 1 1 1)],
        [qw(1 1 1 1 1)]);

for $v (@AllPatterns) {
    ++$h{join(' ',sort {$a <=> $b} @$v)};
} 

for $k (sort keys %h) { print "$h{$k} - $k\n"; }
[download]

This will probably choke (run out of memory) on a "gigantic array" so you might need to write to disk after each step you outlined.

Update: Just struck me that you may have a simple array rather than a 2D as I has assumed. Possibly clear how to adapt the hack but here it is just to get it out of my head:

@AllPatterns = ('1 2 1 1 8',
        '1 6 5 12 12',
        '1 1 1 1 1',
        '1 1 1 1 1');
for $v (@AllPatterns) {
    ++$h{join(' ',sort {$a <=> $b} split(' ',$v))};
}
[download]

Hi, igelkott,

Your suggestion worked smashingly on the first try. Good job!

Thanks,

-fiddler42

sort -k1n -k2n -k3n -k4n -k5n < inputfilename |
   uniq -c > outputfilename
[download]

      2 1 1 1 1 1
      1 1 2 1 1 8
      1 1 6 5 12 12
[download]

Be careful to double-check your locale.

On Linux with GNU sort I've been bitten in the past by the fact that the default locale does not consider some characters (in that case tabs) to be part of the collating operation.

For that reason I find it a good precaution to set $LC_ALL to C before using GNU sort.

2 - 1 1 1 1 1
1 - 1 1 1 2 8
1 - 1 5 6 12 12
[download]

2 - 1 1 1 1 1
1 - 1 2 1 1 8
1 - 1 6 5 12 12
[download]

The following code produces output that seems to conform to the OP's exemplified requirement (the two added values of '0 0 0 0 0' test for proper handling of leading-zero removal):

2 - 0 0 0 0 0
2 - 1 1 1 1 1
1 - 1 2 1 1 8
1 - 1 6 5 12 12
[download]

Code:

use warnings;
use strict;


my @AllPatterns = (
    '1 2 1 1 8',
    '0 0 0 0 0',
    '1 6 5 12 12',
    '0 0 0 0 0',
    '1 1 1 1 1',
    '1 1 1 1 1',
    );

{  # limit scope of array processing variables

my $biggest = 10;  # biggest number (most decimal digits)
my $fmt     = join ' ', ("%0${biggest}ld") x 5;
my $lead0s  = qr{ \b 0{1,@{[ $biggest - 1 ]}} }xms;

my %seen;  # unique-ifing hash

@AllPatterns =                     # 7. save to original array
    map  { "$seen{$_} - $_"      } # 6. make into final format
    grep { not $seen{$_}++       } # 5. only patterns not seen yet
    map  { s{$lead0s}{}xmsg;  $_ } # 4. remove padding
    sort                           # 3. lexicographic ascending sort
    # map  { print "'$_' \n"; $_ } # 2b. FOR DEBUG
    map  { sprintf $fmt, split   } # 2. pad fields to constant widths
    @AllPatterns                   # 1. for all patterns...
    ;

}  # end scope of array processing variables

print join("\n", @AllPatterns), "\n";
[download]

The first is that I always have to wonder what people mean when they say "gigantic". I've seen people say that about data sets from a few thousand lines to billions. Usually people mean something more like the former, which is a much easier question to answer than the latter.

The second is that if you're regularly faced with this kind of problem, you should get familiar with how to use databases. With a database you could give your columns real names, and the sorting and grouping functionality that you asked for is standard. (Along with many more complicated features.)