I refactored the code quite a bit:

#!/usr/bin/env perl

use strict; 
use warnings; 

my $file = $ARGV[0] || 'pickcpua.dat'; 

open my $CPUFILE, "<", $file or die "Unable to open sorted file $!"; 

my @col_tot;
my @col_num;
my @col_avg;
while (<$CPUFILE>) {
    chomp;
    my @f = split /,/;
    shift @f;   # remove the 1st column
    my $num_cols = scalar @f;
    for my $i (0 .. $num_cols-1) {
        my $val = $f[$i];
        if ($val>0) {
           $col_num[$i]++;
           $col_tot[$i] += $val;
        }
    }
}

for my $i (0 .. $#col_num) {
    $col_avg[$i] = $col_tot[$i]/$col_num[$i];
    print "Column ", ($i + 2), " Average = $col_avg[$i]\n";
}

close $CPUFILE or die; 

my $total = 0; 
$total += $_ for @col_avg; 
print "Average CPU Time for Column for all servers in cpuatest.dat fil
+e is ";
print $total/@col_avg, "\n";
[download]

Update: Oops. After checking math (for a 2nd time!), this is what I get:

Column 2 Average = 8.27777777777778
Column 3 Average = 8.33222222222222
Column 4 Average = 8.11
Column 5 Average = 8.77
Column 6 Average = 10.4677777777778
Column 7 Average = 17.8433333333333
Column 8 Average = 6.42571428571429
Column 9 Average = 6.66571428571429
Column 10 Average = 6.36571428571429
Column 11 Average = 5.85714285714286
Column 12 Average = 6.01142857142857
Average CPU Time for Column for all servers in cpuatest.dat file is 8.
+46607503607504
[download]

Is this what you are looking for?