You could change the data structure and merge the values early at the input stage like this. It assumes only one value in any one column. Then the print stage becomes more straight forward.

foreach my $file ( sort glob("*.csv") ) {

  $filenum++;
  open my $fh, "<", $file or die $!;

  while ( my $line = <$fh> ) {
    chomp $line;
    my ( $row_val, @values ) = split /,/, $line;
    $row_val{$row_val} = 1;
# delete   $data{$filenum}{$row_val} = \@values;
# add these 3 lines
    for my $c (1..@values){
      $data{$row_val}[$c-1] .= $values[$c-1];
    }
 }
 close $fh;
}

foreach my $row_val ( sort keys %row_val ) {
    print join ",",$row_val,@{$data{$row_val}},"\n"
}
[download]

poj