I want the order of the keys while printing to be the same as when the keys were read in!

Tie::IxHash will do this for you.

Any tips please!

Yes, I have some more tips...

Use the strictures to find other potential problems with your code:

use warnings;
use strict;
[download]

This would produce the following warnings:

Scalar value @ARGV[0] better written as $ARGV[0] at ...
Scalar value @temparray[0] better written as $temparray[0] at ...
[download]

It will then be necessary to declare all variables with my to get the code to compile again.

Always check success when you open a file, and always close the file.

I refactored your code:

#!/usr/bin/env perl
use warnings;
use strict;
use Tie::IxHash;

tie my %goldhash, "Tie::IxHash";
my $goldcsv = shift;
my @keyarray;
my @temparray;

open my $GOLD_FH, '<', $goldcsv or die "Can not open $goldcsv $!\n";
my $i = 0;
while (<$GOLD_FH>) {
    chomp;
    $i++;
    if ($i eq 1) { 
        @temparray = split /,/;
        my $j=0;
        for my $elem (@temparray) {
            $j++;
            next if ($j eq 1);
            push @keyarray, $elem;
        }
        next;
    }
    @temparray = split /,/;
    my $testcasename = $temparray[0];
    for my $value (1 .. $#temparray) {
        $goldhash{$testcasename}{$keyarray[$value-1]} = $temparray[$va
+lue];
    }
}
close $GOLD_FH or die "Can not close $goldcsv $!\n";

for my $testcase (keys %goldhash) {
    print "$testcase: ";
    for my $value (keys %{ $goldhash{$testcase} }) {
        print "$value = $goldhash{$testcase}{$value} ";
    }
    print "\n";
}
[download]

Here is the output. Is this what you had in mind?

Test1:  Stage1Mem = 44  Stage2Time = 45  Stage2Mem = 43  Stage1Time = 
+45
Test2:  Stage1Mem = 7  Stage2Time = 34  Stage2Mem = 45  Stage1Time = 2
+334
[download]