Using knowledge that 'Number' is the key of longest length:

for my $key ( sort {length($b) <=> length($a)} keys %hash ) {
    my $value = $hash{$key};
    print "$key => $value\n";
}
[download]

or more generalized using auxiliary hash:

my %order = (Number => 1,
             EX     => 2);

for my $key ( sort { ($order{$a}||1000) <=> ($order{$b}||1000)} keys %
+hash ) {
    my $value = $hash{$key};
    print "$key => $value\n";
}
[download]

Update: It's funny but your code prints 'Number' first for me without any sort procedure.