Maybe you want to use a hash to store your address values... that would tend to make things simpler overall:

my %address;

while (<DATA>) {
    if ( /^(To|From|Contact):.*?(<.*?>)/ ) {
        $address{$1} = $2;
    }
}

for my $field ( qw/To From Contact/ ) {
    if ( exists( $address{$field} )) {
        print "The $field address is $address{$field}\n";
    }
}
[download]

(untested)