I belive you'll find that still reads the entire file in before running the grep - so filtering the file first doesn't really give you anything.

If the size of the file is an issue you'll want to use a loop to go over it line by line. Something like this should do it:

my %name2email;
while (<DATA>) {
    next unless m/^(?:To|From|Reply-To): (\S+)\s+<?([^\s>]+)/;
    $name2email{$1} = $2;
};
[download]

Note: the regex doesn't cope with a fair bit (multiple addresses, addresses without names, etc.).