The problem with just having

$defaults{Mail}{From} || $ENV{USER}
[download]

is that when it is converted to:

"Test User" <test@test.com>
[download]

unless it is enclosed with ' ' MIME::Lite tries and sends and email to Test User which doesn't work. :-)

The "Test User" <test@test.com> really needed to come through to MIME::Lite as '"Test User" <test@test.com>'

Thanks all for the suggestions.