Use default variables and avoid abusing map:

$t+=$_ for<>;print$t/$.
[download]

23 characters.

If adding the file to the code, use the poetry optimization:

@ARGV=o;$t+=$_ for<>;print$t/$.
[download]

leaves us at 31. With error handling if the file is not there! :-)

PS: Your approach was nice. :-)

EDIT
Thanks Corion for catching a wrong closing code tag.