It might've been different with a bigger file but my benchmarking indicate considerably faster than Encode::decode()

                 Rate      implicit encode_decode    utf_decode       utf8_on
implicit      34929/s            --          -31%          -60%          -60%
encode_decode 50765/s           45%            --          -41%          -43%
utf_decode    86322/s          147%           70%            --           -2%
utf8_on       88314/s          153%           74%            2%            --   

It was a pleasant surprise to see utf::decode get so close though. Although given utf::decode won't protect me from non-utf8 input either I guess the optimal solution is
$line = Encode::decode('UTF-8', $line) unless (utf8::decode($line));