Technically, there isn't a whole lot of advantage, except its automatic on each push and easy to view at a glance, and like you said, it's nice for non-CPAN modules (or releases you haven't put up yet). I could see another benefit if there's a small team working on a project, instead of everyone having to use Devel::Cover on each commit/push, they can simply go to a single website to monitor testing progress.

I like to browse MetaCPAN every few days and peek into modules that I might like, or for issues in others modules I might want to take a crack at fixing, and I had just found it handy to have a basic coverage percent icon right up front (it shows a (very basic) indicator of effort on testing quality), so I started doing it for my modules.

It wouldn't be worth it if it was difficult or time consuming to configure, but it's brain dead simple :)