I think most CPAN hosted modules have no C/XS involved at all. What you will see and might make it look otherwise is that many of the best, most popular modules do have some. Their popularity/utility makes them prime candidates for optimization. Things like XML::LibXML are interfaces to really great packages not written in Perl. The success of these things is predicated on most everyone being in development environments where compiling the stuff is not difficult.

Perl is the only reason I'm a hacker so I mean no slight at all when I say that without the CPAN Perl would come off somewhere between PHP with Asperger's, Anarchist Java, and the abandoned feral child of C+Unix.

As to the CPAN itself, there are mountains of detritus in there but there is also some of the best software engineering around; ranging from brilliant generalization like Moose to completely specific problems like Spreadsheet::ParseExcel. There are hundreds, if not thousands, of nodes on PM discussing the nature of the CPAN.

MST gave a lightning talk at a recent conference about why the detritus is good for the environment too (audio is not good but I found it worth the trouble): You're a terrible person and should never contribute to open source software, ever.