Yet it means that perl should be flexible, not necessarily that perl should become a glorified API to C libraries

I'd say that for Perl to be flexible, it is necessary (but not sufficient) to be able to access C libraries. It's a glue language. Many libraries are written in C - to connect to a database, or to manipulate images for instance - and are written with a much broader scope than "Perl programs" in mind. Perl should, and does my means of XS, provide access to those libraries.

Of course, you're always free to provide Perl only implemenations of XS modules.