When an installation fails for the user they have no idea whether this is their problem alone or a systematic problem with the dist
When a newer version is released the user is unaware of any problems there might be with it before downloading/installing
If there is a systematic problem with the current version the user has no reference as to which other versions are problem-free
Bad things for authors
No way to tell if a manual bug report received for any given platform is unique to that user
No way to tell if a just-released dist has a glaring error (until hopefully manually notified by someone much later)
Having to go back-and-forth with a bug reporter to get case info rather than having the full smoke tester failure available at a single click
No way to tell if a dependency of their code has a usable current version or if the last n versions are all usable, etc.
Bad things for perl
Another important piece of infrastructure gone making it harder than it used to be for everyone to use the language (see also: AnnoCPAN, CPAN Ratings, etc.)