The names only print out if you use "verbose" mode. See the "-v" flag for prove, the description of the HARNESS_VERBOSE environment flag in Test::Harness and ways to set that automatically with Module::Build and ExtUtils::MakeMaker.

As a side note, you may want to write your test like this:

foreach (@items) {
  ...  a lot of perl code ...
  unless (some condition) {
    $err = "Splunk is missing from Foobar";
  }
}

ok(! $err,"Fancy code") or diag $err;
[download]

Using pass and fail is generally a more tedious way to work with Test::More.