I started by setting up a sample set of test directories:

[mcmahon@joe-desk ~]$ ls -R ./example/
./example/:
file  nonempty_files_only  nonempty_has_dirs

./example/nonempty_files_only:
file1  file2

./example/nonempty_has_dirs:
file1  one  two

./example/nonempty_has_dirs/one:

./example/nonempty_has_dirs/two:
[download]

That's a directory containing files and other (nonempty) directories, one containing only files, and one containing a file and two empty directories.

sub dive {
  my($d) = shift;
  return if ! -d $d;

  my @contents = glob("$d/*");
  return $d unless @contents;

  my @below = map { dive($_) } @contents;
  return @below 
    ? @below     # Stuff below qualifies, this doesn't 
    : $d;        # Nothing below qualifies, this does
}

$d = './example';
print join ", ", dive($d),"\n";
[download]

This prints

./example/nonempty_files_only, ./example/nonempty_has_dirs/one, ./exam
+ple/nonempty_has_dirs/two
[download]

The tricky bit is postponing the decision about whether the current directory is good until you've seen if any subdirectories of it qualify.

Edit: Removed the majority of the comments as they were actually obscuring how short this is; renamed @queue as it was a leftover from a previous, longer, iterative version.