In sticking with the core modules so that you don't have to install any new modules, here's some untested code that is basically a modified version of what Tux provided that handles your new requirement.

use File::Find;
use File::Basename;
use feature 'say';

my @files;
find (sub { m/\.csv$/i && -s $_ < 5500 and push @files, $File::Find::N
+ame }, "C:/Temp");

my %dirs;
foreach my $file (@files) {
    my($filename, $directories, $suffix) = fileparse($file);
    $dirs{$directories}++;
}

my @keys = keys %dirs;
@keys = sort @keys;
foreach my $key (@keys) {say $key;}
[download]

Basically, what I've done is taken the array of files that meet your criteria and then putting their path (minus filename) into a hash where the path is a key in the hash (and the value for the key is the count of files in that folder that meet your criteria). Then I'm sorting the keys and printing them out. Hopefully, this will help get your further into solving your problem.