Nitpicks:

• You're reimplementing the standard File::Find module which does this (correctly) for you (to say nothing of other similar modules available from CPAN such as File::Find::Rule, Path::Tiny, Path::Iterator::Rule, . . .)
• On OSen and/or filesystems which allow sparse files there's usually a way to ask du to account for that fact and get the actual space used by the file; summing the sizes returned by stat(2) (via caling stat or the -s operator) won't.
• You don't check the return value from opendir; ALWAYS CHECK THE RETURN VALUE FROM SYSTEM CALLS.
• You're needlessly stringifying in a couple places, e.g. -s "$file"; not necessarily a problem in this particular case but it's a bad habit to develop. Somewhere down the road you're going to have an object instance for a class which has overloaded stringification that you DIDN'T mean to turn into just a string and then you're going to have to debug why you're getting an error about not being able to locate method foo via package bar from somewhere deep down in your code.