Note that davorg suggested using -s, which for a directory tells you the size of the directory entry itself - which grows (but in many filesystems doesn't shrink) with the number of files. The size reported does not have any relation at all with the sum of the sizes of the files inside a directory. Which to me, seems what the OP wants.

Note that even using -s to sum the sizes of the files in a directory has its problems - you have consider symlinks and hardlinks. The former may cause loops, or point to a totally different part of the filesystem, while hardlinks may cause you to count files two or more times if you aren't tracking inodes.