Honestly, I'd do this with a few command-line tools, rather than Perl, but the concepts would be the same:

find Dir_build1 Dir_build2 -type f -size 0 >/tmp/zeros
find Dir_build1 -type f | sort >/tmp/1
find Dir_build2 -type f | sort >/tmp/2
comm -3 /tmp/1 /tmp/2
[download]

To do it in Perl, I'd use File::Find to recursively scan through each directory, adding the file names to an array (probably a hash, actually) for each, then sort and compare the two. During the scans, I'd also build a list of zero-byte files, to avoid scanning through them twice, of course.