i made my own files instead of making them in Audacity. i think the problem is that Audacity does strange stuff. it is really an editor for audio files and there does not seem to be much concern with the accuracy of single samples. i sort of remember running into this sort of problem in the past. for example, it does not like DC offsets and tries to fix things up at the end of a file. it seems that when it makes "silence" there is some jitter about 0. there are possibly some other quirks.
i tried the code you provided and it works just the same as mine although it is a little more elegant. thanks for helping me to see that there is really nothing wrong on the Perl side of things.