Do you use one of the well known ciphers like e.g. AES or a home-grown one?

Designing a good cipher is extremely hard. Here's an ancient but nevertheless still true statement from a well-known cryptographer - Bruce Schneier - that might be discouraging at first sight, but can save you a lot of trouble in the long run.

Your problem has most probably nothing to do with the AVI file but with the way it is read (binmode?) and processed (character range).

In order to get a better answer, you might need to provide us with a meaningful code sample.

Update: ... forgot to mention SFTP, which handles encryption and file-transfer (see Net::SFTP and friends, esp. Net::SFTP::Foreign).