I can't say "Perl doesn't have a buffer overflow problem" because there are probably some sneaky ways that no-one's figured out an exploit for (yet).
As to questions I can answer: The perl allocator (as well as most system allocators) does nothing with memory after it's released except reset arena pointers if necessary. Some allocation routines (depending on version and such) will guarantee to initialize memory to 0, but I don't know of any outside of specialized applications that will clear memory contents prior to releasing it.
Both perlapp and perl2exe use whatever underlying perl system they packaged the program with, so they inherit all of the basic memory management, thread management, etc., capabilities and features -- both good and bad.