Without knowing anything about your script except for the sparse description above, it's hard to tell what is causing the described behavior.

It does not look like Net::SFTP::Foreign is doing an ls with every put but it does use stat.