Create a daemon to handle the connections and transfers, that opens a local socket and accepts the paths of files to be transferred.

Have it copy or move the files into a private directory when the transfer request is made and delete once it is complete. If the deamon dies, it know what needs to be done when it re-starts.

It can either maintain an open connection to the remote host, or only establish the connection when there are transfers to be done.

I was also thinking about this approach, but I was a bit reluctant to implement it because it was a lot more work than using the lock file.
If the transfer becomes a bottleneck, I will think about implementing it again.
At the moment I am busy reducing my watch cycles for the successful transfers, but if I think it is too slow, then I will go for this approach.

I would go for BrowserUK's approach, and I fail to see how it's going to be a lot more work. In fact, it's probably going to be a lot less work. With a separate program doing the FTP transfers, you only have one place where you have to worry about failed transfers, instead of having to deal with it in all your programs.

Besides, it follows the Unix toolkit approach: separate things are done by separate programs - each program tuned to do its task very well.

This is a very helpful reference<.
I might start to use Proc::PID::File with my server name instead of default $0 instead my own custom implementation.

One approach would be to create a script that transfers files to the ftp server when they appear in a watched directory. This script could be run as a daemon or through cron depending on how quickly you need it to act. This script would need to assure that only one instance was active at a time, but the techniques described in the aforementioned node should work well for this purpose. Your existing scripts would need to be modified to move files to the watched directory rather than sending them to the remote server.

On linux, another approach would be to use the fuse (filesystem in userspace) kernel module with curlftpfs (LftpFS may be a reasonable alternative; avoid FuseFTP as it is not very robust) to mount the remote server as a local file system (I believe this can be done through fstab). This also lets a single entity manage the connection to the remote ftp server. The existing scripts would need to be modified to write files to the appropriate local path (functionally sending them to the remote server via ftp). I believe that this approach will allow the ftp connection to timeout while not in use and will only reconnect when needed; it would be best to verify that this is correct.

Does your advisary locking of a pid file scheme, handle recovery after a system crash?

Polling directories is a silly way to do things.

1. How long do you wait after the filename appears, before you decide that the application writing that file has finished doing so?
2. How do you detect if the process writing the file has hung or crashed part way through writing it?

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Does your advisary locking of a pid file scheme, handle recovery after a system crash?

Yes.

If you review the mentioned thread, the first node by shmem details how the script should verify that the listed PID is actually running the proper script.

Polling directories is a silly way to do things.

Silliness is relative. If the files only need to be pushed to the server every hour, polling doesn't seem so bad.

How long do you wait after the filename appears, before you decide that the application writing that file has finished doing so?

How do you detect if the process writing the file has hung or crashed part way through writing it?

In the first approach, I specifically said "...to move files to the watched directory...." This is an important point; perhaps I should have explicitly stated that files should be moved to the directory rather than written to the directory.

Update: Corrected "rather than copied" to "rather than written" in the last sentence.

Update 2: Corrected HTML formatting of whitespace before update 1.