in reply to recording a stream of data remotely, and reliably
none of this data can be lost however the machine that has the hardware to collect it
And if that machine dies?
Within the constraint that that machine stays running, a RamDisk is as safe as any other part of memory in which to store the data.
So, have the sampler process write the small files to a ram disk, and rename them to a known pattern when the file is complete.
Have a second process monitoring the ramdisk for the known pattern, and ftp them to the remote machine as they appear. It then verifies (crc/md5) the transfer and deletes them from the ramdisk.
Both the sampler and ftp processes would immediately fork on startup and the child would do the actual work. The parent simply monitors the child and should it abort for any reason, the parent immediately forks another copy and monitors that.
At ~42MB/hour, a 128MB ramdisk would give you 3 hour window in which to rectify any problems with the remote machine or the intervening network. 256MB even better if the machine has the memory available.
As a "last gasp" attempt to salvage a long term experiment, you could also arrange that if the communications with the remote machine are lost and the ramdisk is in imminent danger of being filled, then you could offload the ramdisk to the local harddrive just until the connection is restored.
The weak link will always be the one machine with a dodgy disk though.
|
|---|