|
Re: Re: FTP and File Copying
by Anonymous Monk on Nov 27, 2000 at 07:10 UTC
|
That is not one of my options. The files need to be taken as is once they are uploaded. I don't have the clout nor do the submitters have the knowledge to go in and rename files via FTP. Most of the people uploading these files are lucky to be able to FTP them. I cannot enforce new to old naming conventions and expect them to do this, I am lucky I get them to stick to the naming conventions at all. So the issue is I need to be able to tell if a file is currently open and being written to by any other process. I haven't been able to find anything as of yet to do this. I have though of work arounds such as writting filenames and time stamps to a DB and looking to the xferlog for new incoming files but I would much rather do this on the level of the files itself rather then bringing in extraneous resources. I've also tried flock but I can get a full lock even if the file is being written to by another source (such a file becoming a tar can flock'd mid tar) which I don't want if the file is still being written to.
Any process I have control of do go temp to final filenames when using FTP where I can. | [reply] |
|
|
If you are running under a Unix OS, you can use a utility called "lsof" to determine what processes currently have the file open, and the mode used, etc. My lsof binary points to the URL ftp://vic.cc.purdue.edu/pub/tools/unix/lsof, if your system doesn't have it. The exit status of this program is 0 if any programs have the file open.
I really think this is kind of a crappy solution, though, because other programs may have that file open for other reasons that might not be easy to differentiate, and (as mentioned elsewhere), it doesn't help you if the FTP process legitimately closes the file in an incomplete form.
| [reply] |
|
|
Using lsof is not safe, ftp may release the file if the connection is broken an reaquire later this happens seamlessly from the users perspective. It is also not supported by all unix.
The only 'safe' way to do this is to do what merlyn suggests. However if you cant rename then you could upload an empty file after the upload is complete and poll waiting for the control file to arrive, as soon as it arrives you know the preceeding download is complete.
--
Zigster
| [reply] |
|
|
| [reply] |
|
|
That isn't true if you broaden your toolset: utilities like fstat (on BSD), fuser (on Solaris), and lsof (on a wide variety of UNIX(ish) platforms, including the aforementioned) will all quite happily tell you whether a file is currently in use ...
perl -wle '@open = qx|fstat $ARGV[0] 2>/dev/null|; print "YES" if $#open > 0' somefile
(Error checking, and use strict;'ness left as an exercise to the reader. ;)
Unfortunately, because all of those only give you a snapshot of how things looked, none of them are going to help tell whether a file is incomplete or allow you to easily deal with the race condition that follows testing (ie, what if someone resumes an incomplete upload?), but that may be a more acceptable margin of error depending on your circumstances.
Another possibility lies with some of the more advanced (?) ftpd like NcFTPd, which supposedly support event reaction and/or notification. Theoretically, that means you could have the server kick off other processes when it thinks the files are finished uploaded instead of polling every half hour.
--Kanji
Update: D'oh! Didn't see Fastolfe's response before I wrote this ... that'll teach me to post before reading the entire thread.
| [reply]
[d/l] |
|
|
Well I'd like to thank everyone for their input. I had rooted through various sources and didn't find any easy solutions and I was hoping there was an obviously easy one that I just couldn't put my finger on. It looks like even though I can get perl to make coffee for me in the morning, it's not going to grind the beans for me ;) It looks like suggestions are saying I'll have to do this from more of a OS based standpoint which is where I was thinking it was going to need to go. Thanks all... for all the feedback
| [reply] |
|
|
If you really do have to get them as soon as they are
finished, this sounds like a project for a secure web
server. Of course, that won't work for dialin folks, but
could be a good solution.
| [reply] |