Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Net::FTP question

by mmittiga17 (Scribe)
on Dec 17, 2010 at 17:07 UTC ( #877662=perlquestion: print w/replies, xml ) Need Help??

mmittiga17 has asked for the wisdom of the Perl Monks concerning the following question:

Hi All, does anyone know if PERL's FTP module is capable of determining a file is completely done writing to an ftp folder before it pulls it down? Is it possible that the Net::FTP could grab a file before it has completed being writen to site, hence causing me to pull an incomplete file? Thanks!!!

Replies are listed 'Best First'.
Re: Net::FTP question
by BrowserUk (Patriarch) on Dec 17, 2010 at 17:36 UTC

    If you control, or can influence the process that uploads the file, have it append the filesize to the name. Then testing for upload completion before download becomes trivial.

    Ie. mydatafile.tar.gz becomes mydatafile.tar.gz.123456789


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Net::FTP question
by Mr. Muskrat (Canon) on Dec 17, 2010 at 17:27 UTC

    If you have control of the writes on the remote server, create it as a temp file and rename/move it to the correct name once the file writes are complete.

Re: Net::FTP question
by MidLifeXis (Monsignor) on Dec 17, 2010 at 17:34 UTC

    Can you have your remote server deal with the issue of indicating when it is done? One solution that could work is a maildir structure. Create tmp file, rename to include the inode number, do the writing, then move into the "done" directory when complete. Assumption being that the temp and done directories are on the same file system.

    --MidLifeXis

Re: Net::FTP question
by AR (Friar) on Dec 17, 2010 at 17:16 UTC

    If the file is on a remote ftp server, call size on it, sleep a bit and call size again. If the sizes are the same, it's probably done.

    I don't think there's a definitive way of knowing whether some remote process is done writing.

Re: Net::FTP question
by roboticus (Chancellor) on Dec 18, 2010 at 14:45 UTC

    mmittiga17:

    At my previous job, we had the same issue. We solved it by having our data provider add a "summary" report to the end of the upload, so we wouldn't upload any data files until that one appeared. We actually didn't even use the contents of the summary file, though we could have. (It just contained the names and lengths of the data files uploaded, along with the record counts, batch date and batch totals.)

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

Re: Net::FTP question
by zentara (Archbishop) on Dec 18, 2010 at 12:29 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://877662]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (2)
As of 2023-01-29 19:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?