Re: Split large file and upload to ftp server with multiple threads

As I read this (incorrectly, I hope), you are going to end up with multiple chunks (files!) at the server end.

How do you expect to reconstitute the original file there? How will you know which chunk is first and which the next and so on?

And second, why would you expect "multiple connections" "...to speed up the transfer..." unless you have "multiple connections" (in the sense of independent circuits which, combined, provide greater bandwidth than a single circuit)?

If so, does the server end have resources to accept the torrent that will ensue?

Comment on Re: Split large file and upload to ftp server with multiple threads - SANITY CHECK

Replies are listed 'Best First'.
Re^2: Split large file and upload to ftp server with multiple threads - SANITY CHECK by mdc76 (Acolyte) on Oct 02, 2008 at 05:44 UTC
Independent circuits combined providing greater bandwitdh than a single circuit is correct. I wasn't sure how to reconstitue the files either, or if it was even possible with FTP.	[reply]
Re^3: Split large file and upload to ftp server with multiple threads - SANITY CHECK by salva (Canon) on Oct 02, 2008 at 08:43 UTC
Independent circuits combined providing greater bandwidth than a single circuit is correct Unless you have two different physical connections between the local and remote network (and that means two modems, two DSLs, or two whatever at the local side), you don't have two independent circuits. On the old days, when networks had lots of errors and packets dropped, using several TCP connections (all going over the same physical layer) to send data in parallel was an effective mean to increase throughput. But nowadays, that the networks are quite reliable, it doesn't make sense anymore, you will only get a marginal improvement (if any) on the transfer speed. Effective ways to reduce the transfer time are: Use compression, if you are already using compression, use a better compressor (bzip2 is better than gzip and z7 is usually better than bzip2). Sometimes your data can be represented in a more compact format. For instance, CSV is better than XML in this regard. Can you use `rsync`?	[reply] [d/l]