Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello

I'm trying to move files from one server to a new one, so I made a simple script using ftp, problem is Net::FTP broke files, here is my script:

#!/usr/bin/perl use strict; use warnings; use Cwd; use File::Spec; use Getopt::Long; use Net::FTP; use DBI; use Digest::MD5::File qw(-nofatals file_md5_hex); use File::Basename; my $path = File::Spec->catdir(getcwd()); my $host, my $username, my $password; GetOptions ( "dir=s" => \$path, "host=s" => \$host, "user=s" => \$username, "pass=s" => \$password ) or die("Error in command line arguments\n"); $path = File::Spec->catdir(getcwd(),$path); # Connect my $ftp = Net::FTP->new($host,Passive => 0) or die("Can't connect to: +$!"); $ftp->login($username,$password) or die("Couldn't authenticate ftp: $! +"); #$ftp->pasv(); $ftp->binary(); my $dbh = DBI->connect("DBI:SQLite:dbname=moving.db", "", "", { RaiseE +rror => 1 }) or die $DBI::errstr; $dbh->do(q{ CREATE TABLE IF NOT EXISTS "files"( hash TEXT, path TEXT, size INT )}); my @tree = ($path); loop: { $path = pop(@tree); opendir(DIR, $path) or die $!; unless ($ftp->cwd($path)) { $ftp->mkdir($path,1) or die "$! ", $ftp->message; $ftp->cwd($path) or die "$! ", $ftp->message; } while (my $file = readdir(DIR)) { next if $file =~ m/^\./ or ! $file; $file = File::Spec->catdir($path,$file); if (-d $file) { push @tree,$file; next; } $ftp->put($file) or die "$! ", $ftp->message; die(sprintf("$file didn't match size %s <> %s",$ftp->size( +$file),-s $file)) if $ftp->size($file) != -s $file; $dbh->do('INSERT INTO "files"(hash,path,size) VALUES (\''. +file_md5_hex($file).'\',\''.$dbh->quote($path).'\',\''.$dbh->quote(-s + $path).'\')') or die $DBI::errstr; print "Moved: $path\n"; } closedir(DIR); if (scalar @tree) { goto loop; } } $ftp->quit;

When its send a file it add some extra space, even with binary mode, both OS are the same Ubuntu 12.04, do you have an idea what could be wrong? because I'm a bit lost

Thanks

Replies are listed 'Best First'.
Re: Net::FTP bug or is my error?
by kcott (Archbishop) on Aug 29, 2013 at 09:48 UTC

    There's nothing in your code that immediately leaps out at me as being the cause. Had you shown your messages, that may have helped. Also, "it add some extra space" isn't all that useful: disk space? whitespace? how much extra? Furthermore, you included a lot of additional code (database, digest, options) that's irrelevant to the problem: this neither helps nor provides a script we can run to attempt to reproduce your problem.

    Here's some things you can try:

    • See how the data being added relates to the actual file: 1 byte might suggest an end-of-file issue; bytes == lines might suggest an end-of-line issue; some other relationship might suggest something else.
    • If you have access to both servers, login and check (e.g. with ls) whether the sizes reported by your program match the sizes reported by the OS.
    • Look at the files (e.g. with cat -vet, hexdump, or similar tools) and see how they differ.
    • Check whether your source data contains Unicode. If so, look for encoding issues: remembering that ASCII characters are one byte, while Unicode characters are one or more bytes.
    • Write a test script, which is a stripped down version of what you posted, such that it just contains barebones FTP transfer code. Then, using a small, but representative, example of your real data, attempt to reproduce your problem. Doing this may highlight the problem for you; if not, post that minimal script and we can look at it further.

    Finally, and unrelated to your posted issue, instead of "loop: { ... goto loop }", consider:

    while (1) { ... last unless @tree; }

    -- Ken

      I stripped down all unnecessary content and after a while fighting with the problem I realize that I have net::ftp module installed from Ubuntu repository, so, just in case I uninstalled and reinstalled from CPAN and problem solved, to be honest I did a lot of change but i'm pretty sure that was problem

      Thank you Ken for your response was really useful and I have changed the ugly goto logic to a more readable infinity while ;-)