csftpteam has asked for the wisdom of the Perl Monks concerning the following question:

I'm using the Filesys::SmbClientParser module to retrieve files from a file share. The module is working pretty good, but every now and then my script does not pick up one of the files from the share. The output says that it did pick up the file, but it does not actually pick it up. Here is my code:

# if the transfer is a GET } elsif ($cmd eq "GET") { my @remote_list; my @files_to_get; # get the remote directory listing so we can parse through it @remote_list = $smb->dir; # if @remote_list = 1, it means that the share is inaccessible if(@remote_list == 1){ &logEntry($args{'logfile'}, "$jobname-ERROR: Could not retriev +e remote directory listing: ".$smb->err." - Going to next transfer.") +; next; } # parse through the remote directory listing, looking for files th +at we want foreach (@remote_list){ # if no files were specified to get... if ($remote_file eq ''){ &logEntry($args{'logfile'}, "$jobname-ERROR: No files spec +ified to get. Exiting"); exit; # if the file we are looking for is just a filename with no wi +ldcards... } elsif ($remote_file !~ /\*/){ # if the file is not a dir and it matches our pattern, pus +h it onto the array if($_->{attr} ne "D" && $_->{name} eq $remote_file){ push(@files_to_get, $_->{name}); } # if the file we are looking for is wildcarded... } elsif ($remote_file =~ /\*/){ my $remote_file_pattern = $remote_file; $remote_file_pattern =~ s/\*//; $remote_file_pattern =~ s/\./\\./; # if the remote file name matches our pattern, push it ont +o the array # if $remote_file_pattern is empty, it means that "*" was +asked for if($_->{attr} ne "D" && ($_->{name} =~ /$remote_file_patte +rn/ || $remote_file_pattern eq '')){ push(@files_to_get, $_->{name}); } } } if (@files_to_get == 0){ &logEntry($args{'logfile'}, "$jobname-RUNNING: There are no re +mote files to get matching $remote. Going to next transfer."); next; } # loop through for each file in the glob foreach (@files_to_get) { my $file = $_; # THIS DOES NOT WORK - FILES BEING COPIED INTO THE REMOTE DIR +DO NOT SHOW UP AS SUCH # check that the size of remote file is not still changing my $dir_size = 0; my $dir_tmp_size = -1; #while ( $dir_size ne $dir_tmp_size ) { # $dir_tmp_size = $dir_size; # $dir_size = $smb->du; # get the remote directory size # sleep(5); # print "DIR_SIZE:$dir_size;\n"; #} &logEntry($args{'logfile'}, "$jobname-RUNNING: Attempting to g +et $remote_dir".$file." to $local_dir"); my $i = 0; # try to get the file 4 times while($i < 4){ # get the remote file if ($smb->get ("$file")){ &logEntry($args{'logfile'}, "$jobname-SUCCESS: $remote +_dir"."$file retrieved successfully"); $smb_out = $smb->get ("$file"); &logEntry($args{'logfile'}, "$jobname-DEBUG: $smb_out" +); # if config is not set to not delete files; delete the +m if($delete_remote_files ne "NO"){ &logEntry($args{'logfile'}, "$jobname-RUNNING: Att +empting to delete $remote_dir".$file); # delete the remote file if ($smb->del ("$file")) { &logEntry($args{'logfile'}, "$jobname-SUCCESS: + $remote_dir"."$file deleted successfully"); } else{ &logEntry($args{'logfile'}, "$jobname-ERROR: $ +remote_dir"."$file might not have been deleted successfully: ".$smb-> +err); } } $i = 5; } else { &logEntry($args{'logfile'}, "$jobname-ERROR: $remote_d +ir"."$file might not have been retrieved successfully: ".$smb->err." +- Trying again in 1 minute."); sleep (1); $i += 1; } if($i == 4){ &logEntry($args{'logfile'}, "$jobname-ERROR: Transfer +failed for the fourth time. Going to the next transfer."); } } }

Does anyone have any idea why I'm getting sporadic file pickup failures? This code consistently spits out a "1", showing a success even if the file was not retrieved:

$smb->get ("$file")

Thanks! Vic

Replies are listed 'Best First'.
Re: Filesys::SmbClientParser Get Files Randomly Fails with no Error
by Corion (Patriarch) on Nov 21, 2016 at 20:20 UTC

    Have you tried reducing your code to the relevant 10 lines or so? Without seeing and knowing your complete program, it is hard for us to advise you in a good way because we can't tell what is old code, which code is actually run and what values the various variables have.

    Ideally, you can produce a short, self-contained script that uses the relevant module(s) and runs a short loop that at a certain time fails so we can easily reproduce the problem. Note that we will also need the output of that program and an exact description of what you think it does and why that is not what you want, and what exactly you want.

    While trying to read through your code as shown above, I noticed that it tries to call smb->get ("$file") twice. This seems weird. Maybe the lack of error checking on the first (or second) call is relevant?

    Also note that I'm quite unclear on what you mean by 'This code consistently spits out a "1"' - I don't see any print statement, or return statement, or anything else that would spit out a number. Maybe this is hidden somewhere in the logfile that you haven't shown us.

    One thing that might be problematic for you might be filesystem encodings between the remote/SMB filesystem and your local filesystem, and the mechanism by which the module parses the output. Usually, unixishes OSes blindly assume that all filenames are UTF-8 encoded octets. SMB filesystems might have a different encoding for the filenames, but will still return the raw octets to the caller.

Re: Filesys::SmbClientParser Get Files Randomly Fails with no Error
by GrandFather (Saint) on Nov 22, 2016 at 00:30 UTC

    What do you get in the debug logging when you set $smb->Debug(1)?

    Premature optimization is the root of all job security