Perl - Copy files to a newly created folder

Benichouniev has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Perl - Copy files to a newly created folder by afoken (Chancellor) on Apr 18, 2015 at 15:20 UTC
add `use strict;` and `use warnings;` check for errors at runtime, i.e. add `or die $!` to `opendir`, `open`, `copy`, `unlink`, and so on `open (DIRR,$dirr);` makes no sense. `$dirr` is a directory, opening that will fail or cause trouble, depending on the operating system. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply] [d/l] [select]
Re^2: Perl - Copy files to a newly created folder by Benichouniev (Novice) on Apr 18, 2015 at 16:10 UTC
Alexander, thank you for you answer! I changed open to opendir, it was a mistake indeed... But it did not solve the problem. I also added 'use warnings' and 'or die' everywhere but I don't get any error messages when I run the program. I see 'x files have been copied', where x is the number of files I have, which means the loop runs once per file as expected. However, sometimes it copies, sometimes it does not. And I don't know why... Ben	[reply]
Re^3: Perl - Copy files to a newly created folder by afoken (Chancellor) on Apr 18, 2015 at 16:55 UTC
I changed open to opendir, it was a mistake indeed... But it did not solve the problem. Of course not. Neither `open` nor `opendir` makes any sense around `copy`. I don't get any error messages when I run the program. I see 'x files have been copied', where x is the number of files I have, which means the loop runs once per file as expected. However, sometimes it copies, sometimes it does not. And I don't know why... Show your current code. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply] [d/l] [select]
Re^4: Perl - Copy files to a newly created folder by Benichouniev (Novice) on Apr 18, 2015 at 18:45 UTC
Re^5: Perl - Copy files to a newly created folder by afoken (Chancellor) on Apr 18, 2015 at 23:54 UTC
Re: Perl - Copy files to a newly created folder by dasgar (Priest) on Apr 18, 2015 at 17:25 UTC
Although you can use opendir and readdir to search through a directory, I think that using File::Find may be a better approach. I have to admit that I personally struggle to understand how to use File::Find directly, so I opt to use File::Find::Rule instead, which provides a different interface to File::Find. Here's a different approach using File::Find::Rule: `use strict; use warnings; use feature 'say'; use File::Copy; use File::Spec::Functions; use File::Find::Rule; use File::Path qw(make_path); use Cwd; my $pwd = cwd(); my $dir = catdir($pwd,'source'); my $rule = File::Find::Rule->new; $rule->file; $rule->maxdepth(1); my @files = $rule->in($dir); foreach my $file (@files) { my ($extension) = ($file =~ m/.+\.(.+)/); my $subdir = catdir($dir,$extension); if (!(-d $subdir)) {make_path($subdir);} copy($file,$subdir); }` [download] Here's the initial test directory that I used to test the script: `source test1.gif test1.pdf test1.txt test2.gif test2.pdf test2.txt test3.gif test3.pdf test3.txt` [download] Here's the result after running the script: `source test1.gif test1.pdf test1.txt test2.gif test2.pdf test2.txt test3.gif test3.pdf test3.txt gif test1.gif test2.gif test3.gif pdf test1.pdf test2.pdf test3.pdf txt test1.txt test2.txt test3.txt` [download] Although my code above doesn't deal with deleting the original file, I think that you should be able to modify it to meet your needs.	[reply] [d/l] [select]
Re^2: Perl - Copy files to a newly created folder by Benichouniev (Novice) on Apr 18, 2015 at 18:51 UTC
Dasgar, thanks for your answer. Unfortunately, I don't want to have to download anything additional to run my program (for the find rule). Instead, would you have an idea of how to improve my program with what I have right here and my current set of skills? As I said, I am no Perl expert and this code, right above, is what I can do. I really appreciate your help but I need something less complicated. I know it is possible, because I managed to copy a few files with this code already, now I just want to be able to copy everything and understand why it does not work. Also, I apologize if my English if a bit off, but since I am French, you will have to excuse my awkward syntaxes sometimes ;). Thanks! Ben	[reply]
Re^3: Perl - Copy files to a newly created folder by dasgar (Priest) on Apr 20, 2015 at 04:48 UTC
I don't want to have to download anything additional to run my program (for the find rule). I'm not sure that I understand your reluctance to install additional modules. One of the strengths of Perl is leveraging the modules available at CPAN. To install File::Find::Rule, you just need to type `cpan -i File::Find::Rule`. But if you're insisting on not installing additional modules, I won't pressure you to do what you don't want to do. Instead, would you have an idea of how to improve my program with what I have right here and my current set of skills? Unfortunately you really haven't provided enough details. I don't know what kind of files that you have that you're trying to copy/move and you're not really providing enough details about what copied and what didn't. Without more information, I'd have to agree with aaron_baugher's suggestions (Re: Perl - Copy files to a newly created folder) as a good starting point for debugging your script. I really appreciate your help but I need something less complicated. Other than possibly learning a new module, I really don't think that the code in my last post was really complicated. In fact, I personally think that trying to use opendir/readdir/closedir is actually a bit more complicated than using File::Find::Rule since you now have to do more work. Here's a modification of my previous code using only core modules and using opendir/readdir/closedir in place of File::Find::Rule. use strict; use warnings; use feature 'say'; use File::Copy; use File::Spec::Functions; use File::Path qw(make_path); use Cwd; # getting the "source" directory my $pwd = cwd(); my $source_dir = catdir($pwd,'source'); my @files; opendir(my $dir,$source_dir) or die "Unable to open directory '$source +_dir': $!"; # search for files in the specified directory while (my $item = readdir($dir)) { # skip . and .. next if ($item =~ m/^\./); # skip directories next if (-d $item); # if this is a file, add it to the @files array my $test = catfile($source_dir,$item); if (-f $test) {push @files,($item);} } closedir($dir); # process the files found foreach my $file (@files) { # get the file's extension my ($extension) = ($file =~ m/.+\.(.+)/); my $subdir = catdir($source_dir,$extension); # if the folder for the extension does not exist, create it if (!(-d $subdir)) {make_path($subdir);} my $source = catfile($source_dir,$file); my $target = catfile($subdir,$file); # copy the file to the appropriate folder copy($source,$subdir); # move the file to the appropriate folder #move($source,$target); } [download] In this version, I'm using a while loop to process through the contents of the directory. In that loop, I'm needing to be aware of and handle the special cases of . and .. plus and I need to ignore directories. Actually, I might have gotten away without checking for those items and using just the -f check. However, I put those additional checks in there, because I have gotten in trouble in the past for not checking them. So I now have a habit of including those checks whenever I use readdir to look at a directory's contents. (I did notice that fishmonger showed a different method for finding files in a directory by using grep, but I thought I'd stick with using readdir since that's what you originally were trying to use.) Also, notice that in the foreach loop that the @files array has only relative file names and not full path file names (i.e. has file.txt and not C:\directory\file.txt). Although I chose to "fix" this in the foreach loop, I could have done the "fix" back in the while loop before adding each file name to the @files array. Let me share something else. I used to use the readdir method to go find items in a directory. But it can get real cumbersome - especially if you wish to search through subdirectories too. I kept seeing a lot of folks recommending File::Find as a better way to handle that kind of task. As I mentioned in my last post, I never could get my head wrapped around how to use File::Find. At some point I discovered File::Find::Rule and I found it to be super easy to understand how to use. Now when I need to go find and process directory contents, I just use File::Find::Rule. Doing so, I can accomplish the search with 10 or less lines instead of using one or more loops that feel complicated and hoping that I have code that handles everything properly. Although you may be more comfortable with using readdir for this current work, I'd recommend that you look into learning File::Find and/or File::Find::Rule at some point.	[reply] [d/l] [select]
Re: Perl - Copy files to a newly created folder by fishmonger (Chaplain) on Apr 18, 2015 at 19:31 UTC
This is the way I'd do it, which only uses core modules. #!/usr/bin/perl use warnings; use strict; use File::Copy; use File::Path qw(make_path); use File::Spec::Functions; use File::Basename; my $dir = 'C:/dev'; my @files = grep { ! -d } <$dir/>; FILE: foreach my $file (@files) { my ($name, $path, $suffix) = fileparse($file, qr/\.[^.]/); $suffix or do { warn "$name does not have an ext\n"; next FILE; }; $suffix =~ s/\.//; # (optional) strip leading . my $subdir = catdir($path, $suffix); make_path($subdir) unless -d $subdir; move($file, $subdir) or warn "Failed to move $file to $subdir <$!> +\n"; } [download] EDIT: If you have spaces in the source path, you will probably need to use quotes when building `@files`. Change `grep { ! -d } <$dir/>` to `grep { ! -d } <"$dir/">`	[reply] [d/l] [select]
Re^2: Perl - Copy files to a newly created folder by Benichouniev (Novice) on Apr 19, 2015 at 09:15 UTC
Fishmonger, thanks for your answer! Your code works perfectly (I changed the `move` to `copy` to be able to test it several times without having to copy the files again). I also used the quotes around `$dir/*` for the `@files` building. Now the thing is, I think I understand how your code works, but could you please explain each line so I know exactly how it runs, step by step? You have been a great help but now I want to know how it works to be able to do it again in the future, if necessary... Could you add `#commentaries like this` on each line to guide my through? That would be extremely helpful! Thanks! Ben	[reply] [d/l] [select]
Re: Perl - Copy files to a newly created folder by aaron_baugher (Curate) on Apr 18, 2015 at 21:45 UTC
There are two possibilities: 1) It's unable to copy some of the files because of a permissions issue or some other filesystem restriction (but this should produce errors), or 2) it's not always doing what you think it's doing. To investigate the second possibility, add some print statements to your code that show what it's doing, like this: `print "Copying '$source' to '$destination'\n"; copy($source,$destination) or die $!;` [download] I put single quotes around the filenames so any whitespace in the names will be apparent. Make sure you print the same arguments that you're passing to the copy() routine, and then you'll be able to see if they are what you expect them to be. This is a very simple debugging method: when your program isn't doing what you expect, check the values of the variables that control its behavior. Aaron B. Available for small or large Perl jobs and *nix system administration; see my home node.	[reply] [d/l]
Re: Perl - Copy files to a newly created folder by Marshall (Canon) on Apr 18, 2015 at 22:44 UTC
Maybe this will help you? I don't see a requirement for recursive copying. That is possible and not that much harder. The basic idea is to create a hash of the .ext's and then use that to either print that table or create a directory for each .ext. Read the directory(s) all at once, create a hash table and use that to do the copies. #!usr/bin/perl -w use strict; my $directory = 'C:/PerlTemp'; opendir (DIR, $directory) or die "unable to open $directory\n"; my %ext2files; #hash of array for each extension #.pl => name, .txt => name etc. foreach my $file (readdir DIR) { next if ($file =~ /^\./); #skip . and .. directories my ($ext) = ($file =~ /\.(\w+)$/)[0]; #get "xxx/yyy.ext" next unless defined $ext; # no ".extension" #now keep track of .ext to files... push @{$ext2files{$ext}},$file; } foreach my $ext (sort keys %ext2files) { # this prints the .ext ending and the number of files. # More than one syntax can do this, but I hope that # this is understandable... print "$ext \t", scalar @{$ext2files{$ext}}," files\n"; #it is possible to use the printf() formatting strings #in Perl. I didn't use that here, but it is possible. } __END__ PL 3 files SCP 1 files TXT 1 files bak 1 files bat 4 files bin 1 files c 28 files cav 1 files cmdline 1 files cpp 1 files csv 4 files dat 8 files db 4 files dbi 1 files doc 4 files exe 23 files h 2 files html 9 files ico 1 files ini 1 files jpg 1 files new 1 files out 2 files p 1 files pdf 3 files pl 1113 files pll 2 files pm 2 files script 1 files stackdump 3 files temp 1 files txt 173 files txtC 1 files xls 1 files xml 6 files zip 2 files Process completed successfully [download]	[reply] [d/l]
Re: Perl - Copy files to a newly created folder by Anonymous Monk on Apr 18, 2015 at 20:55 UTC
But this is where it infuriates me: It only copies certain files! Give us some examples of files that it copies and doesn't copy (that is, file names, permissions - not the contents of files, of course).	[reply]
Re^2: Perl - Copy files to a newly created folder by Benichouniev (Novice) on Apr 19, 2015 at 09:10 UTC
I already checked the file names, contents (whether they were blank files or not), properties, security options (read only, etc.), and ALL the files have the same properties. I tried changing the file names to all lower case but did not change anything, same with spaces... In my folder for example, there are two .txt files : detail.txt and notes.txt. When it creates the txt subfolder, it only copies detail.txt to it. I don't understand. Thanks for your answer!	[reply]