ckaspar has asked for the wisdom of the Perl Monks concerning the following question:

I have an array

@tmp qw/1.zip 2.zip 3.zip 4.zip 5.zip 6.zip 7.zip 8.zip 9.zip/;

1. I need to search a location to find each zip file and size of each zip file.

2. Copy the files over to a new location.

3. If the directory reaches 4gb's, make a new directory and start to populate it, if that reaches a limit make another, and so on.

E.G.

my @tmp = qw/1.zip 2.zip 3.zip 4.zip 5.zip 6.zip 7.zip 8.zip 9.zip/;

Directory location:

Source-----                                      Destination
/home/svr/1.0                                  
/home/des/Disk1/Linux                  
                     /CD_RH_1
                     /CD_RH_2
                     /CD_RH_3
                     /CD_RH_4
foreach (@tmp) to find

/home/svr/1.0/CD_RH_2/1.zip => 2 gb [copy to destination] 
/home/svr/1.0/CD_RH_1/2.zip => 1 gb [copy to destination]
--------------So only 1.zip and 2.zip can fit in /home/des/Disk1/Linux
                  Adding 3.zip would go over 4 gb 
--------------mkdir  /home/des/Disk2/Linux
/home/svr/1.0/CD_RH_3/3.zip  => 2gb [copy to destination]
--------------only 3.zip can ! fit in /home/des/Disk2/Linux
                  Adding 4.zip would go over 4 gb 
--------------mkdir  /home/des/Disk3/Linux
/home/svr/1.0/CD_RH_4/4.zip => 2.5gb [copy to destination]
/home/svr/1.0/CD_RH_2/5.zip => .5mb [copy to destination]
/home/svr/1.0/CD_RH_1/6.zip => .25gb [copy to destination]
/home/svr/1.0/CD_RH_3/7.zip => 112.5mb [copy to destination]
/home/svr/1.0/CD_RH_4/8.zip => 22.5mb [copy to destination]
/home/svr/1.0/CD_RH_4/9.zip => 122.5kb [copy to destination]
--------------copy 4.zip 5.zip 6.zip 7.zip 8.zip 9.zip to /home/des/Di
+sk3/Linux

[download]

Code tags added by GrandFather and much editing of misguided HTML

Code I have This is the code I have.

I now need to use my @tmp = qw/1.zip 2.zip 3.zip 4.zip 5.zip 6.zip 7.zip 8.zip 9.zip/; with this to find each element in /home/svr/1.0 copy to /home/des/Disk$dest/Linux and if /home/des/Disk$dest/Linux reaches 4gb make another and start coping. The copy has to be in the order of the array.

  opendir(DIR,"/home/svr/1.0"); 
  @dir_list = grep(/^RH_CD\d+$/,readdir(DIR)); 
 
  closedir(DIR); 
  foreach $dir (@dir_list) 
  { 
   opendir(DIR,"/home/svr/1.0/$dir"); 
 
   @file_list = grep(!/^\.\.?$/,readdir(DIR)); 
 
   closedir(DIR); 
   foreach $file (@file_list) 
   { 
     $filename = "/home/svr/1.0/$dir/$file"; 
 
     $base{"$dir/$file"} = $file; 
     if (-f $filename) 
     { 
       $filesize{"$dir/$file"} = -s $filename; 
     }elsif (-d $filename) 
     { 
       chomp($filesize{"$dir/$file"} = qx(du -sk $_)); 
       $filesize{"$dir/$file"} *= 1024; 
     } 
   } 
 } 
 $limit = 4*1024*1024*1024; 
 $dest = 1; 
 chomp($dest_size = qx(du -sk "/home/des/Disk1$dest/Linux")); 
 
 $dest_size *= 1024; 
  while (%filesize) 
 { 
   $got_onfiltered= 0; 
   foreach (sort {$filesize{$a} <=> $filesize{$b}} 
  keys %filesize) 
   { 
     last if $dest_size + $filesize{$_} > 
  $limit; 
     system("cp -Rp /home/svr/1.0/$_ /home/des/Disk$dest/Linux/$base{$
+_}"); 
 
  $dest_size = $dest_size + $filesize{$_};
     delete $filesize{$_}; 
     $got_onfiltered= 1; 
   } 
   if (!$got_one) 
   { 
     if ($dest_size == 0){ 
       print "ERROR: all remaining files too big:\n"; 
       foreach (keys %filesize) 
       { 
         print "$_: $filesize{$_}\n"; 
 
       } 
       die; 
     }else{ 
       $dest++ 
       mkdir ("/home/des/Disk$dest/Linux)
       $dest_size = 0; 
     } 
   } 
 }

[download]

20060317 Corion: Un-htmlified addendum

Replies are listed 'Best First'.
Re: Using an array, how do I search one directory area and copy to another area
by Tanktalus (Canon) on Mar 18, 2006 at 04:02 UTC

    I've done exactly the same thing with CD sizes of 650MB. It's not hard - what have you tried?

    Hints: -s will get you the size. You may want to use cog's Filesys::DiskUsage instead - I do.

    Updated: Fixed the link to Filesys::DiskUsage - woops.

[reply]
Re: Using an array, how do I search one directory area and copy to another area
by Corion (Patriarch) on Mar 18, 2006 at 14:51 UTC

    You already mentioned in the CB that you're trying to use File::Find for that solution, and I really recommend it. The below skeleton should get you closer to your solution:

    use strict;
use File::Find;

my $limit = 4 * 1024 * 1024 * 1024;

my @source_directories = qw(
);

my @files;

File::Find::find( sub { push @files, $File::Find::name }, @source_dire
+ctories );

my $total;
my $basename = "part-";
my $count = 1;
my $dirname = $basename . $count;

for my $file (@files) {
    my $size = -s $file;
    if ($total + $size > $limit) {
        $total = 0;
        $count++;
        $dirname = $basename . $count;
        print "# 4GB Limit reached, starting new volume $dirname";
    };
    print "$file => $dirname/$file\n";
};

    [download]
[reply]
[d/l]

      Please note that simply looking at a file's size does not do the same thing as du. The du command will tell you the amount of disk space that each file is using by rounding up to the current sector size. This discrepency was the whole purpose of cog's Filesys::DiskUsage 0.04 version - to allow you to better approximate what the du command does. Note that you have to tell Filesys::DiskUsage what sector size to use, which is moderately (but not completely) unfortunate.

      This is completely fortunate, however, in the OP's scenario. I'm betting that the 4GB boundary is for DVDs. And DVDs will not have the same sector size as their ext3 or jfs or ntfs or whatever filesystem. So what du thinks is the total space and what it will be on the DVD may not match. It may be close enough, but maybe not.

      As I said above, I did exactly the same thing for CDs. And the sector size on CDs is 1K. Which doesn't often match the sector size on hard disks. So I use Filesys::DiskUsage and pass in a sector-size of 1024, and I know what the disk usage will be on my target media. In my case, the difference will be dozens of MB between what du reports on the hard disk and what it would report on CDs, and that difference could significantly alter the layout of the CDs, possibly even reducing the number of CDs that I need by one - multiplied over the cost of manufacturing, it can save some real dollars, not only likely saving the cost of developing that patch and using it, but also reducing the weight of the box the CDs are in, and producing less ill will from customers who don't like how many CDs we already have in those boxes - that's worth real money, too.

[reply]
Re: Using an array, how do I search one directory area and copy to another area
by ww (Archbishop) on Mar 18, 2006 at 14:55 UTC
[reply]

Back to Seekers of Perl Wisdom