in reply to Grouping files before zipping!

There's a few problems. The reason it is going into an infinite loop: there is nothing incrementing the $next_index variable. You increment while finding the first file small enough, but then never do again.

Another issue is that by the time you try to use the @files array, it is an array of file sizes, not file names. Fine to find a group of file sizes under a limit, but no way to actually access the file names to add them to the zip file.

Thirdly, you are pushing elements onto the end of an array, then using the first array element in calculations. You probably should use the element just pushed instead.

Update: Oh yeah, the splice length needs to be non zero also, or it won't modify the array.

Putzing around a bit... (note: I do not particularly recommend this, just fixed to be a working example. No guarantees about correct logic.)

#!/usr/bin/perl -w

use strict;
use File::Find;
use File::Basename;
use File::stat;
use File::Slurp qw(read_dir);

my @directories = qw( /all_files_a /all_files_b);
my @files;

my $min_size = 1024; # bytes => 1K
my $max_size = 10485760; # bytes =>10MB

for my $d (@directories) {    
    push @files, { name => "d/$_", size => -s $_ } for  grep { -f && -
+s _ >= $min_size && -s _ <= $max_size && /\.pdf/} read_dir( $d, prefi
+x => 1 );
}

my $limit = 5_000_000;

#Create zip with $limit of files per zip
print "Creating zips...\n";
my $zip_number = 1;
my $total_size = 0;
my @other_group;
my $next_index = 0;

while(@files) {
    #Find biggest file that will fit
    $next_index++ while( ($next_index <= $#files) && ($total_size + $f
+iles[$next_index]->{size} > $limit) );
        
    #If there was a file that will fit, add it to @other_group        
    if($next_index <= $#files) {
          push @other_group, splice(@files, $next_index, 1);
          $total_size += $other_group[-1]->{size};
          $next_index++;
    }
    #Otherwise, zip this group, and start a new group
    else {
        # print this for testing, these files will be zipped in the $z
+ip_number.zip file
        foreach my $testfiles (@other_group) { 
             print $testfiles->{name}," - $zip_number.zip\n";
        }        
        
        #Clear this group info, and increment zip number
        @other_group = ();
        $total_size = 0;
        $zip_number++;
        $next_index = 0;
    }
}

[download]

Replies are listed 'Best First'.
Re^2: Grouping files before zipping!
by Anonymous Monk on Sep 09, 2011 at 13:50 UTC
    After some research on another way of doing this I found that it could be done using Archive::Zip. I have a sample code that does check the size of a zip before zipping all the files in a directory, it only needs to be implemented to build multiple zip files if the files size exceed the permitted volume. Anyone? 
    use strict;
use warnings;

use Archive::Zip qw/AZ_OK/;
use File::Temp qw/tempfile/;

use constant MB => 1024 * 1024;

my $dir = '/allfiletozip/';
my @files = do {
 opendir my $fd, "$dir" or die $! or die $!;
 grep -f, map  "$dir$_", readdir $fd;
};

my $zip = Archive::Zip->new;
my $total;
my $limit = 50*MB;

foreach my $file (@files) {

 my $temp = Archive::Zip->new;

 my $member = $temp->addFile($file);
 next unless $member->compressedSize;

 my $fh = tempfile();
 $temp->writeToFileHandle($fh) == AZ_OK or die $!;
 
 $zip->addMember($member);
 $total += $member->compressedSize;
 die "$total bytes exceeds archive size limit" if $total > $limit;
}

print "Total archive size: $total bytes\n\n";

$zip->writeToFileNamed('zipped.zip') == AZ_OK or die $!;

    [download]

    Thanks!
[reply]
[d/l]

In Section Seekers of Perl Wisdom