atcroft has asked for the wisdom of the Perl Monks concerning the following question:

I was experimenting with code (below the READMORE tag) that would seek the images present in a subdirectory or subdirectories and present them using Tk::Thumbnail, and have encountered quite the problem. When I run it on a directory tree containing of 20-30 JPEGS (900x1200 or 1600x1200, approximately 400-500KB in size each), no issues; however, when I run it against a directory tree containing 100-150 similar images, it locks my machine because of a lack of memory (512MB RAM, and approximately the same for swap space). I suspect it is in part because of the processing to create the thumbnails (and considered trying to use other means, such as Image::GD::Thumbnail, but no joy).

Any suggestions/assistance appreciated.

#!/usr/bin/perl -w use strict; use vars qw($ft @filelist @typelist); use File::Find; use File::Glob qw(:glob); use File::Type; use GD; use Image::GD::Thumbnail; use Image::Size; use Tk; use Tk::JPEG; use Tk::PNG; use Tk::TIFF; use Tk::Thumbnail; use Tk::Stderr; $| = 1; my (@directories_to_search); foreach my $dir ( ( scalar(@ARGV) > 0 ? @ARGV : '.' ) ) { push( @directories_to_search, bsd_glob( $dir, GLOB_TILDE | GLOB_ERR ) ); } print join( "\n", @directories_to_search ), "\n\n"; $ft = File::Type->new(); find( { wanted => \&wanted, follow_skip => 2, no_chdir => 1 }, @directories_to_search ); print "\n"; my (%images); { my @templist = sort { $a cmp $b } @filelist; @filelist = @templist; } my $mw = MainWindow->new(); my $stderrw = MainWindow->new->InitStderr; $mw->title($0); { my (@imagelist); foreach ( 0 .. $#filelist ) { print $filelist[$_], "\n"; my $temp = $mw->Photo( -file => $filelist[$_] ); push( @imagelist, $temp ); } eval { my $thumb = $mw->Thumbnail( -images => [@imagelist], -ilabels => 1 )->pack; }; warn($@) if ($@); } MainLoop; sub wanted { return unless ( -R $File::Find::dir ); return unless ( defined($File::Find::name) ); return unless ( -f $File::Find::name ); return unless ( -R $File::Find::name ); my ($type_from_file); eval { $type_from_file = $ft->checktype_filename($File::Find::name); printf "Unknown type: %s\n", $File::Find::name unless ( defined($type_from_file) ); }; return unless ( defined($type_from_file) ); if ( $type_from_file =~ m/image/ ) { push( @filelist, $File::Find::name ); push( @typelist, $type_from_file ); } }

Replies are listed 'Best First'.
Re: Memory issue while experimenting with Tk::Thumbnail
by SciDude (Friar) on May 30, 2004 at 00:49 UTC

    In the "TK Gotchas" section of this tutorial page you will find the following comment:

    if you create a Canvas with Photos in it, it appears to be impossible to destroy them without leaking memory, no matter how many lexical scopes you leave or destroys and deletes you scream at the interpreter. The only work around I found was to load the images in, and simply hide them when not required.

    Hopefully a TK guru will post a more concise solution to this problem.

    On a related note, Steve Cook has a wonderful set of introductory tutorials related to perl/tk. See his pages for details.

    SciDude
Re: Memory issue while experimenting with Tk::Thumbnail
by zentara (Cardinal) on May 30, 2004 at 14:14 UTC
    I ran your code, and see what you mean. Without alot of testing, it appears that you are keeping all the large full-sized photo objects, and that takes quite a bit of memory. In your code below, your are pushing the full-photo-objects into @imagelist. Then you are letting Tk::Thumbnail to make the thumbnails from them. You are wasting a heck of alot of memory in @imagelist.
    { my (@imagelist); foreach ( 0 .. $#filelist ) { print $filelist[$_], "\n"; my $temp = $mw->Photo( -file => $filelist[$_] ); push( @imagelist, $temp ); } eval { my $thumb = $mw->Thumbnail( -images => [@imagelist], -ilabels => 1 )->pack; }; warn($@) if ($@); }
    So I would say you have a basic design problem. From my quick look, I would say you need to decide how many thunbs you want to display as a maximum. Then change your thumbnail generation method, and maybe "page them".

    With Photo objects, you need to create a set of them once, and REUSE them. You can reuse a photo object with

    $thumbs{0}->blank; $thumbs{0}->read($info{'admin'}{'thumbnail'});
    So you could create a hash of %thumbs of say 100 keys, and that is what you display on each page....a 10 x 10 array of thumbs. And have a pager mechanism, to display as many pages as you need. But you don't fill in those subsequent pages, what you do is reuse the %thumbs and just reconfigure them with the next batch of thumbs.

    Also when you create the thumbs, just make 1 "big-photo object" and reuse it in a loop, to make the thumb-objects. Then push the smaller "thumb-objects" into the @imagelist, instead of the huge full-photo-objects.

    Here is a sample bit of code which shows how to make thumbnails in a loop, with just a single imager object. Now, I'm actually creating the thumbnails in the thumbs dir, but you could just push them into a hash. However....you will still reach memory limits if you try to push too many thumbs into a hash. So you must carefully think about juggling and reusing everything, so as to keep your mem use as low as possible. There are many ways to do this, so you have to find the modules which you like working with, and figure out how to use them in unison, without wasting objects.

    sub make_thumbs{ my @pics = <pics/*.jpg pics/*.gif pic/*.png>; my $imager = Imager->new(); foreach my $pic (@pics){ my ($basename,$path,$suffix) = fileparse($pic,@exts); $info{$basename}{'key'} = $basename; $info{$basename}{'name'} = $basename; $info{$basename}{'pic'} = "pics/$basename.jpg"; #convert to jpg $info{$basename}{'thumbnail'} = "thumbs/$basename.jpg"; $imager->open(file=>$pic) or die $imager->errstr(); # Create smaller version my $thumb = $imager->scale(xpixels=>100); print "Storing thumbnail as: thumbs/$basename.jpg\n"; $thumb->write(file=>"thumbs/$basename.jpg", jpegquality=>30) or die $thumb->errstr; #Create resized picture my $resizer = $imager->scale(ypixels=>400); print "Storing resize as: pics/$basename.jpg\n"; $resizer->write(file=>"pics/$basename.jpg", jpegquality=>90) or die $resizer->errstr; } undef $imager; print "\n#################endofthumbcreation########################## +#####\n"; }

    I'm not really a human, but I play one on earth. flash japh
Re: Memory issue while experimenting with Tk::Thumbnail
by andyf (Pilgrim) on May 30, 2004 at 13:36 UTC
    Within your main thumbnail processing loop try creating a image object, doing the thumbnail, then explicitly clean up by calling $image->destroy() and undef $image.
    Sure, it will be significantly slower (you make a fresh instance, use it, and dispose of it each time), but you should not leak memory. Update: This worked for me with a very similar problem using ImageMagic, but from the above reply it seems there is an intrinsic fault with destroy() in Tk.
Re: Memory issue while experimenting with Tk::Thumbnail
by zentara (Cardinal) on May 30, 2004 at 19:13 UTC
    Well I was just flipping thru my Mastering Perl/Tk and came across this code, which almost does what you want, just with Tk::Thumbnail. I just globbed for all jpgs , but it uses a small amount of memory, and you should be able to figure out how to work your file-find routine into it.

    It also has a command to identify the thumbnail which you could use to open the photo in another toplevel window when you click on a thumb.

    #!/usr/bin/perl -w use Tk; use Tk::Thumbnail; use strict; #adapted from Mastering Perl/Tk my $mw = MainWindow->new; my $tn = $mw->Thumbnail( -images => [<*.jpg>], -command => sub { my $i = $_[0]->cget(-image); print "args=@_, image=$i\n"; }, @ARGV); $tn->pack; MainLoop;

    I'm not really a human, but I play one on earth. flash japh