Excellent demonstration as usual, marioroy!
Well, just slightly offtopic, w/r/t/ the efforts
to speed up animated GIF generation
the disturbing truth is that the main time-waster, above, appears to be Imager collecting, in single thread, 999(9...) decoded GIF images into final animation. Just look what it does: GIF files/scalars are read into internal Imager format (thankfully, kept palletized i.e. not converted to "true color"), the LZW compression is discarded/forgotten, it apparently tries palette(s) optimization(?), then individual frames are LZW-compressed again, very likely to the state they were initially stored in files/scalars. What a waste. To be fair, depending on image/scene, quantization requires much more work than LZW compression, so parallelization, as is, pays off well. However, one can't help but wonder if it all can be accelerated.
Actually, yes, some tools appear to be more fit for the job, -- GD. It also, I think, decompresses GIF frames/scalars, but then either keeps (and then uses) originals, or is just optimized 100% ++.
The example below is marioroy's code, with a few changes to setup. I run it on 4 cores, image is slightly larger (to give a computer some work to do, maybe closer to real life use) and has "interesting" background, otherwise compression cancels our larger image size, and it may again be closer to kind of images/animations OP is working with, it seems to me, -- or at least that was an idea. Number of frames, chunk size, etc. are changed according to setup above.
use strict; use warnings; use feature 'say'; use Imager; use MCE::Loop; use Time::HiRes 'time'; use GD; STDOUT->autoflush; my $start = time; my $count = 0; my @data; my $bkg; my $TEST_GD = $ARGV[ 0 ] // 1; MCE::Loop->init( max_workers => MCE::Util::get_ncpu(), chunk_size => 40, init_relay => '', gather => sub { if (@_ == 1) { print "\r", $count++; } else { push @data, @{ $_[1] }; } }, user_begin => sub { $bkg = Imager->new(xsize=>800, ysize=>600); $bkg->filter(type=>"gradgen", xo=>[ 100, 300, 600 ], yo=>[ 100, 300, 100 ], colors=>[ qw(red blue green) ]); $bkg->filter(type=>"noise", amount=>12, subtype=>0) }, ); mce_loop { my ($mce, $chunk_ref, $chunk_id) = @_; my @i_data; for my $x (@{ $chunk_ref }) { my $i = $bkg-> copy; $i->string( text => $x, color => Imager::Color->new('ffffff'), font => Imager::Font->new( # file => '/usr/share/fonts/truetype/msttcorefonts/cour.ttf', # file => '/System/Library/Fonts/Courier.dfont', face => 'Courier New', # mswin size => 420, aa => 1), x => 25, y => 500, ); $i->write(data => \my $data, type => 'gif'); push @i_data, $data; MCE->gather($x); } MCE::relay { MCE->gather($chunk_id, \@i_data) }; } [ 0 .. 319 ]; MCE::Loop->finish; print " frame GIF done!\n"; printf "compute time: %0.3fs\n", time - $start; if ( $TEST_GD ) { my $image = GD::Image-> newFromGifData( $data[ 0 ]); my $gifdata = $image-> gifanimbegin(0,0); $gifdata .= $image-> gifanimadd( 1,0,0,1,1 ); for (1..$#data) { my $frame = GD::Image-> newFromGifData( $data[ $_ ]); $gifdata .= $frame-> gifanimadd( 1,0,0,1,1 ); } $gifdata .= $image-> gifanimend; open my $fh, '>', 'gd.gif'; binmode $fh; print $fh $gifdata; close $fh; } else { Imager->write_multi({ file => 'im.gif', type => 'gif', gif_loop => 0, gif_delay => 1, }, map { Imager->new(data => \$_) } @data) or die Imager->errstr; } printf "Total: %0.3fs\n", time - $start; __END__ 319 frame GIF done! compute time: 10.932s Total: 45.646s 319 frame GIF done! compute time: 10.738s Total: 17.107s
1st is testing (final animation to be collected by) Imager, 2nd -- GD. Nice. But then, from here it's natural to also parallelize what's being collected in "$gifdata" -- staying in the same "mce_loop" block! 2 fragments: (1) a "push" to be replaced with (should be re-written better, it's too late here already):
if ( $TEST_GD ) { my $gd = GD::Image-> newFromGifData( $data ); push @i_data, !$x ? $gd-> gifanimbegin . $gd-> gifanimadd( 1,0,0,1,1 ) : $gd-> gifanimadd( 1,0,0,1,1 ) } else { push @i_data, $data }
and (2) final "if" to be replaced with:
if ( $TEST_GD ) { open my $fh, '>', 'gd.gif'; binmode $fh; print $fh join '', @data; close $fh; } else { Imager->write_multi({ file => 'im.gif', type => 'gif', gif_loop => 0, gif_delay => 1, }, map { Imager->new(data => \$_) } @data) or die Imager->errstr; } __END__ 319 frame GIF done! compute time: 11.765s Total: 12.015s
Now that's some "speeding up animated GIF generation". The whole test -- using Imager to generate a GIF, then GD to re-save -- is maybe somewhat artificial as an example, and may or may not be close to "real life". Just to show that parallelization (== optimization) helps as much as other tools/setup allow.
In reply to Re^5: MCE segmentation fault
by vr
in thread MCE segmentation fault
by Anonymous Monk
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |