in reply to Re^2: Testing image output
in thread Testing image output

omg, ain't GD so very difficult. I'm looking at Image-Square 0.01_4 testers matrix, what was supposed to be walk in the park is like blood covered battlefield.

Half failures are from gd native output format unsupported, who could expect. I'm sorry. It isn't really a problem, because ".gd" is just 11 bytes header plus raw data:

use strict;
use warnings;
use feature 'say';
use GD;
use Digest::MD5 'md5_hex';

say $^V; 
say $GD::VERSION;
say eval { GD::VERSION_STRING() } || '-';

GD::Image-> trueColor( 1 );

my $fn = 'CoventryCathedral.png';
my $i = GD::Image-> new( $fn );

use constant W => 100;

my $j = GD::Image-> new( W, W );

$j-> copyResampled( $i,
    0, 0,
    ( $i-> width - $i-> height ) * .5, 0,
    W, W, 
    $i-> height, $i-> height );

say eval { md5_hex( $j-> gd )} || '-';
say md5_hex( my_gd( $j ));

sub my_gd {             # same as gd() for truecolor images
    my $gd = shift;

    my ( $w, $h ) = $gd-> getBounds;
    my $s = '';
    for my $y ( 0 .. $h - 1 ) {
        for my $x ( 0 .. $w - 1 ) {
            $s .= pack 'L>', $gd-> getPixel( $x, $y );
        }
    }
    return "\xff\xfe" .
           ( pack 'S>2', $w, $h ) .
           "\1\xff\xff\xff\xff" .
           $s
}

__END__

v5.38.0
2.78
2.3.2
-
c97e63fc792ef75b5ff49c078046321e

v5.32.1
2.76
2.2.5
c97e63fc792ef75b5ff49c078046321e
c97e63fc792ef75b5ff49c078046321e

v5.24.3
2.66
2.1.1
adc191aea66fdf99fd74aaeb20b34e5e
adc191aea66fdf99fd74aaeb20b34e5e

[download]

Note, one checksum is exactly what "t/02-image.t line 41" was expecting, but the latter is what many (but not all) failures have "got".

It appears that copyResampled (and interpolation in general, see further) is unstable between versions and plagued with bugs. Then, even generating synthetic gradient or whatever, and checking for just couple of pixels (e.g. lower left and upper right points) is NOT reliable way to test anything with GD, let alone calculating checksum over whole re-sampled image.

No CoventryCathedral for tests below, simply a red 8 by 8 square to reduce to smaller squares:

use strict;
use warnings;
use feature 'say';
use GD;

say $^V; 
say $GD::VERSION;
say eval { GD::VERSION_STRING() } || '-';

GD::Image-> trueColor( 1 );

my $i = GD::Image-> new( 8, 8 );
$i-> filledRectangle( 0, 0, 7, 7 ,$i-> colorAllocate( 255, 0, 0 ));

for my $w ( 1 .. 7 ) {
    my $j = GD::Image-> new( $w, $w );
    $j-> copyResampled( $i, 0, 0, 0, 0, $w, $w, 8, 8 );

    print "\t\t\t$w\n";
    for my $y ( 0 .. $w - 1 ) {
        for my $x ( 0 .. $w - 1 ) {
            my ( $r ) = $j-> rgb( $j-> getPixel( $x, $y ));
            printf '%x ', $r;
        }
        print "\n";
    }
}

__END__

v5.38.0
2.78
2.3.2
                        1
ff
                        2
ff ff
ff ff
                        3
fe fe fe
fe fe fe
ff fe ff
                        4
ff ff ff ff
ff ff ff ff
ff ff ff ff
ff ff ff ff
                        5
fe ff ff fe ff
ff fe ff ff ff
fe fe ff ff ff
fe fe ff ff fe
ff ff ff fe ff
                        6
fe ff ff ff ff ff
ff ff ff ff ff fe
ff ff ff fe ff fe
ff ff ff ff ff ff
ff fe ff ff ff fe
ff fe fe ff fe ff
                        7
fe ff ff fe fe ff fe
ff ff ff ff ff ff fe
fe ff ff ff ff ff ff
ff ff ff ff ff ff ff
fe ff ff ff ff ff ff
ff ff ff ff ff ff ff
ff fe ff ff ff ff ff

v5.24.3
2.66
2.1.1
                        1
ff
                        2
ff ff
ff ff
                        3
ff ff ff
ff ff ff
ff ff ff
                        4
ff ff ff ff
ff ff ff ff
ff ff ff ff
ff ff ff ff
                        5
fe fe fe fe fe
fe fe ff fe fe
fe fe ff fe ff
ff fe fe ff fe
fe fe ff fe ff
                        6
ff ff ff ff ff ff
ff fe ff ff ff ff
ff ff ff ff ff ff
ff ff ff ff ff ff
ff fe ff ff ff ff
ff ff ff ff ff ff
                        7
fe fe ff fe ff fe fe
fe ff fe ff ff fe ff
ff fe ff ff ff fe fe
fe ff ff ff fe ff ff
ff ff ff fe fe ff fe
fe ff ff ff ff ff ff
fe ff ff ff fe ff fe

[download]

Oh, I thought, but I'm copying red pixels to another (smaller) canvas, filled with default black. Maybe, instead, plain simple resize would preserve pure red colour? Note, plain "resize" was not implemented in old versions anyway.

use strict;
use warnings;
use feature 'say';
use GD;

say $^V; 
say $GD::VERSION;
say eval { GD::VERSION_STRING() } || '-';

GD::Image-> trueColor( 1 );

my $i = GD::Image-> new( 8, 8 );
$i-> filledRectangle( 0, 0, 7, 7 ,$i-> colorAllocate( 255, 0, 0 ));

for my $w ( 1 .. 7 ) {
    my $j = $i-> copyScaleInterpolated( $w, $w );

    print "\t\t\t$w\n";
    for my $y ( 0 .. $w - 1 ) {
        for my $x ( 0 .. $w - 1 ) {
            my ( $r ) = $j-> rgb( $j-> getPixel( $x, $y ));
            printf '%x ', $r;
        }
        print "\n";
    }
}

__END__

v5.38.0
2.78
2.3.2
                        1
ff
                        2
ff ff
ff ff
                        3
ff ff ff
ff fd fd
ff fd fd
                        4
ff ff ff ff
ff ff ff ff
ff ff ff ff
ff ff ff ff
                        5
ff ff ff ff ff
ff fd fd fd fd
ff fd fd fd fd
ff fd fd fd fd
ff fd fd fd fd
                        6
ff ff ff ff ff ff
ff fd fd fd fd fd
ff fd fd fd fd fd
ff fd fd fd fd fd
ff fd fd fd fd fd
ff fd fd fd fd fd
                        7
ff ff ff ff ff ff ff
ff fd fd fd fd fd fd
ff fd fd fd fd fd fd
ff fd fd fd fd fd fd
ff fd fd fd ff fd fd
ff fd fd fd fd fd fd
ff fd fd fd fd fd fd

[download]

Wait, but there are a few dozen interpolation methods:

use strict;
use warnings;
use feature 'say';
use GD;

say $^V; 
say $GD::VERSION;
say eval { GD::VERSION_STRING() } || '-';

GD::Image-> trueColor( 1 );

my $i = GD::Image-> new( 8, 8 );
$i-> filledRectangle( 0, 0, 7, 7 ,$i-> colorAllocate( 255, 0, 0 ));

my @ok_methods;

for my $m ( 1 .. 30 ) {
    eval {
        for my $w ( 1 .. 7 ) {
            $i-> interpolationMethod( $m );
            my $j = $i-> copyScaleInterpolated( $w, $w );

            for my $y ( 0 .. $w - 1 ) {
                for my $x ( 0 .. $w - 1 ) {
                    my ( $r ) = $j-> rgb( $j-> getPixel( $x, $y ));
                    die unless 255 == $r;
                }
            }
        }
        1;
    } or next;
    push @ok_methods, $m;
}

say 'looks like ok methods are: ', join ' ', @ok_methods;

__END__

v5.38.0
2.78
2.3.2
looks like ok methods are: 1 2 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2
+0

[download]

I have no idea why 3,4,5 i.e.

GD_BILINEAR_FIXED,
GD_BICUBIC,
GD_BICUBIC_FIXED,

are not ok i.e. don't preserve dumb uniform fill of dumb square canvas. I'd laugh out load if asked will this list stay stable for near future. I have much sympathy for GD, but above was a little bit too much.

use strict;
use warnings;
use feature 'say';
use Imager;

my $i = Imager-> new( xsize => 8, ysize => 8 );
$i-> box( filled => 1, color => Imager::Color-> new( 255, 0, 0 ));

for my $w ( 1 .. 7 ) {
    my $j = $i-> scale( xpixels => $w,
#                        qtype => 'mixing',
#                        qtype => 'preview',
    );
    
    print "\t\t\t$w\n";
    for my $y ( 0 .. $w - 1 ) {
        for my $x ( 0 .. $w - 1 ) {
            my ( $r ) = $j-> getpixel( x => $x, y => $y )-> rgba;
            printf '%x ', $r;
        }
        print "\n";
    }
    
}

__END__

                        1
ff
                        2
ff ff
ff ff
                        3
ff ff ff
ff ff ff
ff ff ff
                        4
ff ff ff ff
ff ff ff ff
ff ff ff ff
ff ff ff ff
                        5
ff ff ff ff ff
ff ff ff ff ff
ff ff ff ff ff
ff ff ff ff ff
ff ff ff ff ff
                        6
ff ff ff ff ff ff
ff ff ff ff ff ff
ff ff ff ff ff ff
ff ff ff ff ff ff
ff ff ff ff ff ff
ff ff ff ff ff ff
                        7
ff ff ff ff ff ff ff
ff ff ff ff ff ff ff
ff ff ff ff ff ff ff
ff ff ff ff ff ff ff
ff ff ff ff ff ff ff
ff ff ff ff ff ff ff
ff ff ff ff ff ff ff

[download]

Replies are listed 'Best First'.
Re^4: Testing image output
by Bod (Parson) on Sep 14, 2023 at 22:53 UTC
    omg, ain't GD so very difficult. I'm looking at Image-Square 0.01_4 testers matrix, what was supposed to be walk in the park is like blood covered battlefield.

    I know my knowledge of images is lacking but I was beginning to feel I had done something terribly wrong...

    No CoventryCathedral for tests below, simply a red 8 by 8 square to reduce to smaller squares

    Isn't the whole point of the tests to check that the module does what it is supposed to in real situations?

    Users of the module (me if I am the only one) will be using ti to process large, if not huge, images. If it passes the tests on little tiny images but fails on large ones, doesn't that sort of render the tests meaningless?

    My original tests were based on those in Image::Resize in the < href="https://metacpan.org/release/SHERZODR/Image-Resize-0.5/source/t/1.t"1.t</code> file which just checks the dimensions of the generated file. But that module doesn't crop images which is why I wanted to include tests of the actual output.

    It appears that copyResampled (and interpolation in general, see further) is unstable between versions and plagued with bugs

    I did originally use copy but changed to copyResampled when I decided it would be sensible to add the facility to change the size at the same time...

[reply]
[d/l]
[select]
      Isn't the whole point of the tests to check that the module does what it is supposed to in real situations?

      To some extent, the "large image" tests that you are suggesting have nothing to do with the behavior of the specifics of your module, because all the logic of your module can be tested with a 3x1 and a 1x3 image. Whether GD::Image::copyResampled works under every situation is outside of your control -- but it is to be expected that libgd and/or the GD::Image wrapper around that library together have sufficient tests on copyResampled to ensure that it works in any situation they claim it will work.

      The copyResampled docs say, "using a weighted average of the pixels of the source area rather than selecting one representative pixel" -- what happens if they decided to slightly tweak the weights used between one version of the library and another, because it gives similar but slightly better results from the visual perspective on certain images? Your test would be looking for the specific results based on factors outside of your control, so two different machines with different versions of libgd might give a different signature, even though the image is still reasonably resized and resampled.

      Because the underlying libgd library can be changed without changing the GD wrapper distribution, or the GD wrapper distribution could be changed without changing libgd, you cannot (even if it is possible) just restrict your module to require an exact version of GD -- because the version of GD doesn't guarantee a specific version of the library. So you have no way of restricting to a specific libgd version, and thus no way to guarantee that the underlying behavior of that function will always deterministically give a single known output for a given input, across all versions of the library that might be on a user's machine now or in the future.

      The two three-pixel images that I suggested in my github comment seem to me to be immune to such differences, because if you specify the squares so they land on exact pixel boundaries (which is what I did), there should never be a need for averaging/interpolating, and so as far as I can tell, they would not be affected by any reasonable changes to the copyResampled algorithm -- though I might be proven wrong at some point if they got really "creative" with their implementation.

      If you are really worried about large images behaving weirdly, I can think of a few options:

      1. Just have one or more large images on your development PC, that you use to test, but don't have them in the repo and don't distribute them when you release (so neither GH Actions nor cpantesters would get them); that way, you can see that they, in general, work, but rely on the test suite of libgd/GD to provide confidence that if it works locally, it will work with reasonable results (though possibly not exactly-equivalent results) on other platforms and/or other versions
      2. Have the two large images in your github repo, and with a variable similar to RELEASE_TESTING, but have it be CONFIDENCE_TESTING or some such to turn on/off that big-image testing. That env-var could be true on your PC or in GH Actions, but not true for smoketesters or the average user; and using MANIFEST.SKIP, you could prevent the large images from being in the distro's tarball, so that it doesn't send huge files to smoketesters or actual users.
      3. If even GH Actions coverage isn't enough to make you confident, you could use GD::Image to generate two large-dimension images -- and since your library even includes the ability to run Image::Square->new($gd) , you don't even need the ability to write to a temporary file like I was originally thinking (plus it gives you coverage for new-from-GD instead of only new-from-file, which is a free bonus from doing it that way). For example, if you made a grid of 144 different-colored 120x120 squares in a 16x9 or 9x16 pattern (to get your vertical and horizontal aspect ratios), you could then pick a few different squares that could still be deterministic: by picking the correct ->square(1080/$n,$pos) , you could pick downscaling-factor $n and offset-factor $pos in such a way that the down-sampled squares have a portion in their middle that should be consistently the right color. It might take some experimentation, but I think you could craft one that would be enough to verify it's working with a large image, hopefully without running into problems with variations in the algorithm.
      (I was originally leaning towards #1 or at most #2, but as I started writing the description #3, I realized that if it were my project, that's the direction I'd go.)

      edit: finished a dangling sentence, and rephrased slightly.
[reply]
[d/l]
[select]
        it is to be expected that libgd and/or the GD::Image wrapper around that library together have sufficient tests on copyResampled to ensure that it works in any situation they claim it will work

        I think that's the key...
        I have been getting too focussed on testing everything but I don't need to test the things that are out of the control of my module!

        Following on from your 3 points I have tried this approach... 

        use strict;
use warnings;

use Test::More;

use GD::Simple;
use Image::Square;

plan tests => 3;

my $horizontal = GD::Simple->new(160, 90, 1);
$horizontal->bgcolor('red');
$horizontal->rectangle(10,10,150,80);

my $left = GD::Simple->new(90, 90, 1);
$left->bgcolor('red');
$left->rectangle(10,10,90,80);

my $center = GD::Simple->new(90, 90, 1);
$center->bgcolor('red');
$center->rectangle(0,10,90,80);

my $right = GD::Simple->new(90, 90, 1);
$right->bgcolor('red');
$right->rectangle(0,10,80,80);

my $square = Image::Square->new($horizontal->gd);

my $im_left = $square->square(0, 0);
cmp_ok($im_left->png, 'eq', $left->png, 'left image');

my $im_center = $square->square(0, 0.5);
cmp_ok($im_center->png(5), 'eq', $center->png(5), 'center image');

my $im_right = $square->square(0, 1);
cmp_ok($im_right->png(5), 'eq', $right->png(5), 'right image');

        [download]
        Drawing a rectangle with a smaller, red rectangle inside it. Then drawing the three parts of this simple shape and comparing them to the output from Image::Square. This works for the left image but not centre or right. Examining the image files, the output from Image::Square is one byte less. Not doubt from copyResampled.

        Given your excellent point about libgd and GD::Image being tested, I don't think it is worth me testing the images other than to ensure they are the expected dimensions.

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom