LDS/GD-2.56.tar.gz
Has already been unwrapped into directory /home/uce/.cpan/build/GD-2

I think that one way to avoid this is to run 'cpan install GD' at the command prompt - instead of opening a cpan shell and running 'install GD' as you are currently doing.

The advantage of doing 'cpan install GD' is that whenever you run that command it will unpack GD-2.56.tar.gz to a location whose suffix is randomly selected (and therefore unlikely to be repeated) - rather than re-using the *same* (original) suffix.

Or, you could probably just exit the existing cpan shell and open a new one before re-running 'install GD'.

Update: I missed this initially:

LDS/GD-2.56.tar.gz
'/usr/local/bin/perl Build.PL --installdirs site' returned status 512, not re-running

Somewhere between 2.50 and 2.56 the author added a Build.PL. It would be interesting to know what he thought he was achieving by doing that.
However, the Makefile.PL is still present and it seems odd to me that the cpan shell chooses to use the (recently-thrown-out-of-Core) Module::Build over the tried and proven (and Core) ExtUtils::MakeMaker.

I would:
1) Unpack GD-2.56.tar.gz to some location;
2) cd to that location and run sequentially 'perl Makefile.PL', 'make test' and 'make install'

This avoids all problems with cpan whingeing about something being up to date or already unpacked - and if one needs to start over afresh one just runs 'make realclean' bfore re-attempting the 'perl Makefile.PL' command.
It also uses ExtUtils::MakeMaker instead of Module::Build.

As others have noted, if the OP can get GD-2.56 built and installed using his perl-5.24 (/usr/local/perl) then his problems with GD should be solved.

Cheers,
Rob