I don't know exactly why you are hitting that limit, but I will say, that when I have had trouble with GD, it was because of a mismatch between the GD c-libs version and the perl GD module. If in doubt, get the latest versions of both.

Maybe you could run the script through strace and see where it fails?