We are doing exactly the same type of thing here, using IM to rework some JPEG scans of POW records from WWII, and we found IM to be ... somewhat hackish. The only redeeming feature of IM is that it (somehow) works. However, whenever we get the chance, we use GD - smaller, faster, better documented, easier to install etc etc. I hope saying 'dont use IM' isn't too annoying an answer.
In this particular case we couldnt use GD becuase we ended up getting an annoying 'moire'-like pattern in the images processed by GD, but a nice smooth background in IM. I think it is a result of the paper we are scanning interacting with the particular algorithms that GD and IM use. This paper is old, being produced in 1930's to 40's. Being paper used for a government contract, it is quite low quality, being very coarse in appearance. You can also make out quite clearly the wire grid paper rested on as it came out the 'wet-end' of the pulp mill, before going through the felt rollers. GD doesn't seem to use any kind of image filtering, so I think the wire grid elements react quite strongly with the small image elements that JPEG uses. IM, by contrast, always puts an image through a filter, by default it is a filter called a Lanczos filter. This results in no patterning, but obviously filtering takes time.