Well, just for anyone else's sake. What I ended up doing was just dropping the use of Image Magick, as it is terrible. After looking at some of the documentation, it looks like Image Magick does all kinds of unneccisary conversions which cause it to use more memory than it should.

I then tried using just the Imager module. That would've worked great, but it didn't seem to support multi-page tiff's.

So, on to my solution. I used tifftools (available on sourceforge), specifically tiffcp to recompress the file. I wrote a simple perl script to run the command line arguments (and watch for errors). The solution that worked fastest, and best for me was to compress the tiff with tiffcp -t in tile mode. Then, turn around and recompress that tile mode tiff in strip mode. That fixed the "squished" tiff problem and took up little memory and time.