This sounds like a good place to use PDL (on CPAN, of course). It's fast, handles large amounts of data (and hides the underlying data structures so you don't have to worry about it). And image processing is one of its main applications - check it out. :-)