I can fix the problem by using a temporary variable to hold the file handle, but is there some obvious (or otherwise) syntactic construct I'm missing that allows me to use a file handle stored in a hash reference directly inside the diamond operator?
No. See perlop

If what's within the angle brackets is neither a filehandle nor a simple scalar variable containing a filehandle name, typeglob, or typeglob reference, it is interpreted as a filename pattern to be globbed, and either a list of filenames or the next filename in the list is returned, depending on context. This distinction is determined on syntactic grounds alone. That means <$x> is always a readline() from an indirect handle, but <$hash{key}> is always a glob(). That's because $x is a simple scalar variable, but $hash{key} is not--it's a hash element.