Using the documentation tilly dug up I was able to add more debugging. It turns out the handles are defined. They're just not printable. The first three handles I get, as bytes are:

00 00 00 00
00 00 00 01
00 00 00 02
[download]

I was debugging like this, expecting to see something:

my $h = $sftp->do_open($file, SSH2_FXF_READ);
print "handle=$h\n";
[download]

... and not seeing anything. Not a complete waste of time since I learned a bit about the protocol. And maybe this prevents someone else from wasting a few hours like I did.