I just did something very similar, to wrap FTP connections in a seekable filehandle. See Extracting individual files from a Zip archive over FTP... and Net::FTP::RetrHandle. Also see Tie::FTP, which takes a different approach more similar to the one you describe.

Let me know if you're able to abstract some of this functionality out, and I'll rework my module to use yours. :)