Is there a way I can pass an array or a hash across the socket without formatting it to ASCII first then having to reassemble on the other end?

Net::EasyTCP will do it for you transparently. You can essentailly do

$client->send(\%hash);
[download]

and %hash will be available at the server, (and vice-versa)