I think you do not need the cgi step, you can connect directly to the selenium server.

So on the server machine you start your selenium server: java -jar selenium-server.jar

On the client, you run a perl script like this:

use Test::WWW::Selenium

my $sel = Test::WWW::Selenium->new( 
   host => "your server's ip",
   port => 4444,
   browser => "*firefox",
   browser_url => "http://www.google.com",
);

# your test script goes here
[download]

Maybe you have to tweak the firewall settings on the server to make this work.