If you're loading CGI.pm for each request, then you must not be using mod_perl, which would give you a performance boost just by avoiding having to load perl repeatedly.

Once you're there, you can also start caching data from one request to the next by using apache's fastcgi module or by turning your cgi into an apache module itself. If you need to keep the list of available files up-to-the-second, you can store a database connection for future requests. If it can lag a bit, then just keep the list of files around - each server process (assuming apache) will, by default, die after handling 50 requests, at which point your cached data will disappear and be reloaded by the new server spawned to replace it.

Going to SQL without mod_perl/fastcgi, though, would just make things worse, due to the overhead of opening a new database connection for each request.