Sometimes, the first file in the directory is the oldest one:

my $file;
opendir(DIR, $dir) or die "Can't read $dir: $!";
$file = readdir(DIR) until $file =~ /^[^.]/; 
print "First file is $file\n";
closedir DIR;
[download]

This should be pretty fast and memory efficient, since it doesn't actually read the whole directory. Of course, it won't work if the files in the directory can be modified after being placed there, as the oldest file will no longer be the first.