If the hit of processing the 20,000 files in one chunk is giving you greif, don't do that:)

Rather than scanning all the files and building an array of the results in one go, and then processing them in what ever way. you have to, why not overlap the finding of the files with the processing?

You could do this by getting one file at a time from readdir and the processing them in a while loop

opendir my $dh, '/path/to/files', or die ...;
while( my file = readdir $dh ) {
    next unless $file =~ m[\.xml$];
    # do something with the file
}
closedir $dh;
[download]

However, you could (maybe) get a more efficient overlap if you read the directory in one thread and process the matched files in another. Depending on the nature of the load on the machine in question, the filesystem and various other inponderables, this should allow the processing thread to process stuff whilst the read thread is blocked in IO states waiting for the OS.

This might be a starting point if the idea interests you. Try varying the -N option to adjust the buffer size so that the processing thread always has a file to precess when it is ready, without the size of the Q becoming unwieldy.

#! perl -slw
use strict;
require 5.008;
use threads qw[yield];
use threads::shared;
use Thread::Queue;
use vars qw[$N];

die "Usage: $0 [-N=nn] dir .*\.xml" unless @ARGV == 2;

$N ||= 100;

my $signal : shared =  0;
my $Q = Thread::Queue->new();

sub readdir_asynch {
    my ($dir, $mask) = @_;
    print $mask;
    opendir my $dh, $dir
        or die "Couldn't open $dir";
    while( not $signal ) {
        yield if $Q->pending > $N;
        my $file = readdir( $dh );
        last unless defined $file;
        $Q->enqueue( $dir . '/' . $file ) if $file =~ m[^$mask$];
    }
    $Q->enqueue( "QUITING!!" );
}

my $thread = threads->create( \&readdir_asynch, $ARGV[0], $ARGV[1] );
yield;

while( ( my $file = $Q->dequeue ) ne 'QUITING!!' ) {
    printf "%s [%d]\n", $file, -S $file;
}

$thread->join;
[download]

Setting $signal to a non-zero value will cause the read thread to terminate before it completes reading the files, and allow a clean exit from the main thread, if that becomes necessary.

Caveat: You might need to change the 'QUITING!!' message if your ever likely to have a file with that name.