You can test whether a file is empty with -z, and -s will return a size (so you can use it to test for "non-empty").

if ( -s $infile ) {
    my $parser = Text::CSV::Simple->new;
    my @data = $parser->read_file($infile);
}
[download]

Or...

if ( -z $infile ) {
    unlink $infile or die "Can't unlink empty file '$infile': $!";
    next FILE;
}
[download]

Update: If the files you have are not literally zero size, you may have to stick your parsing in an eval to catch the error and deal with it at that point.

eval {
    my $parser = Text::CSV::Simple->new;
    my @data = $parser->read_file($infile);
};  # don't forget the semicolon!

if ( $@ ) {
   if ( $@ =~ /the error you expect/ ) {
       unlink $infile;
       next FILE;
   }
   # unexpected error!
   die $@;
}
[download]

Have a look at perlvar for documentation on $@ (aka $EVAL_ERROR, if you use English).