I see a couple of issues with this code:

• You're using a foreach where a while would be better. The former will read the entire file into memory and then iterate over the lines, while a while will read the file line-by-line.
• On each iteration of the loop, you replace the entire contents of @windsordigits. If the last line read does not match the regex, @windsordigits will end up empty, which would explain your error. As AnomalousMonk already pointed out, you probably want to use push instead.
• You're attempting to read what looks like a CSV file by hand. You would be much better off using Text::CSV (plus Text::CSV_XS for speed), as that will also handle the case of a comma inside quotes, which you've got there according to your sample data.

use warnings;
use strict;
use Data::Dump; # Debug
use Text::CSV;
use List::Util qw/sum/;

my $filename = 'part4.csv';
my $csv = Text::CSV->new({binary=>1, auto_diag=>2});
open my $fh, '<', $filename or die "$filename: $!";
my @windsordigits;
while ( my $row = $csv->getline($fh) ) {
    dd $row; # Debug
    if ( $row->[1] =~ /WINDSOR\s+RIVERSIDE/ ) {
        push @windsordigits, $row->[3];
    }
}
$csv->eof or $csv->error_diag;
close $fh;
dd @windsordigits; # Debug
die "No digits" unless @windsordigits;
my $average = sum(@windsordigits) / @windsordigits;
dd $average;

__END__

["CA006139520", "WINDSOR RIVERSIDE, ON CA", "2018-01-02", 10]
["CA006139520", "FOO", "2018-01-02", 99]
["CA006139520", "WINDSOR RIVERSIDE ON CA", "2018-01-02", 24]
["CA006139520", "RIVERSIDE WINDSOR", "2018-01-02", 99]
["CA006139520", "WINDSOR RIVERSIDE, ON CA", "2018-01-02", 59]
(10, 24, 59)
31
[download]