Depends how you want to access it more than anything else. Either a HoHoA, a HoAoA or a HoAoH would fit the bill. This is a HoHoA variant:

use strict;
use warnings;

my %data_sets;
my @dataHeaders = split /,/, ($_ = <DATA>, chomp, $_);
my @setHeaders = split /,/, ($_ = <DATA>, chomp, $_);

splice @setHeaders, 2; # Assume paired columns and the same for all da
+ta sets

while (<DATA>) {
    chomp;
    my @fields = split /,/;

    Outer: for my $dataSet (@dataHeaders) {
        for my $field (@setHeaders) {
            last Outer unless @fields;
            push @{$data_sets{$dataSet}{$field}}, shift @fields;
        }
    }
}

my $lu = $data_sets{'Data Set 3'}{'X units'}[2]; # Data Set 3's x3 val
+ue
my @matches = grep {$data_sets{'Data Set 1'}{'X units'}[$_] eq $lu}
    0 .. $#{$data_sets{'Data Set 1'}{'X units'}};

for my $index (@matches) {
    print "Data Set 1 element X" .
        ($index + 1) . "'s Y = $data_sets{'Data Set 1'}{'Y units'}[$in
+dex]\n";
}


__DATA__
Data Set 1,Data Set 2,Data Set 3
X units,Y units,X units,Y units,X units,Y units
1,y1,x1,y1,2,y1
2,y2,x2,y2,3,y2
3,y3,x3,y3,1,y3
[download]

Prints:

Data Set 1 element X1's Y = y1
[download]

The s/\t/,/g was to avoid unnoticed editor foibles.