I'm kinda confused as to what you did, so I'm going to try to write a translation of what you say in Perl:

my $file_A = read_file( "file_A" );
my $file_B = read_file( "file_B" );

foreach (keys %$file_A) {
  delete $file_A->{ $_ } unless exists( $file_B->{ $_ } );
}

sub read_file {
  my $file = shift;
  open FILE, $file or die "Can't open '$file' for reading $!";
  my @file = <FILE>;
  close FILE;

  my $hoa = {};
  foreach ( @file ) {
    my (@fields) = split /:/;
    my $key = $fields[0];
    $hoa->{ $key } = [ @fields ];
  }
  return $hoa;
}
[download]

If this is how you parsed the files, then it seems to me you could easily do the following:

sub change_field_to_check {
  my $hoa = shift;
  my $field = shift;
  $field -= 1;

  my $new_arrangement;

  foreach (keys %$hoa) {
    $new_arrangement->{ $hoa->{ $_ }[ $field ] } = $hoa->{ $_ };
  }

  return $new_arrangement;
}

$file_A = change_field_to_check( $file_A, 4 ); # Field 4
$file_B = change_field_to_check( $file_B, 4 ); # Field 4

foreach (keys %$file_A) {
  delete $file_A->{ $_ } unless exists( $file_B->{ $_ } );
}
[download]

This ought to extrapolate to other files and other fields. If your fields are named something else, having a hash that translates the field name to their order will give you the right key.

Hope this helps.

- m.