It seems to me like you're over-thinking your algorithm. What you want to express in code is "for each line of data, if the value in the second column is contained in my pre-existing dataset (@all), print the line". Perl is very expressive, so it allows you to say that almost directly:

#!/usr/bin/env perl
use strict; use warnings;

@all = ("7723","7725","9908","7765","7874");

while( <DATA> ) {
    my @column = split /\s+/;  #get columns out of data line
    for my $x (@all) {
      next unless $x == $column[1]; # compare second column, move on i
+f no match
      print $_;  # we found a match, print out the original line
      last;      # we're done, so stop comparing this line
    }
}

__DATA__
77876    8543    CA84985    54E
77873    8003    CA84985    54E
77875    7725    CA84985    54E
77872    8511    CA84985    54E
77873    8123    CA84985    54E
77822    9908    CA84985    54E
77819    8503    CA84985    54E
77826    8040    CA84985    54E
77822    7874    CA84985    54E
77884    8543    CA84985    54E
77809    7211    CA84985    54E
[download]

However, a knowledge of design patterns yields a more elegant solution suggested by the Schwartzian Transform.

#!/usr/bin/env perl
use strict; use warnings;

@all = ("7723","7725","9908","7765","7874");

# partial application of the Schwartzian Transform
# each unique value in @all will now be a key in the %all hash
my %all = map { $_ => 1 } @all;

while( <DATA> ) {
    my @column = split /\s+/;     #get columns out of data line
    print $_ if $all{$column[1]};
}

__DATA__
77876    8543    CA84985    54E
77873    8003    CA84985    54E
77875    7725    CA84985    54E
77872    8511    CA84985    54E
77873    8123    CA84985    54E
77822    9908    CA84985    54E
77819    8503    CA84985    54E
77826    8040    CA84985    54E
77822    7874    CA84985    54E
77884    8543    CA84985    54E
77809    7211    CA84985    54E
[download]

The transformation of array to hash allows the check to be very simple: "does our dataset contain this value?".