G'day f77coder,

[Please put your data, as well as your code, within <code>...</code> tags. Parts of your data have been rendered as links due to the presence of square brackets. See "Writeup Formatting Tips" for more on this.]

"so i want to clean the old file by removing comments, and duplicates, so far i have"

Here's some comments on what you have "so far":

• You mostly seem to be on the right track with %seen, split and grep.
• I think you may have become bogged down with too many arrays; reading /etc/hosts into an array was probably a mistake (line-by-line would have been better); and your use of '...', instead of '..', suggests rereading perlop: Range Operators would be useful for you.
• I'd strongly recommend lexical variables, in limited scope, for filehandles: using globally-scoped, package variables, with nondescript names such as TEMP, is likely cause problems in anything but the most trivial scripts (see open).
• Note that sort, by default, uses a string comparison. It will order '.' before digits; digits before ':'; and ':' before letters. Numbers starting with 1 (e.g. 1,000,000), will be ordered before numbers starting with 2 (e.g. 2).

Here's how I might have approached this. I've added some additional test data.

#!/usr/bin/env perl -l

use strict;
use warnings;

my (%data, %seen, @order);

while (<DATA>) {
    s/\s*#.*$//;
    next if /^\s*$/;
    my ($key, @rest) = split;
    push @order, $key unless $seen{$key}++;
    push @{$data{$key}}, grep { ! $seen{$_}++ } @rest;
}

print join ' ', $_, @{$data{$_}} for @order;

__DATA__
127.0.0.1 c2.gostats.com #SpySweeper.Spy.Cookie

127.0.0.1 ads.goyk.com

# 1-800-hostingAS3321069.41.160.0 - 69.41.191.255

127.0.0.1 2a02:598:2::1095
127.0.0.2 2a02:598:2::1096
 
127.0.0.2 2a02:598:2::1096
127.0.0.2 2a02:598:2::1096 127.0.0.2
[download]

Output:

127.0.0.1 c2.gostats.com ads.goyk.com 2a02:598:2::1095
127.0.0.2 2a02:598:2::1096
[download]