G'day f77coder,

[Please put your data, as well as your code, within <code>...</code> tags. Parts of your data have been rendered as links due to the presence of square brackets. See "Writeup Formatting Tips" for more on this.]

"so i want to clean the old file by removing comments, and duplicates, so far i have"

Here's some comments on what you have "so far":

• You mostly seem to be on the right track with %seen, split and grep.
• I think you may have become bogged down with too many arrays; reading /etc/hosts into an array was probably a mistake (line-by-line would have been better); and your use of '...', instead of '..', suggests rereading perlop: Range Operators would be useful for you.
• I'd strongly recommend lexical variables, in limited scope, for filehandles: using globally-scoped, package variables, with nondescript names such as TEMP, is likely cause problems in anything but the most trivial scripts (see open).
• Note that sort, by default, uses a string comparison. It will order '.' before digits; digits before ':'; and ':' before letters. Numbers starting with 1 (e.g. 1,000,000), will be ordered before numbers starting with 2 (e.g. 2).

Here's how I might have approached this. I've added some additional test data.

#!/usr/bin/env perl -l

use strict;
use warnings;

my (%data, %seen, @order);

while (<DATA>) {
    s/\s*#.*$//;
    next if /^\s*$/;
    my ($key, @rest) = split;
    push @order, $key unless $seen{$key}++;
    push @{$data{$key}}, grep { ! $seen{$_}++ } @rest;
}

print join ' ', $_, @{$data{$_}} for @order;

__DATA__
127.0.0.1 c2.gostats.com #SpySweeper.Spy.Cookie

127.0.0.1 ads.goyk.com

# 1-800-hostingAS3321069.41.160.0 - 69.41.191.255

127.0.0.1 2a02:598:2::1095
127.0.0.2 2a02:598:2::1096
 
127.0.0.2 2a02:598:2::1096
127.0.0.2 2a02:598:2::1096 127.0.0.2
[download]

Output:

127.0.0.1 c2.gostats.com ads.goyk.com 2a02:598:2::1095
127.0.0.2 2a02:598:2::1096
[download]

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.