A more normal way to test for duplicates is to use a hash. You can set the input record seperator to @: to make this easy. There are a couple of ways to go about this depending on if order is important to you, if you need to know when a duplicate was removed and if the entire file can be read into memory at one time.

# here is an example to get you started in untested code.

my ($in, $out, %hash);
open $in, "<", $file_name; # you set the name somewhere else
open $out, ">", $out_file;
local $/ = "@:";           # set the input record seperator
while (<$fh>) {
    if ($hash{$_}++) {     # will be undef first time then >0
        print "Warning duplicate found\n$_";
    } else {
        print $out $_
    }
}
[download]

If the file is too big to hold in the hash then perhaps use the first line of each record as a key and store the ofset in the outfile where it can be found. If you find that key again read the one you wrote back to see if the entire record matches. Of course the value under the key would be an array of ofsets to allow multiple differing records with the same first line.

If you control the writing program perhaps you can better fix that not to write duplicte records.

You would also get more out of the Monastery if you had a quick look here How do I post a question effectively?

Cheers,
R.

Random Walk is quite right... This is just to further demonstrate the case where the file is too big to hold in memory.

If each of your 'blocks' begin with an @:1107532976::1-like pattern, and each of those are expected to be unique, then just focus on the lines that contain that pattern:

use strict;
use warnings;

my %entries = ();
while (<DATA>) {
    chomp;
    if (/^\@:\d+/) { #if line starts with aht-colon-digit(s)
        if (exists $entries{$_}) { #$_ is the current line
            #$. is the current line number
            print "Line $. duplicates line $entries{$_} [$_]\n"; 
        }
        else
        {
            $entries{$_} = $.;
        }
    } 
}
1;
__DATA__
@:1107530184::1
kkkkkkkkkkkkmkmkmk
kkkkkk
confused.gif
@:1107530257:1107530439:1
kmkmkm
<br>kmkmkm
<br>
<br>Fri Feb  4 10:17:37 2005
<br>
mad.gif
@:1107530709::1
ygyg
ygygygyg
lol.gif
@:1107530717::1
ygyg
ygygygyg
lol.gif
@:1107530963::1
cool
help
cool.gif
@:1107532649:1107532689:1
k
<br>kkkkkkkkkkkkkkkkk
<br>
<br>Fri Feb  4 10:57:29 2005
<br>
lol
lol.gif
@:1107530257:1107530439:1
kmkmkm
<br>kmkmkm
<br>
<br>Fri Feb  4 10:17:37 2005
<br>
mad.gif
@:1107532758::1
lll
Lets
mad.gif
@:1107532976::1
lll
Lets
mad.gif
[download]

Hanlon's Razor - "Never attribute to malice that which can be adequately explained by stupidity"

Just out of curiosity - what regular expression have you already tried? What is the code you've got so far to attempt this?

#! /usr/bin/perl

use strict;
use warnings;

my ($key, %hash, @keys);
while (<DATA>) {
        if (/^\@:\d{10,}:(?:\d{10,})?:1$/) {
                chomp;
                $key = $_;
                # needed to process keys in file order
                push @keys, $key;
        } else {
                $hash{$key} .= $_;
        }
}

my ($value, %revhash);
# changed to foreach to process keys in file order
# while(($key, $value) = each %hash) {
foreach $key (@keys) {
        $value = $hash{$key};
        if(exists($revhash{$value})){
                print "$key is a duplicate of $revhash{$value}\n";
        } else {
                $revhash{$value} = $key;
        }
}

__DATA__
@:1107530184::1
kkkkkkkkkkkkmkmkmk
kkkkkk
confused.gif
@:1107530257:1107530439:1
kmkmkm
<br>kmkmkm
<br>
<br>Fri Feb  4 10:17:37 2005
<br>
mad.gif
@:1107530709::1
ygyg
ygygygyg
lol.gif
@:1107530717::1
ygyg
ygygygyg
lol.gif
@:1107530963::1
cool
help
cool.gif
@:1107532649:1107532689:1
k
<br>kkkkkkkkkkkkkkkkk
<br>
<br>Fri Feb  4 10:57:29 2005
<br>
lol
lol.gif
@:1107532758::1
lll
Lets
mad.gif
@:1107532976::1
lll
Lets
mad.gif
[download]

@:1107530717::1 is a duplicate of @:1107530709::1
@:1107532976::1 is a duplicate of @:1107532758::1
[download]

undef $/;
my $file = <DATA>;
$file =~ s/^\@:\d{10,}:(?:\d{10,})?:1\n(.*?)
          (^\@:\d{10,}:(?:\d{10,})?:1\n\1)/$2/msxg;
print $file;
[download]