in reply to Re^3: How to eliminate redundancy in huge dataset (1,000 - 10,000)
in thread How to eliminate redundancy in huge dataset (1,000 - 10,000)

... or paragraph mode.

$/ = q{};

[download]

Probably safest to localise it's scope to avoid unwanted side effects.

...
{
    local $/ = q{};
    while ( <> )
    {
        ...
    }
}
...

[download]

Cheers,

JohnGG

In Section Seekers of Perl Wisdom