Re: Re: Removing duplicates in large files

How about using a wondows OS? Right now I am using an extremely simple code utilising a foreach loop to scan through the e-mails and find dups this is not efficient. Because the server times out before it can even get halfway through all the e-mail addys. Even if it didn't it would still take 100 years to load the page doing it this way. Is there a window equive to what you are proposing? TIURIC

Comment on Re: Re: Removing duplicates in large files

Replies are listed 'Best First'.
Re: Removing duplicates in large files by b10m (Vicar) on Jan 30, 2004 at 20:10 UTC
Well, of course you could get the GNU tools from http://unxutils.sourceforge.net/, but I'm sure some monk will push this thread into something more Perl'ish ;-) -- b10m All code is usually tested, but rarely trusted.	[reply]
Re: Re: Re: Removing duplicates in large files by iburrell (Chaplain) on Jan 30, 2004 at 20:48 UTC
What server? You didn't say anything about a server in your initial description. What is the CGI process doing? A 120,000 line file is not that big. Perl should be able to tear through it in a couple of seconds. The issue is probably somewhere else, either a slow algorithm or network issues. Since you didn't say where the script is running, how it is getting the data, or how it is communicating with the user, we can't help you.	[reply]