Re: Re (tilly) 2: millions of records in a Hash

Thanks tilly, for your advice. I had been trying to load a simple %hash variable with the key/value pairs and I have to take care of the duplicates. The key/value pairs exist in a oracle db, its not indexed on the key. wud hitting the db using dbi module be more efficient than trying to load up a %hash? I am using a monster dec alpha box with atleast 3 gb ram.

Comment on Re: Re (tilly) 2: millions of records in a Hash

Replies are listed 'Best First'.
Re: Re: Re (tilly) 2: millions of records in a Hash by mpeppler (Vicar) on Mar 07, 2002 at 01:26 UTC
I haven't read the whole exchange (sorry!), but if your reason for doing this is to weed out duplicates then I suggest you do that directly in SQL. Something like `select distinct key, value into unique_table from duplicate_table` [download] should work, and shouldn't tax your "monster dec alpha box" excessively. If you want to find which rows have duplicate keys you may have to add a COUNT() and a GROUP BY clause... Michael Update* Note that the select statement above will only work if you have "create table" priviledges in the database...	[reply] [d/l]

Replies are listed 'Best First'.

Re: Re: Re (tilly) 2: millions of records in a Hash
by mpeppler (Vicar) on Mar 07, 2002 at 01:26 UTC

Something like

select distinct key, value into unique_table
  from duplicate_table
[download]

If you want to find which rows have duplicate keys you may have to add a COUNT(*) and a GROUP BY clause...

Michael

Update Note that the select statement above will only work if you have "create table" priviledges in the database...

[reply]
[d/l]