in reply to Any limit to large hashes?

Thanks for the advice. I went with a simple hash and was able to buzz through a test of 1,000 in 3 seconds. I then kicked it up a notch and tried to do 2.5 million. It looks like it ran linearly up until the memory was full (on a SGI Indigo2 with 192Mb RAM).

I put the time points I collected below if anyone's interested. It looks like I'll need something more advanced to get significantly above 2 million sequences.

Thanks for the help. 
Minutes   thousands of sequences
1.0000    50.000
2.0000    87.000
3.0000    131.00
4.0000    174.00
5.0000    218.00
7.5000    326.00
10.000    435.00
12.000    523.00
15.000    652.00
17.500    762.00
69.000    2097.0
77.000    2097.0
90.000    2097.0

[download]

Replies are listed 'Best First'.
Re (tilly) 2: Any limit to large hashes?
by tilly (Archbishop) on Dec 08, 2000 at 03:09 UTC
    If you have BerkeleyDB installed, slip this in at the beginning (where %seq_hash is whatever your big hash is): 
    use DB_File;
tie(%seq_hash, "DB_File", "tmp.db", O_RDWR|O_CREAT);

    [download]
    This will be significantly slower, but should reduce the memory requirements drastically.
[reply]
[d/l]

In Section Seekers of Perl Wisdom