in reply to Re^2: DBI::SQLite slowness
in thread DBI::SQLite slowness

I imagine BrowserUk is using vec

See Re: vec overflow?, Re: searching a vector array, Re^2: Array vs. Hash for sparsely integer-indexed data (bit vectors)

This takes about 33MB of ram, 

perl -e " my $f='' ; vec($f, 2**28,1)=1; "

[download]

I think that's marking as seen (as duplicate) the id 268435456 or 268_435_456

I think BrowserUk is probably assuming a minimum starting id, 268_435_456 is a lot of IDs :) 256MiB

Replies are listed 'Best First'.
Re^4: DBI::SQLite slowness ( mini vec tutorial )
by Anonymous Monk on Sep 20, 2013 at 13:10 UTC

    Following up on Re: vec overflow? an adjustable example (mini vec tutorial)

    Read more... (2 kB)

    The @ in the output it produces is used for meaning of "@at", its not an actual array :) for example  0@[ 0][ 0] means the number zero is stored in the first(zero-th,0-th) seen_vecs string, and its the first bit of the string (0-th bit) ; neat how that works, id-zero is zero-th bit, is offset-th-ed-bit :)

    Read more... (2 kB)

    This can help with the vec syntax :) Bit::Vector::Minimal - Object-oriented vec wrapper

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom