Re: KinoSearch - is there a way to iterate over all documents in an index?

I've never used KinoSearch, so this is just based on the docs and looking at the code. It should work, but I haven't actually compiled or run it.

The place to start is KinoSearch::Store::InvIndex. That is the abstract class the various stores are build around (I'm going to assume you are using KinoSearch::Store::FSInvIndex). You want the list and slurp_file methods.

As per the docs for KinoSearch::Store::FSInvIndex and KinoSearch::Store::InvIndex:

    my $invindex = KinoSearch::Store::FSInvIndex->new(
        path   => '/path/to/invindex',
        create => 0,
    );
    my @list_of_filenames = $invindex->list;
    foreach my $filename (@list_of_filenames) {
        my $file_data = $invindex->slurp_file($filename);
        # Do stuff with data
    }
    $invindex->close;
[download]

Comment on Re: KinoSearch - is there a way to iterate over all documents in an index? Select or Download Code

Replies are listed 'Best First'.
Re^2: KinoSearch - is there a way to iterate over all documents in an index? by isync (Hermit) on Sep 12, 2007 at 16:28 UTC
A big thanks for the effort. But it did not get me far, as my index won't fit into ram. So, since there's no method to handle big indexes, I am stuck. (is it true only the searcher can read large indexes chunk-wise?)	[reply]
Re^3: KinoSearch - is there a way to iterate over all documents in an index? by snowhare (Friar) on Sep 13, 2007 at 00:49 UTC
Looking at the code, the KinoSearch::Store::FSInvIndex data store is just a simple directory filled with files. You should be able to open the directory and iterate over the files in it yourself. ;)	[reply]