in reply to Re^3: Using the unicode61 tokenizer in DBD::SQLite
in thread Using the unicode61 tokenizer in DBD::SQLite

I seem to have missed that bit. Are you saying that the unicode61 tokenizer works out of the box in 1.41_03?
  • Comment on Re^4: Using the unicode61 tokenizer in DBD::SQLite

Replies are listed 'Best First'.
Re^5: Using the unicode61 tokenizer in DBD::SQLite
by Anonymous Monk on Jan 07, 2014 at 10:20 UTC

    I seem to have missed that bit. Are you saying that the unicode61 tokenizer works out of the box in 1.41_03?

    I don't know; If the stuff I copy/pasted means unicode61 is working, then I guess it is.

      I got 1.41_03 installed using the cpan shell with install ISHIGAKI/DBD-SQLite-1.41_03.tar.gz.
      Still no dice:
      use strict; use warnings; use DBI; my $dbh = DBI->connect( "dbi:SQLite:dbname=test.db", "", "", { # RaiseError => 1, # we'll hand +le the errors ourselves sqlite_unicode => 1, } ) or die "Can't connect to database: $DBI::err +str"; my $sqliteversion = $dbh->{sqlite_version}; print "\nSQLite version: $ +sqliteversion\n"; $dbh->do("DROP TABLE IF EXISTS data") or print "drop table failed"; $dbh->do("CREATE VIRTUAL TABLE data USING fts4 (col1 TEXT, col2 TEXT, +tokenize=unicode61)") or print "failed to create fts4 table";

      Output:
      SQLite version: 3.8.0.2 DBD::SQLite::db do failed: unknown tokenizer: unicode61 at e:\uni612.p +l line 22. failed to create fts4 table

        To expand a bit on the steps Anonymous Monk gave you, the steps would be:

        1. Start the CPAN shell:
          cpan
        2. Download and unpack the DBD::SQLite distribution:
          look DBD::SQLite
        3. Recompile DBD::SQLite with the options that Anonymous Monk proposed:
          perl Makefile.PL DEFINE=-DSQLITE_ENABLE_FTS4_UNICODE61
        4. Watch the output for errors.
        5. Run the tests
          make test
        6. Verify that Unicode now works in the way you expect it to.
        7. Install the module if all tests pass and things work for you
          make install

        I got 1.41_03 installed using

        Previously I said I think recompile with perl Makefile.PL DEFINE=-DSQLITE_ENABLE_FTS4_UNICODE61 and give it a shot , so using the way you did doesn't magically give Makefile.PL thos extra arguments, so I'm not surprised nothing changed for you