Dear Monks,

I've written a module for implementing the backend of a Google suggest-like autocompletion system, and I would like to upload it to CPAN. I'm not so sure about the name. I've been working with the "AutoComplete" name, but I know that people usually complain about top-level namespaces, etc.

The full code is here for your enjoyment.

Update: I have thought about something like (WWW|HTML|JavaScript)::Form::AutoComplete::BackEnd, but since the back-end is independent of the front-end, it is not limited to HTML forms, or to the WWW, or to JavaScript clients...

Update2: I have figured a use for $prefix; I'll update the POD before releasing the module.

Update3: The use that I have found for $prefix is the following: if $prefix = [], you get the drop-down list of options but the word in the input box is not completed; if $prefix = [''], the word is completed and the completed part is selected (that's the value used by Google suggest). I haven't investigated other values yet. 

package AutoComplete;

$VERSION = '0.10';

use strict;
use warnings;

=head1 NAME

AutoComplete - Google Suggest-compatible autocompletion backend

=head1 SYNOPSYS

    package MyAC;
    use CGI;
    use base qw(AutoComplete);

    my @NAMES = qw(alice bob charlie); 

    MyAC->run;

    sub expand {

        my ($self, $query) = @_;
        # do something to expand the query
        my $re = qr/^\Q$query\E/i;
        my @names = grep /$re/, @NAMES;
        (lc $query, \@names, [], []);
    }

=head1 DESCRIPTION

This is a base class for implementing an autocomplete server with the 
+same
protocol used by Google Suggest ( http://www.google.com/webhp?complete
+=1&hl=en
).

The front-end JavaScript code is discussed in
http://serversideguy.blogspot.com/2004/12/google-suggest-dissected.htm
+l .

This module is used by creating a subclass, which should override the
C<expand> method, which takes care of searching for the autocompletion
+ results.

=head1 METHODS

=over

=item $class->run(%args)

Run the whole autocompletion process in one fell swoop. Prints everyth
+ing to
standard output, including the HTTP headers. The arguments %args are p
+assed to
the constructor.

=cut

sub run {
    my $class = shift;
    my $self = $class->new(@_);
    print $self->header;
    print($self->no_js), return unless $self->param('js');
    my ($query, $names, $values, $prefix) = $self->expand($self->query
+);
    print $self->output($query, $names, $values, $prefix);
}

=item $class->new(%args)

Create a new AutoComplete object. Currently the only argument is C<cgi
+>, which
should provide a L<CGI> or CGI-compatible object. If none is provided,
+ a
new CGI object is created by default.

=cut

sub new {
    my ($class, %args) = @_;
    my $cgi = $args{cgi} || CGI->new;
    bless { 
        cgi => $cgi, 
    }, shift;
}

=item $obj->query

Returns the string to be expanded.

=cut

sub query {
    shift->param('qu');
}

=item $obj->cgi

Returns the CGI object being used.

=cut

sub cgi { shift->{cgi} }

=item $obj->param($name)

Get a CGI parameter. Just delegates the call to $self->cgi.

=cut

sub param { shift->cgi->param(@_) }

=item $obj->header(%args)

Return the HTTP headers. By default delegates to $self->cgi, but it us
+es the
UTF-8 encoding by default.

=cut

sub header { shift->cgi->header( -charset => 'utf-8', @_ ) }

=item $obj->output($query, $names, $values, $prefix)

Converts the expanded values into JavaScript, as expected by the calli
+ng
frontend script. $query is a string; all the other parameters are arra
+y refs.

=cut

sub output {
    my ($self, $query, $names, $values, $prefix) = @_;
    my $ret = qq{sendRPCDone(frameElement, "$query", };
    $ret .= $self->as_array($names). ", ";
    $ret .= $self->as_array($values). ", ";
    $ret .= $self->as_array($prefix). ");\n";
    $ret;
}

=item $obj->expand($query)

Provide the autocompleted values for the query. Returns a 4-element li
+st:
($query, $names, $values, $prefix). $query is the query as returned to
+ the
frontend script (typically converted to lowercase). $names is an array
+ ref
of results. $values is an array ref of values that are usually shown o
+n the
right-hand side of the drop-down box in the front end; it is used by G
+oogle
for the estimated result count. The purpose of $prefix is not certain 
+at this
time.

=cut

sub expand {
    my ($self, $query) = @_;
    (lc $query, [],[],[]);
}


=item $obj->as_array(\@arr)

Convert an array ref into a JavaScript Array constructor. Returns a st
+ring.

=cut

sub as_array {
    my ($self, $a) = @_;
    'new Array(' . join(", ", map { qq("$_") } @$a) . ')';
}

=item $obj->no_js

Returns that message that is returned by the Google backend when the C
+<js>
CGI parameter is not true.

=cut

sub no_js {
    print <<HTML;
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<script>
function bodyLoad() {
  if (parent == window) return;
  var frameElement = this.frameElement;
  parent.sendRPCDone(frameElement, "", new Array(), new Array(), new A
+rray(""));
}
</script></head><body onload='bodyLoad();'></body></html>
HTML
    exit;
}

=back

=head1 SEE ALSO

http://www.google.com/webhp?complete=1&hl=en

http://serversideguy.blogspot.com/2004/12/google-suggest-dissected.htm
+l

=head1 AUTHOR

Ivan Tubert-Brohman E<lt>itub@cpan.orgE<gt>

=head1 COPYRIGHT

Copyright (c) 2004 Ivan Tubert-Brohman. All rights reserved. This prog
+ram is
free software; you can redistribute it and/or modify it under the same
+ terms as
Perl itself.  

The original JavaScript frontend code is Copyright (c) 2004 Google, In
+c. Use it
at your own risk or write or find a free version.

=cut

1;

[download]

Replies are listed 'Best First'.
Re: RFC - Module for Google-like autocompletion
by The Mad Hatter (Priest) on Dec 21, 2004 at 03:53 UTC
    Hmm, you say it's not Javascript dependent, but this module outputs JS data structures, no?
[reply]
      You are right. Sorry, I think I was too hasty when I said that it does not depend on JavaScript (although if one really wanted to use it without JavaScript one could parse the JavaScript array declaration instead of evaluating it, or override the methods that generate it ;-) ).
[reply]
Re: RFC - Module for Google-like autocompletion
by Cody Pendant (Prior) on Dec 21, 2004 at 03:36 UTC

    Surely the key aspect of the Google Suggest system is that it uses the XMLHttpRequest object, so that ought to determine the name of your module?


    ($_='kkvvttuubbooppuuiiffssqqffssmmiibbddllffss')
    =~y~b-v~a-z~s; print
[reply]
      That is a good point, but on the other hand, from the point of view of the server this is just a plain vanilla CGI using a GET parameter (and certainly not much XML), which makes me doubt whether I should include XMLHttpRequest as part of the name.
[reply]
Re: RFC - Module for Google-like autocompletion
by dragonchild (Archbishop) on Dec 21, 2004 at 13:45 UTC
    Out of curiousity, was there a reason you chose not to use CGI::Application as your base? You already have 80% of the API the same as C::A ...

    Also, it would be really really cool if you (or someone ... maybe me?) were to code this up as a CGI::Application plugin.

    Being right, does not endow the right to be rude; politeness costs nothing.
    Being unknowing, is not the same as being stupid.
    Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
    Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

[reply]
      I have to confess that I still haven't learned CGI::Application, but it seems I converged on a similar solution to a similar problem... ;-)

      I've given a cursory look at CGI::Application and thought it was too complex for my needs. This CGI is really simple: always the same runmode, only three parameters (out of which only one is really important so far), only one line of output... perhaps if I were proficient using CGI::Application I could figure out how to use it quickly for simple scripts like this, without getting intimidated by the multi-runmode multi-stage talk.

[reply]
        I'm thinking about further applications. For example, I've worked on an application that did reports. Each report had between 8 and 14 parameters to choose from. Most of those were simple drop-down boxes. But, some were searches. You could have more than one searchable parameter on the screen at a time. But, you want the javascript to be simple - so you pass another parameter. That parameter is the runmode you want to use to generate the array(s). That runmode calls a set of standard functions to help it out (like as_string, etc.), but it's still different runmodes.

        Alternatively, this could also serve more than one application in your company. C::A is designed not just for one web application, but to standardize how you build web applications in your company. Oh - and it works seamlessly with mod_perl. :-)

        Being right, does not endow the right to be rude; politeness costs nothing.
        Being unknowing, is not the same as being stupid.
        Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
        Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

[reply]
Re: RFC - Module for Google-like autocompletion
by ryantate (Friar) on Dec 21, 2004 at 18:11 UTC

    It's incredible that you've coded this up so quickly, and I was surpised this morning to see that your node was not frontpaged (after skimming it last night). Thank you for your efforts, I can't wait to try this module.

    What I find most interesting is that while XMLHttpRequest is at the core of this capability, your module actually doesn't need to generate or parse any XML at all!

    It would be interesting to see an even more general server-side XMLHttpRequest handler. Not sure how useful that would be -- too simple? But there are definitely other interesting applications for the technology, outside of autocompletion. Save-as-you-type textfields, for one.

[reply]
      What I find most interesting is that while XMLHttpRequest is at the core of this capability, your module actually doesn't need to generate or parse any XML at all!

      Thank Google for that. I just tried to duplicate the interface, which is to take a string from the "qu" CGI parameter and to return some JavaScript code.

      I started with a very simple CGI script, but then when I started writing another second one for another form I decided to put the common parts into a module (basically the JavaScript generation).

[reply]
Re: RFC - Module for Google-like autocompletion
by eric256 (Parson) on Dec 21, 2004 at 19:48 UTC

    I don't suppose we could get a sample HTML file that uses this?


    ___________
    Eric Hodges
[reply]
      I don't have any publically available html file yet, but you can use http://teknikill.net/cpan/ . I started by copying and modifying the form from that page. Basically it does the following:
      1. Includes a form with a name
      2. Loads the ac.js script
      3. Configures the script by giving it a reference to the form and the location of the CGI

      You may need to make a very minor modification to the ac.js file, because the directory that holds the CGI is hard-coded there.

[reply]

        Thanks... Now to get yours to work with ModPerl and it will be perfect. /me considers porting it to CGI::Application.


        ___________
        Eric Hodges
[reply]
Re: RFC - Module for Google-like autocompletion
by dpavlin (Friar) on Dec 21, 2004 at 22:22 UTC
    This topic forced me into thinkging mode: not long ago, I wrote binary search in JavaScript which can be used to produce comboboxes easily (with larger data files and without dynamic requests from server).

    It has perl script which produce data, but now I think that I might write perl module which produce same data for easy inclusion in existing projects. I'm wondering if someone would find such module useful.

[reply]

Back to Meditations