<form action="http://www.google.com/search" method=GET>
<INPUT TYPE=hidden name=site value=swr>
<INPUT TYPE=hidden name=q value="site:stonehenge.com">
<INPUT TYPE=text name=as_q size=31 maxlength=256 value="">
<INPUT TYPE=submit name=btnG VALUE="Search stonehenge.com with Google"
+>
</form>
[download]

-- Randal L. Schwartz, Perl hacker
Be sure to read my standard disclaimer if this is a reply.

I think it only works if your site has webpages have somehow been indexed by Google. Otherwise, nothing returns.

If you don't want to wait for Google to find you, you can always submit your URL to Google. After a few days or so, your site will be indexed and searchable.

jeffa

L-LL-L--L-LL-L--L-LL-L--
-R--R-RR-R--R-RR-R--R-RR
B--B--B--B--B--B--B--B--
H---H---H---H---H---H---
(the triplet paradiddle with high-hat)

Along the lines of what merlyn said a product I've found very useful is called Perlfect. Its GPL and written in Perl, its pretty fast and easy to set up as well. I've used it on lots of small to medium sites.

We use perlfect at work, and it seems pretty good. On the other hand, I didn't set it up, and when I tried setting it up on my own site, I found it to be a pain in the arse. So I stuck with the one that I'd written a couple of years before and which does a sufficiently good job that I am disinclined to try any harder with perlfect.

--
[ e d @ h a l l e y . c c ]

As of Version 3.23.23, MySQL has support for full-text indexing and se
+arching. Full-text indexes in MySQL are an index of type FULLTEXT. FU
+LLTEXT indexes are used with MyISAM tables only and can be created fr
+om CHAR, VARCHAR, or TEXT columns at CREATE TABLE time or added later
+ with ALTER TABLE or CREATE INDEX. For large datasets, it will be muc
+h faster to load your data into a table that has no FULLTEXT index, t
+hen create the index with ALTER TABLE (or CREATE INDEX). Loading data
+ into a table that already has a FULLTEXT index will be slower.

Full-text searching is performed with the MATCH() function. 

mysql> CREATE TABLE articles (
    ->   id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,
    ->   title VARCHAR(200),
    ->   body TEXT,
    ->   FULLTEXT (title,body)
    -> );
Query OK, 0 rows affected (0.00 sec)

mysql> INSERT INTO articles VALUES
    -> (NULL,'MySQL Tutorial', 'DBMS stands for DataBase ...'),
    -> (NULL,'How To Use MySQL Efficiently', 'After you went through a
+ ...'),
    -> (NULL,'Optimising MySQL','In this tutorial we will show ...'),
    -> (NULL,'1001 MySQL Tricks','1. Never run mysqld as root. 2. ...'
+),
    -> (NULL,'MySQL vs. YourSQL', 'In the following database compariso
+n ...'),
    -> (NULL,'MySQL Security', 'When configured properly, MySQL ...');
Query OK, 6 rows affected (0.00 sec)
Records: 6  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM articles
    ->          WHERE MATCH (title,body) AGAINST ('database');
+----+-------------------+------------------------------------------+
| id | title             | body                                     |
+----+-------------------+------------------------------------------+
|  5 | MySQL vs. YourSQL | In the following database comparison ... |
|  1 | MySQL Tutorial    | DBMS stands for DataBase ...             |
+----+-------------------+------------------------------------------+
2 rows in set (0.00 sec)
[download]

your search page can then parse the key words, look up the words individually in the database with a join or a union (depending on type of return you want) as well as searching on partial words.

Enjoy
-John

From their site: "mnoGoSearch (formerly known as UdmSearch) is a full-featured web search engine software for intranet and internet servers. mnoGoSearch for UNIX is a free software covered by the GNU General Public License"

It's C based code, and the compiled CGI frontend works far better than the provided Perl script. Maybe some day someone will write an XS interface...

Hi there I am about to launch a Perl Application which will do mostly what you need. It will be available for demo soon at http://www.minigoogle.co.uk It searches msql databases, entire web directories and contains an engine which allows the admin to specify a url on another website to be spidered and indexed in the database. So you could group websites together under a topic and search these sites by keyword. I am hoping the concept will take off and I can "chain" the search engines together to make a much bigger searchable repository (similar to the way filesharing works but with information). The intention is to create a service which allows a more focused search with more accurate results. BUt it is early days yet. I also have a Lite version of the script which searches only flat file databases. It has been a team effort with several programmers working on it from all over the world. Pretty soon there will also be a version which will work like a Yahoo style directory engine but the plans for this have only just been drawn up. Does anyone have any comments, thoughts or ideas? cheers Dataferret

Does anyone have any comments, thoughts or ideas?

yes, i don't think google will be particularly happy with your choice of application name.

~Particle *accelerates*

Note that my approach is probebly overkill in your situation, it is meant for large data sets (tested on 250MB of text & works surprisingly well).
I found that the best way is to set up an inverted-index of all the terms as well as an index which shows the position of each word within each document.
I then use an algorithm which gives a bonus if the words that are being searched appear close to each other in a ducument -- this proximity-search algorithm is described at http://citeseer.nj.nec.com/cachedpage/550719/1 .
Also to improve the inverted-index words are indexed by their stem (a stemming algorithm can be found here http://www.ldc.usb.ve/~vdaniel/porter.pm ).
Aswell I have implemented an algorithm similar to google's pagerank (a good description of it is at http://citeseer.nj.nec.com/cachedpage/368196/1 ), the popularity of a page is taken into account when returning results.

I use MySQL for all the storage / indexes.

Richard

There are three types of people in this world, those that can count and those that cannot. Anon