I find it really interesting that poeple dropped out when asked to do a bit for themselves.. how hard can simple search be..?
It's not totally an issue of difficulty (warning: Adrian is about to wander off-topic and ramble about usability. Anybody interested in perlmonks should leave now :-)
Some of the issues we found were:
- Instead of one action (posting a message) you now have two (searching and posting a message). Each one has some finite possibility of a user fouling it up. More actions means that there are more chances for confusion and failure.
- Since posting questions was more complex people skipped it completely and just used the search engine, or fell back to calling people they thought might be able to answer the question, email, etc. They got an answer (eventually) but, since it occured off-site the information never got onto the intranet.
- It's hard to force the search and not annoy the user. We looked at two alternatives:
- The user composes the whole message and then prompted to search. Some users feels they have invested the time to compose a message - so don't bother checking the search results.
- Get them to enter the subject line first, then force the search. Users found entering subject without associated message confusing. People rapidly started writing lists of keywords instead of good subject lines to try and get better search results.
Think of it this way. The users wants to get an answer to their question with the minimum effort expended. Posting a question will get you an answer. Searching may get you an answer. Writing a question takes a period of time that people can estimate well. Finding information using a search engine takes you an indeterminite amount of time - you decide when you want to give up playing with keyword variations.
- Users didn't return to ask a question after the search task. They get into looking at their search results and "forget" that they were originally trying to post a question. Even if they fail to find a useful search result they never go back to ask their question!
I could go on, but I'm sure everbody is bored by now!
In my experience people tend to over-estimate the utility of the information built up using sites like perlmonks, and under-estimate the utility of easing communication. Weird as it may seem, people actually like to help :-)