Some (possibly biased - decide for yourself) opinions on this:
- The Perl test by Brainbench
does as well as one can do with a 40 question, web-based test.
The test uses an adaptive scoring technique, so you will
get harder questions if you answer the first ones correctly,
and easier questions, if you don't. I doubt that you can
build a web-based multiple choice test that can do much better.
- Obviously, a multiple-choice test is not able to test
a complex skillset like programming adequately. Neither is
an interview, nor a resume. Put bluntly, there are no known
methods to test complex skillsets like programming in an
adequate, efficient and economical way. Proven track records, customer references,
assessment centers, extensive structured interviews, examples of former work, and trial periods are probably
the most effective means of evaluating the skill of a programmer.
- A web-based, free service like Brainbench is a Good Thing
for people without a formal education or similar documentable
achievements to provide some indication on their abilities.
- Tests like the one at Brainbench are good screening
tools, IMHO. If somebody is not able to pass this test
(privately, on his own, I mean, not as
part of an interview procedure), this is a strong indication
that this person has not a deep knowledge of Perl, or has
no motivation to show his knowledge in this test. Thus, this
test can save a lot of time in evaluating possible candidates,
at a minimum cost.
I think the main line of critique against web-based certification
services like Brainbench is in essence a critique of
formalized testing procedures (like multiple-choice tests)
as such. This may have some
merit, but in the end anybody arguing against formalized
testing procedures will have to provide some alternatives
when facing the decision on how to evaluate candidates for
a given job. Usually these alternatives will be more subjective
(face-to-face or even telephone interview, recommendations, etc.)
and/or more expensive (assessment center, trial periods, etc.).
Christian Lemburg
Brainbench MVP for Perl
http://www.brainbench.com