The closure requirement isn't really all that awful, if you think about it: it translates directly into two other requirements that seem quite reasonable, for most purposes, to impose. The first is lexical scoping (or equivalently protected scope) and the second is the ability to return a function from a function and pass it around in a variable, while still being able to execute it.

I'm generally a little unfairly biased against a language that doesn't support closures, perhaps, but I'm not so sure that a bias against languages that don't support everything needed to construct closures is at all unfair (despite the fineness of that distinction). While I'm keen on closures, that doesn't mean that closures don't provide a simple, clean, succinct, and fair means of demonstrating other positive characteristics of programming languages. In other words, the bias is conscious, but (as far as I can tell) largely irrelevant in this case.

As for the notion of comparing on a series of tasks, that's sorta what's happening that prompted me to come up with this in the first place. There's sort of an ongoing series of task-examples being dreamed up by me and a small number of others who make use of various languages for the enjoyable exercise of comparing implementations across languages. In fact, it was a Pythonista that suggested doing something involving list processing in the first place.

Single tests can demonstrate good points, though. You simply have to ensure that everyone involved in the comparisons is honest in the analysis, including an analysis of the shortcomings of the specific test task in demonstrating much. For instance, a succinctness test on "hello world" programs certainly won't declare a "winner" between Perl and Python, but it does demonstrate to a useful degree the significant and unwieldy scaffolding overhead required by Visual Basic for the simplest of tasks.