Just remembered something else that might be useful..

When it comes time to rate projects in terms of qualities X1..Xn, don't ask people to look at a single project and assign numbers for each axis. The numbers you get will vary from person to person, according to context. Most people would say that a 50' frog is big, for example. But those same people would also say that the ocean is big, and that the entire known universe is big. If you ask people rate the 'bigness' of those three items, the numbers you get will vary based on the order in which they're presented.

OTOH, pretty much everyone will agree that the ocean is bigger than a 50' frog, but smaller than the known universe. In other words, people's perception of relative X-ness tends to be more reliable than their absolute rating.

So instead of rating projects in isolation, give people two projects and ask them to compare them in terms of qualities X1..Xn. For each pair, you'll get a set of n more/same/less ratings that place the two projects relative to each other. Then once you collect enough ratings, you can distribute items along each axis by sorting them topologically.

It is possible that the ratings you collect will be inconsistent, especially if you have more than one person ranking projects. One person may say P1 is bigger than P2, another may say P2 is bigger than P3, and a third may say P3 is bigger than P1. Topologically, that's a loop, and will screw you up if you try to convert ratings to absolute positions along the scale.

You solve that problem by collecting more data, and making the relative values fuzzy. If 60% of people say P1 is bigger than P2, you say that 60% of P1 is bigger than P2, but 40% is smaller. If 30% of people say P2 is bigger than P3, P2 outs P3 by a 30/70 margin. If only 10% of people say that P3 is bigger than P1, it means that 10% of P3 is still bigger than some part of P1:

    P1           ----------
    P2     ----------
    P3  ----------
[download]

Yes, that's a contrived example.. I'm doing text-graphs, here. ;-) The same idea applies for less typeable values, though.

If you really want to obsess about it, you can generate a range and distribution function for each item along each axis, which is pretty much the definition of a fuzzy point. Then you can select a region from the space, and rate projects in terms of how much each one falls inside that region.

Hope I haven't stepped over the line into hopeless incomprehensibility.