Re^2: Sorting Votes, Confidence & Deviations

While I agree, with the use of Margin of Error/Confidence values, I think your approach is overly pessimistic. Take for example an entry that gets a single minimum vote (1 or 0) out of 5. This is just as likely to be deceptive and your approach would, incorrectly I believe, drive this value farther below minimum. As this problem is most noticeable only at/near 100% & 0%, I suggest another approach. Generate 2 values, Score + Error and Score - Error. Trim these values to the range in question, and average the Trimmed values for a final score. This would have the effect of moving scores that are pegged at the end of the range inward by Half their *possible* error. Scores near the center of the range will be left largely unaffected.

Comment on Re^2: Sorting Votes, Confidence & Deviations

Replies are listed 'Best First'.
Re^3: Sorting Votes, Confidence & Deviations by Anonymous Monk on Apr 13, 2007 at 19:35 UTC
I agree that the approach is probably too pessimistic. It essentially takes the view that all uncertainty is downside. I would be reluctant to fudge the calculation of the mean. Which aspect of the data has more "information" to it? The mean or the confidence? That should guide you about how to combine the two. To my eye the least invasive treatment would be to compute the means and the standard errors, and then use the standard error to break ties when sorting by the means. Obviously you need the actual values in the population for this (to compute the standard deviation). For the other ones, maybe you can stick in a "default" standard error (don't know what you're scoring, so who knows).	[reply]

Replies are listed 'Best First'.

Re^3: Sorting Votes, Confidence & Deviations
by Anonymous Monk on Apr 13, 2007 at 19:35 UTC

I would be reluctant to fudge the calculation of the mean. Which aspect of the data has more "information" to it? The mean or the confidence? That should guide you about how to combine the two.

To my eye the least invasive treatment would be to compute the means and the standard errors, and then use the standard error to break ties when sorting by the means. Obviously you need the actual values in the population for this (to compute the standard deviation). For the other ones, maybe you can stick in a "default" standard error (don't know what you're scoring, so who knows).

[reply]