I disagree, the current system is just too inflexible and static. To give an idea, lets look at Module::Build. The rating system gives a 3 star. However, the ratings apply to the module over the course of 3+ years which has had many changes.

On the other hand, should we reset the ratings for a particular module everytime they release a new version? I don't that either.

I would rather see a way of inventorying of modules that get used in scripts or modules that using cpan modules, but they, themselves are not on cpan. It would give an idea of which modules are actually being used by developers. It would not be perfect, but seeing what people are actually using would be incredibly useful.

If someone complains about a bug from 3 years ago is that rating properly reflective of the utility of the current module?

What if we let votes for older versions of a module count less than votes for recent versions? E.g. by using the pheromone decay principle from ant colony optimization.
--
No matter how great and destructive your problems may seem now, remember, you've probably only seen the tip of them. [1]

...pheromone decay principle...

But that's the problem! There's no pheromone decay module on CPAN!

Of course... there will be soon if there's a demand for it :)

/me begins work on Acme::PheromoneDecay::Versioned

Then the terrorists win:P