This ought to be a script: run it against a source tree and it'll find the modules you used, ask you if you have an opinion on it, solicit some text and prompt for a number of stars.

C:\Documents and Settings\diotalevi\My Documents> submit-cpan-ratings 
+Devel-Foobar\

Modules used:
   DBI-1.21
   POE-0.21
*  WWW::Mechanize-0.12 (already rated)

Comment on DBI-1.21 (enter two blank lines to finish your comment):


Comment on POE-0.21 (enter two blank lines to finish your comment):
I love it. I get the benefits of IPC and/or threads all without having
+ to use threads or forking!


Use this comment? (Y/n):

Rate POE-0.21
  Documentation (0 - 5)? 4
  Interface (0 - 5)? 3
  Ease Of Use (0 - 5)? 4
  Overall (0 - 5)? 5

Thank you! Now uploading your ratings to http://ratings.cpan.org.
[download]

Added: The easy bits are: ExtUtils::MakeMaker does something File::Find-like to fine .pm files, do some basic source regexing to find a list of packages, then just load them and say things like $version = POE->VERSION and DBI->VERSION to get the proper version. From there on out, cpan-upload is an example of storing your cpan credentials locally (and of uploading a script to cpan), use WWW::Mechanize (or something lesser) to submit the results, make a note locally on which module versions have been rated.