I'd base the "default" setting on the business decision whether you want to maximize the ads shown or minimize your risk. If you want to maximize the ads shown, show them when an image is in (undecided,clean), otherwise, only show ads when an image is clean.

Based on the same risk/gain assessment, I'd modify the decision for the hierarchy of flags. For the "pure clean" approach, any contesting flag marks the image as "unclean".

I wouldn't let the users contest the porn-opinion of the contest owners and authors. This could be abused by authors tagging all their images as "porn" to subvert ads being shown together with their images, but this could also be abused by users by tagging everything as "porn" to move images into the "unclean" state. I think the risk of authors not wanting ads weighs less than the risk of (anonymous?) users tagging everything as porn.

This may be of some interest Porn Image filter.

What works will depend significantly on your user base. I've been in forums where voting just resulted in chaos but moderators worked well. Here voting works well and moderators would probably be an issue. So look at your user base and from that decide what will work best. I would definitly weight moderator and author votes much higher than user votes, maybe 10 unapposed votes from users would mark it unclean, where a 10/4 vote would make it undecided? There is probably some threshold in there that you will only find through trial and error, and it will all depend on your user base.

I don't know if there is an existing solution, but you might even see about generalizing it more and making it a tag system, where clean/unclean are just two tags, and others could be like/dislike, favorite, colors/subjects/medium/etc. Could make for a very interesting system if everyone can tag photos and different user roles have different weights in tagging.

Thank you all for your helpful advice.

@javafan: I was thinking the same but know no language-agnostic forum with such a concentration of people who are experts in all programming-related matters, who will assess a new problem quickly and precisely and who are very helpful. Besides, the site is built in Catalyst and I asked about existing solutions, which I would prefer in Perl.

@eric256: Since my site is just starting up and we have about ten regular users at this moment, I'll use the user flags as reports for admin-review. I'll be gathering the flagging actions and when I have a usable data sample, I'll try to machine-learn something from it.

If it turns out possible and worthwhile, I'll try to make a general tool out of this when it's ready.

Thanks to all for the insight once again... and happy new year! :-)

#!/usr/bin/perl

use strict;
use warnings;
use Data::Dump::Streamer;
use Voting::Condorcet::RankedPairs;

my $rp = Voting::Condorcet::RankedPairs->new(
         ordered_input => 1  );
$rp->add('clean', 'porn', 0.7);
$rp->add('null', 'undecided', 0.6);

my @winners = $rp->strict_winners;
my @results = $rp->strict_rankings;

my $graph = $rp->graph;
print $graph, "\n";
print Dump $graph, "\n";
[download]

#!/usr/bin/perl

use strict;
use warnings;
use GraphViz::Data::Grapher;
use Voting::Condorcet::RankedPairs;

my $rp = Voting::Condorcet::RankedPairs->new(
         ordered_input => 1 );
$rp->add('clean', 'porn', 0.9);
$rp->add('null', 'undecided', 0.1);

my (@winners) = $rp->strict_winners;
my (@results) = $rp->strict_rankings;

my $graph = $rp->graph;
$rp->compile;
print $graph, "\n";

my $grapher = GraphViz::Data::Grapher->new(@winners);
print $grapher->as_svg, "\n";
[download]