So, how do you feel about the rating system?
By
Alari_Azure,
Posted on: 2009-05-10 18:28 in Mission Architect
[ QUOTE ]
[ QUOTE ]
... Third, it's not griefing to give someone a low rating. There are a lot of reasons why someone might have given your friend's arc a low rating. ...
[/ QUOTE ]
It is griefing to give someone a low rating without even entering the mission. If casting a vote required some kind of time investment, even as small as five minutes, the number of potential griefings would be drastically reduced.
Right now someone could hack the client, the way they seem to have done for sending in-game email spam, and automatically grief every arc in the system.
By requiring that you spend some amount of time in the mission or complete at least the first mission before you can cast a vote, the devs would eliminate most of the opportunities for griefing.
[/ QUOTE ]
With the changes recently made to custom critters, for some this would be impossible.
Again NO.
If an arc creator hasn't tested his arc to make sure that the first mission doesn't require a tank, scrapper, or brute to be even completable, that's not something that deserves to be played before it's rated down.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
[ QUOTE ]
[ QUOTE ]
Except that leaves "excellent but unadvertised" arcs in the same bin as "terrible crap that 600 people have already played and abandoned".
Under that system, I don't want to be the author of the "excellent but unadvertised" arc, and I don't want to be player #601 of the "terrible crap" arc. We need to do something, but I don't think this suggestion is it.
[/ QUOTE ]
All you would need is one person to play through your arc and rate it to get it above the "abandoned crap" arcs.
I understand the reasoning behind allowing players to rate arcs they didn't finish, but I think the downside outweighs the up. Mission creators don't just want ratings, they want plays. Players abandoning the their arcs is already punishment to them, they don't need the extra negative reinforcement of one/zero-starring.
(And given that there's a "trick" to zero-starring, that option doesn't seem to be intentional.)
[/ QUOTE ]
The negative ratings allow folks to know what nonsense arcs to avoid.
If the system could show number of abandons then I would get behind having to finish an arc to be able to rate it. Until then, not just NO, but [censored] NO!
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
[ QUOTE ]
They need to remove the ability to "downrate" arcs.
Let people either select to "thumbs up" the arc or not rate it at all. There are too many arcs that get low ratings not because of anything wrong with the arc, but because of personal taste ("I don't like carnies, 1 star!" "I don't like office maps, 0 stars!")
Good ones would still rise, but arcs wouldn't be as subject to suffering from nonsensical ratings as they are now.
[/ QUOTE ]
Only if they display the no ratings as well.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
[ QUOTE ]
I think that any proposed change to the rating system comes with this understanding: No matter what is done, people will find a way to grief, exploit, and try to do things which are dishonest to the spirit of any quality indicator system. Some people are just jerks; and the badges, and tickets, only act as an enticement for this behavior. I wouldn't blame the devs for somebody's mother failing to raise them properly.
What I read as the major concern of this discussion is: 'How do I ensure that my arc is only being played by people who will actually give it a fair shot?" The best way I can think of is to hide the arcs from those who won't. A preference system wouldn't be out-of-line (and would be an excellent tool for finding what you want). Give players a checkbox to say what kind of mission they are looking for (i.e. no custom mobs, no AVs, or anything with a 'racy' theme), while giving creators the ability to designate their arcs with those same terms (or use the same information that is given in the basic arc description). This isn't a cure-all, but probably more of a stalling tactic to allow more 'honest' people to play before a 'griefer' happens upon it.
A major benefit I see with this is that it would allow people to avoid content that they do not wish to play (the lesbian hellion issue comes to mind with this), while also focusing on those that they want. This would also give the devs a tool, as they can now ask a person the question, "If you hate arcs with AVs in it so much, why do you keep on playing them?"
This is just an idea. But, I feel that the popularity of the MA lies with people being able to easily find the kinds of arcs they want.
[/ QUOTE ]
Best idea in the thread.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
Ok - so clearly there are a large number of folks who think that it is MORE important to be able to down rate a bad mission than it is to prevent a good misison from being down rated inappropriately. I am not sure you are right, but I think we are down to fairly basic opposing points so I don't see how any more discussion will help.
I do think that divorcing the rating system from tickets and badge awards is probably the correct way to go, combined with a better seach engine that has reasonable tags so you can actually find missions you want to play. I suspect the dev's need to do something like that first, then look to see how it affects the rating system before they make changes to it.
Globals: @Midnight Mystique/@Magik13
[ QUOTE ]
Ok - so clearly there are a large number of folks who think that it is MORE important to be able to down rate a bad mission than it is to prevent a good misison from being down rated inappropriately. I am not sure you are right, but I think we are down to fairly basic opposing points so I don't see how any more discussion will help.
I do think that divorcing the rating system from tickets and badge awards is probably the correct way to go, combined with a better seach engine that has reasonable tags so you can actually find missions you want to play. I suspect the dev's need to do something like that first, then look to see how it affects the rating system before they make changes to it.
[/ QUOTE ]
Yes on the last paragraph: badges, tickets and more importantly the freaking free slot should be totally REMOVED from being linked to ratings.
EDIT: I would have no issue with being forced to play an arc, if it showed the number of quits on an arcs as well.
That would be a great flag to folks to not even bother with such a crappy arc.
I'm sorry but I shouldn't have to play through something painful (my time is MY time, not the freaking missh creators--sorry), and I SHOULD be able to warn others away from it.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
[ QUOTE ]
[ QUOTE ]
Ok - so clearly there are a large number of folks who think that it is MORE important to be able to down rate a bad mission than it is to prevent a good misison from being down rated inappropriately. I am not sure you are right, but I think we are down to fairly basic opposing points so I don't see how any more discussion will help.
I do think that divorcing the rating system from tickets and badge awards is probably the correct way to go, combined with a better seach engine that has reasonable tags so you can actually find missions you want to play. I suspect the dev's need to do something like that first, then look to see how it affects the rating system before they make changes to it.
[/ QUOTE ]
Yes on the last paragraph: badges, tickets and more importantly the freaking free slot should be totally REMOVED from being linked to ratings.
EDIT: I would have no issue with being forced to play an arc, if it showed the number of quits on an arcs as well.
That would be a great flag to folks to not even bother with such a crappy arc.
I'm sorry but I shouldn't have to play through something painful (my time is MY time, not the freaking missh creators--sorry), and I SHOULD be able to warn others away from it.
[/ QUOTE ]
You know - I was thinking about this some more and I wonder if what might be best is some kind of multiple rating system.
Have one set of ratings that have nothing to do with tickets, badges, etc and are can be set whether or not you can play through the entire arc. There could even be 2-3 (or more) different categories - playability, story content, etc. These would be individually indexed and would allow you to be more specific about WHY you down rated a mission without having to get personal about it, which is a problem I have now. I hate giving a mission a low rating without providing feedback but if the feedback is simply 'mission is trash' I am not going to send that. However, being able to just flag the 'playability' rating at 1 star would be nice. You shouldn't even be required to fill in all the ratings - just the ones you think are appropriate for the arc.
Then badges/tickets/whatever can be tied to actually completing an arc, possibly even without having to rate it at the end. Obviously some attention would have to be payed to ways to keep someone from gaming the system by creating instantly finished single mission arcs - but you can do that already and as long as my ability to find arcs I want to play is not linked to a reward system I don't really care how the developers implement the reward system. (Selfish of me - I don't actually design arcs - I just want to play them :-).
Globals: @Midnight Mystique/@Magik13
[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
Ok - so clearly there are a large number of folks who think that it is MORE important to be able to down rate a bad mission than it is to prevent a good misison from being down rated inappropriately. I am not sure you are right, but I think we are down to fairly basic opposing points so I don't see how any more discussion will help.
I do think that divorcing the rating system from tickets and badge awards is probably the correct way to go, combined with a better seach engine that has reasonable tags so you can actually find missions you want to play. I suspect the dev's need to do something like that first, then look to see how it affects the rating system before they make changes to it.
[/ QUOTE ]
Yes on the last paragraph: badges, tickets and more importantly the freaking free slot should be totally REMOVED from being linked to ratings.
EDIT: I would have no issue with being forced to play an arc, if it showed the number of quits on an arcs as well.
That would be a great flag to folks to not even bother with such a crappy arc.
I'm sorry but I shouldn't have to play through something painful (my time is MY time, not the freaking missh creators--sorry), and I SHOULD be able to warn others away from it.
[/ QUOTE ]
You know - I was thinking about this some more and I wonder if what might be best is some kind of multiple rating system.
Have one set of ratings that have nothing to do with tickets, badges, etc and are can be set whether or not you can play through the entire arc. There could even be 2-3 (or more) different categories - playability, story content, etc. These would be individually indexed and would allow you to be more specific about WHY you down rated a mission without having to get personal about it, which is a problem I have now. I hate giving a mission a low rating without providing feedback but if the feedback is simply 'mission is trash' I am not going to send that. However, being able to just flag the 'playability' rating at 1 star would be nice. You shouldn't even be required to fill in all the ratings - just the ones you think are appropriate for the arc.
Then badges/tickets/whatever can be tied to actually completing an arc, possibly even without having to rate it at the end. Obviously some attention would have to be payed to ways to keep someone from gaming the system by creating instantly finished single mission arcs - but you can do that already and as long as my ability to find arcs I want to play is not linked to a reward system I don't really care how the developers implement the reward system. (Selfish of me - I don't actually design arcs - I just want to play them :-).
[/ QUOTE ]
I agree with anything that allows folks to avoid the nonsense that some "creators" put up as missions.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
[ QUOTE ]
Not if the number of plays is also displayed. One of those would be played 0 times and rated 0 times. The other would be played 600 times but never rated. That would be a huge red flag to person #601.
Which, sadly, gives a new way to grief: start the mission, then quit. Over and over. It will look like the mission is unplayable, or not worth finishing.
[/ QUOTE ]
The solution to that is trivial. Multiple ratings from the same account only count once, and only the most recent, and there's no good reason multiple abandons (or completions) from the same account shouldn't obey all the same rules. One account, one vote, regardless of how many forms that vote may be expressed in.
[ QUOTE ]
I question the existence of one star "griefing." I've heard this argument before - that there are cadres of malcontents who patrol the internet deliberating downranking everything out of sadism - and it sounds ridiculous. And some of these countermeasures you're suggesting are insane... It's like you want to start a witch hunt over this stuff.
[/ QUOTE ]
Whether you believe it or not, it does happen. I've typed out a lot of proof before, but those who don't think it's happening just seem to want to think that those of us that have seen it with our own two eyes (including people saying "HAY I'M DOING THIS RITE NOW!!!") are believers in some giant conspiracy theory.
We're not saying aliens caused 9/11, we're showing you that an arc that hits page one that has never gotten anything but 4 or 5 star votes will suddenly get a dozen or so 1 or 0 star votes, fall to page eleventy billion, and then proceed to get 4's and 5's again. The only time it gets 1's and 0's is when it's on the front page. People are saying "HAY, WE VOTE !'a AND 0'S FOR THINGS ON THE FRONT PAGE." I don't know what else to tell you.
I have to agree with most of the folks that have posted above me, the rating system is definitely flawed. Having a simple 5 star system does not cover all of the possible reasons for high or low ratings. There needs to be a better way of submitting ratings. I liked the idea mentioned about having different categories to select a rating for, that would separate the reasoning behind them.
Something like this with a 1-5 or 1-10 star for each, or a choice for certain categories:
Was it a fun story arc? (y/n)
Rate the overall difficulty of the arc. (1-10?)
Rate the difficulty of the Boss(es). (1-10?)
Did you complete it? (y/n) If no, why not? (choices to rate for difficulty/errors/boredom, etc?)
Was the arc played by a solo char? Team? (y/n)
Type/level of char rating it? Rank? (by player choices or generated by the system to keep it honest)
Rate the story: Was it interesting? Were there typographical/grammatical errors? Was the contact info, mission sendoff, Nav text, etc., helpful/understandable? (y/n or 1-10)
Rate custom characters look/difficulty/appropriateness to the story. (1-10?)
All of the above could be used as criteria to be averaged for ratings that could be searchable in the MA list, and offer people looking for certain types of arcs to see what positive points or problems may exist in the arc. I realize it's a lot of info, but it would give people more ways to rate it than just a pure thumbs up or down, also more detailed and harder to 'fudge'. There could also be a general star rating that could be given to the arc as a whole. Maybe a 10 star that would have more wiggle room.
I would also like to see feedback made anonymous by the player's choice. Sometimes, even though we intend to provide feedback to be a means for suggesting improvements, they may not always be seen that way by the author. This would eliminate the chances of an author being vindictive and retaliating for what they perceive as an 'unfair' rating.
I know I'm a relative n00b on the boards, but this is just my $.02. FWIW 
No AV/EBs Deal with The Devil's Pawn-207266 Slash DeMento and the Stolen Weapons-100045 Meet the Demon Spawn-151099 Feedback
I really like the idea of community input, but the way it's turned out, it's not really helping us find great arcs. Two things contribute to this: The search function doesn't let us sort by # of Ratings, and 0-star griefer votes require 9 5-star votes to keep an arc at the 4.5 star breakpoint.
This means that people try to find good arcs only by searching for 5 stars (or if they're familiar with the flaws in the system, 4 stars). The problem with that is the grief-votes keep 5-stars off the top, and I think other creators can support me in saying that once that 5-star visual rating drops, so does the rate of "random players".
I've had a hard time getting Real Life to let me spend more time reviewing and working with the support community here, so I also haven't been able to spread much word about the Tutorial Arc I created to help out other players. Luckily, a few helpful players got some more word out for me, and one weekend the Tutorial got back up to a 5-star visual. In one day, over 150 players rated the arc... but enough "grief" votes came in to knock it back down to 4. Since then, I've had maybe a dozen a week, many of them people I've been able to play/rate theirs.
It's not that I must collect all stars, here, that got me thinking about this. It's that I've got a 20% rate of people giving me feedback, and nearly all of it is grateful for the help. This makes me want to help more people find the arc so they can play it. But the people who need the help the most won't be on the forums... they'll be looking in the game. How can I try to help get my rating up so people can find it using the current system, when it merely being at a 5-star point just invites the griefers to knock it down?
Personally, I'm mourning @Muu's loss (again) of Hall of Fame status... I'm rambling. But I wish I had a better solution than what we have.
#28470 - MA & YOU! Quick Tutorial
Poster 1481: Cause of How Some Silly Stealed My Wings
Cultist fun in 3586 Project: Perilous - Into the Chthonian Pit
Formerly of Perils of Paula!
I give truly abysmal arcs one star. Some get zero stars, if they really went into wall-banger territory.
Most 'bad' missions get two stars. Three is reserved for 'ok' missions that could possibly be improved. Four for the good ones. Five for the truly awesome ones.
All of my stars come enclosed with a comment. Read the comment. You'll know why your arc got X stars. Simple.
As for the griefing / twinking potential in the system... I'm not really paying attention to the number of stars everyone else has, anymore, cos it's obvious the system is broken vis a vis honest ratings. Instead I select arcs by reviews, or by requests to play them. I might (!) stumble across something by browsing, but then only if I'm bored.
"City of Heroes. April 27, 2004 - August 31, 2012. Obliterated not with a weapon of mass destruction, not by an all-powerful supervillain... but by a cold-hearted and cowardly corporate suck-up."
I find 4 star combined with SFMA to be a pretty reliable search for quality story missions.
It's imperfect but passable.
0 star ratings should not have been allowed.
I rate newspaper quality 3 stars.
Well-written story based arcs get 4 stars.
Incredible arcs that should be in game contacts get 5 stars.
Questions about the game, either side? /t @Neuronia or @Neuronium, with your queries!
168760: A Death in the Gish. 3 missions, 1-14. Easy to solo.
Infinity Villains
Champion, Pinnacle, Virtue Heroes
The problem with giving well written, even exceptional arcs 4 stars, is that even though this is by all accounts an excellent rating to give a story arc, if the writer is hoping for the arc to eventually become player's choice, that 4-star rating is actually a punishment. Each 4-star rating requires 2 5-star ratings to counteract it, or it will never see Player's Choice status.
This is probably a big reason why there's no stable player's choice arcs - even people who think they are being kind and generous by 4-starring an arc, are ultimately punishing that arc writer's chance of the arc staying there.
[ QUOTE ]
I find 4 star combined with SFMA to be a pretty reliable search for quality story missions.
[/ QUOTE ]
That is so funny, I search for 4 star medium maps now and have way better luck than I ever did when searching for 5 stars. Rating's broken. Plan and simple.
You know, I thought during beta that you'd get HoF with 1000 4 or 5 star votes.
This was probably a mis-hearing on my part, but it still sounds like a good idea.
A 4.5 average means you need 1 5 vote and 1 4, or 3 5s and 1 3, or 5 5s and 1 2, or 7 5s and 1 1, or 9 5s and 1 0.
Somebody deliberately down-voting is 9 times as powerful as somebody deliberately up-voting.
Scaling it back to 7 times as powerful won't fix anything.
But you know, nothing is going to ensure that a good arc gets plays without even being advertised. There are 6 digits' worth of arc numbers.
Even if the hypothetical "everybody who zero-stars anything not made by their SG mates" gets booted off the system tomorrow and their ratings rescinded, they've been doing that to everybody and the front pages will be just as crowded again. If you don't put your arc in the public eye it's almost certainly not going to get there on its own, no matter what happens.
Trying to come up with a system where that happens is just doomed to fail.
Should tag systems go official, and searches reflect them, how are you going to enforce tags? People searching for story-focused arcs don't want to find LOL THIS IS MY FREAK TANK FARM LOL, after all, and you know that one's going to get every tag under the sun.
Up with the overworld! Up with exploration! | Want a review of your arc?
My arcs: Dream Paper (ID: 1874) | Bricked Electronics (ID: 2180) | The Bravuran Jobs (ID: 5073) | Backwards Day (ID: 329000) | Operation Fair Trade (ID: 391172)
[ QUOTE ]
The problem with giving well written, even exceptional arcs 4 stars, is that even though this is by all accounts an excellent rating to give a story arc, if the writer is hoping for the arc to eventually become player's choice, that 4-star rating is actually a punishment. Each 4-star rating requires 2 5-star ratings to counteract it, or it will never see Player's Choice status.
This is probably a big reason why there's no stable player's choice arcs - even people who think they are being kind and generous by 4-starring an arc, are ultimately punishing that arc writer's chance of the arc staying there.
[/ QUOTE ]
Another possibility is that the folks who rate it 4 stars feel its good, but NOT good enough to be player's choice or hall of fame, and definitely not Dev's Choice.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
[ QUOTE ]
You know, I thought during beta that you'd get HoF with 1000 4 or 5 star votes.
[/ QUOTE ]
You have ears that turn bad ideas into good ones, my carrot-seeking friend. 
That blue thing running around saying "Cookies are sometimes food" is Praetorian Cookie Monster!
Shoot on sight, please.
[ QUOTE ]
[ QUOTE ]
By requiring that you spend some amount of time in the mission or complete at least the first mission before you can cast a vote, the devs would eliminate most of the opportunities for griefing.
[/ QUOTE ]
With the changes recently made to custom critters, for some this would be impossible.
Again NO.
If an arc creator hasn't tested his arc to make sure that the first mission doesn't require a tank, scrapper, or brute to be even completable, that's not something that deserves to be played before it's rated down.
[/ QUOTE ]
Perhaps I was insufficiently clear. I said, "By requiring that you spend some amount of time in the mission or complete the mission" I mean that as an either-or.
If you gave it your best shot and spent five minutes in the arc and found that you couldn't get anywhere because only scrappers and tankers have a chance, then your vote would be registered. I also said that the amount of time required is a good topic for debate. Even a time as short as one or two minutes would be sufficient to put a crimp in griefing.
The autocomplete on entry missions are just a bug in MA that should be fixed. An Ally goal set to to Single is not a legitimate goal, and arcs containing them should not be publishable. If there are similar immediately completable goals they too should inhibit publishing.
The goal is not to take away your right to poke someone in the eye. It's to eliminate griefing and voting cartels who can vote up undeserving arcs that they never even entered.
I feel that the rating system is irrelevant.
5-stars or even Dev's Choice are not a guarantee of an excellent arc. Hidden gems are now scattered among millions of search results and, honestly, I have no patience to look for them.
[ QUOTE ]
The autocomplete on entry missions are just a bug in MA that should be fixed. An Ally goal set to to Single is not a legitimate goal, and arcs containing them should not be publishable. If there are similar immediately completable goals they too should inhibit publishing.
[/ QUOTE ]
You mean "marked as required", not just "containing", right?
There are lots of ways that unescorted allies or captives can add to a story, to say nothing of those objectives that can't help but auto-complete by showing up (take a new character into an MA mission with ambushes, watch your "workaholic" progress bar go up).
Optional allies and captives without guards are just fine.
Up with the overworld! Up with exploration! | Want a review of your arc?
My arcs: Dream Paper (ID: 1874) | Bricked Electronics (ID: 2180) | The Bravuran Jobs (ID: 5073) | Backwards Day (ID: 329000) | Operation Fair Trade (ID: 391172)
[ QUOTE ]
[ QUOTE ]
You know, I thought during beta that you'd get HoF with 1000 4 or 5 star votes.
[/ QUOTE ]
You have ears that turn bad ideas into good ones, my carrot-seeking friend.
[/ QUOTE ]
Well, it might fix the "nobody in HoF ever" problem but it isn't going to fix the "grief downvote out of the ratings sort" problem.
And you could pretty easily argue that unless you go viral the latter problem is going to lead to the former anyway.
Up with the overworld! Up with exploration! | Want a review of your arc?
My arcs: Dream Paper (ID: 1874) | Bricked Electronics (ID: 2180) | The Bravuran Jobs (ID: 5073) | Backwards Day (ID: 329000) | Operation Fair Trade (ID: 391172)
Some of us rate something one star because it's bad. It's not up to you to decide whether you think it was 'justified' - ratings are subjective, which means there is no right or wrong way to rate a story arc. If I think it sucks, I'm allowed to express that opinion.
I question the existence of one star "griefing." I've heard this argument before - that there are cadres of malcontents who patrol the internet deliberating downranking everything out of sadism - and it sounds ridiculous. And some of these countermeasures you're suggesting are insane... It's like you want to start a witch hunt over this stuff.
"Are you now or have you ever been a one-voter?"
Better yet, let's force a costume change so all the one-voters have a star on their sleeves. Then we transfer all their characters to a laggy, buggy server so we don't ever have to see them. And we'll roll it back to that sweet spot in between ED and the IO system. No Patrol XP, takes longer to level, serves 'em right for disliking my story arc! And then, a few months later, we start running the delete script...
Don't submit something to a rating system if you can't handle getting a low rating.