When are you going to fix the Ratings?
By
Alari_Azure,
Posted on: 2009-05-23 06:58 in Mission Architect
I'd hug you and give you cookies but there's a monitor and internet in the way, so instead I'll hug my monitor and smear chocolate chip cookies all over it.
*hugs* 
That was genuine and not sarcastic. Here's hoping I15 brings some ratings changes like tweaking or removing the star system. *crosses fingers*
If there is a rating system -- any player rating system -- in place, there will be high raters and low raters. Nothing one can do about that. What do you propose they do to change it?
Not everybody who gives us low ratings is griefing. They may pretty well hate our arcs, while others may love them.
However, if you get comments saying that they low-starred you because they don't like you on the forums, report them. That is harassment as well as griefing. It's not allowed by the rules.
But there is nothing programmatically that can be done about this. Programs can't gage the intent behind the button-pushing, they can only recognize what buttons are pushed and record it accordingly.
Programs can see that someone didnt even step foot inside your arc. People can rate your story just bt accepting it. They don't have to even go inside.
People can accept your mission go inside the first mission and rate it, without even killing a foe.
I am sure int his day and age our computers can figure out that accepting an arc and doing either of those things and rating it 0 stars... is griefing.
The ratings system is terrible and everyone knows this including the devs as they were told in a thread several hundred strong during test.
The development team and this community deserved better than this from NC Soft. Best wishes on your search.
I was one of those people who was cautiously optimistic about the ratings system in beta... but then it went live, and I quickly realized it's not working. It's deeply flawed. So, PP, you have my sympathies, and I too hope they rework the system for i15.
And for a while things were cold,
They were scared down in their holes
The forest that once was green
Was colored black by those killing machines
[ QUOTE ]
Programs can see that someone didnt even step foot inside your arc. People can rate your story just bt accepting it. They don't have to even go inside.
People can accept your mission go inside the first mission and rate it, without even killing a foe.
I am sure int his day and age our computers can figure out that accepting an arc and doing either of those things and rating it 0 stars... is griefing.
[/ QUOTE ]
And that's about the extent of what they could do.
However, it is entirely possible to identify really poorly done arcs or farm arcs very quickly; one needn't go completely through the mission in every case to know when they've hit something that is just bad.
For myself, normally I just quit such missions without rating, though there have been two or three that I one-starred for inappropriate content issues. Again, those were easily identified very early.
I do sympathize with ratings griefing (I've gotten one or two myself), it just seems that there's not much that can be done in many cases. It's not so much a technology issue, it's a people issue. There's no way a program can gage the intent of the rater.
[ QUOTE ]
However, it is entirely possible to identify really poorly done arcs or farm arcs very quickly; one needn't go completely through the mission in every case to know when they've hit something that is just bad.
For myself, normally I just quit such missions without rating, though there have been two or three that I one-starred for inappropriate content issues. Again, those were easily identified very early.
I do sympathize with ratings griefing (I've gotten one or two myself), it just seems that there's not much that can be done in many cases. It's not so much a technology issue, it's a people issue. There's no way a program can gage the intent of the rater.
[/ QUOTE ]
By requiring the voter to spend some minimum amount of time in the first mission (say, five minutes) the devs could eliminate most griefing and ticket padding as well.
Yes, it means we would have to wait five minutes to poke someone in the eye and give them a zero rating. But seriously, if we're not willing to give someone even the common courtesy of actually looking at some minimal part the mission, how much can our rating be worth?
[ QUOTE ]
[ QUOTE ]
However, it is entirely possible to identify really poorly done arcs or farm arcs very quickly; one needn't go completely through the mission in every case to know when they've hit something that is just bad.
For myself, normally I just quit such missions without rating, though there have been two or three that I one-starred for inappropriate content issues. Again, those were easily identified very early.
I do sympathize with ratings griefing (I've gotten one or two myself), it just seems that there's not much that can be done in many cases. It's not so much a technology issue, it's a people issue. There's no way a program can gage the intent of the rater.
[/ QUOTE ]
By requiring the voter to spend some minimum amount of time in the first mission (say, five minutes) the devs could eliminate most griefing and ticket padding as well.
Yes, it means we would have to wait five minutes to poke someone in the eye and give them a zero rating. But seriously, if we're not willing to give someone even the common courtesy of actually looking at some minimal part the mission, how much can our rating be worth?
[/ QUOTE ]
You're kidding, right? All it would mean is any Griefer would need to sit at the mission entrance for 5 minutes and/or kill one or two MOBs, and then give a zero start rating. Also, if you make it too long; people will not stick around to rate anything, and then authors wil also get frustrated as it seems no one rates their arcs.
As othershave said above you no matter what ratings system you employ; if ppeople want to grief it, they WILL find a way to do it. No amount of software checks/balances will ever stop that behaviour.
No matter how well you write something (or think you write something); you WILL ALWAYS GET three types of responses:
1) Someone will LOVE it, and think it's the best thing ever done.
2) Somone will be 'Meh'; and not really have a strong felling regarding the work either way.
3) Someon will LOATHE it and think is's the worst piece of garbage ever created.
There is no literary (or other type) of work released to the public that is 100% accepted in type 1) above...NONE.
When you're dealing with such a diverse audience (which IS what we have in this game); you will never have something that is 100% positively accepted. The best you can hope for is that more people react (and rate) favoribly then those who do not.
But again, as longer as human perception is involved; there will be no way to eliminate those who want to grief; nor can you expect everyone to give you a 'fair shke'. If a mission has a real pet peeve of someone's - they will probably give you a zero star rating an just move on.
The rating system is terrible. They need to remove the ability from players to downrate an arc at all. Either give it a "thumbs up" or no rating at all. You'll be able to see if there are a significant number of people that liked the arc, not some average that can be heavily skewed by a few griefers.
[ QUOTE ]
You're kidding, right? All it would mean is any Griefer would need to sit at the mission entrance for 5 minutes and/or kill one or two MOBs, and then give a zero start rating. Also, if you make it too long; people will not stick around to rate anything, and then authors wil also get frustrated as it seems no one rates their arcs.
As othershave said above you no matter what ratings system you employ; if ppeople want to grief it, they WILL find a way to do it. No amount of software checks/balances will ever stop that behaviour.
No matter how well you write something (or think you write something); you WILL ALWAYS GET three types of responses:
1) Someone will LOVE it, and think it's the best thing ever done.
2) Somone will be 'Meh'; and not really have a strong felling regarding the work either way.
3) Someon will LOATHE it and think is's the worst piece of garbage ever created.
There is no literary (or other type) of work released to the public that is 100% accepted in type 1) above...NONE.
When you're dealing with such a diverse audience (which IS what we have in this game); you will never have something that is 100% positively accepted. The best you can hope for is that more people react (and rate) favoribly then those who do not.
But again, as longer as human perception is involved; there will be no way to eliminate those who want to grief; nor can you expect everyone to give you a 'fair shke'. If a mission has a real pet peeve of someone's - they will probably give you a zero star rating an just move on.
[/ QUOTE ]
I had intended to say something snippy about Perfect's dog icon and then give his arc 0 stars out of faux spite, but your misunderstandings of human behavior and false trichotomy have distracted me with anger.
Anger I say!
I just felt like sharing.
You would think they would have learned from the forums themselves. Way back in the day, we could rate posters on these forums. This was removed due to griefing, so I was mildly surprised they used a system that has been proven to not work.
There have been many, many solutions given which I think would help. A few I've posted:
1) Implement a rating system that drops the top and bottom 10% of the ratings, like they do in some sporting events.
This would eliminate some of the 1-star bandits (i.e. Russia grading an American gymnast) , as well as SG buddies padding your score (i.e. Eastern block country voting on another in whatever sporting event ).
2) Create a separate star rating that includes only those votes from people who have completed the arc, then allow people to also search by that rating. ( i know that's the one I would search by )
3) Show statistics that show the number of people who started the mission vs those who completed it.
Lots of things they could do. Question is, do they really care?
131430 Starfare: First Contact
178774 Tales of Croatoa: A Rose By Any Other Name ( 2009 MA Best In-Canon Arc ) ( 2009 Player Awards - Best Serious Arc )
[ QUOTE ]
3) Show statistics that show the number of people who started the mission vs those who completed it.
[/ QUOTE ]
I like this minus one detail: show death count instead of start count. The reason is that people can still easily go in and quit as a form of griefing to lower the start/end ratios. Just show how many ended it as a mean of displaying true popularity (some one that didnt liked it may not finish it nor recomend it) and a kill ratio would just help provide a difficulty idea (not if the arc is good or not).
Actually, an arc with high death ratios and high complete count may point at an amazing story that no one would stop playing just because they kept dying.
Maybe the problem isn't so much 'the star system' as it is the '5 star system'. Perhaps it would work better if it were a 9 star system. That way the numbers would tend to even out a little more when averaged, and it would give people a better range of possibilities to choose for honest ratings. The percentage of ratings would balance out and give a clearer picture of what most people thought about it as the number of plays increased. If an arc could maintain a high rating (6/9 or higher) then it would seem to be held in higher esteem by the greatest number of players.
Just a thought.
No AV/EBs Deal with The Devil's Pawn-207266 Slash DeMento and the Stolen Weapons-100045 Meet the Demon Spawn-151099 Feedback
[ QUOTE ]
Maybe the problem isn't so much 'the star system' as it is the '5 star system'. Perhaps it would work better if it were a 9 star system. That way the numbers would tend to even out a little more when averaged, and it would give people a better range of possibilities to choose for honest ratings. The percentage of ratings would balance out and give a clearer picture of what most people thought about it as the number of plays increased. If an arc could maintain a high rating (6/9 or higher) then it would seem to be held in higher esteem by the greatest number of players.
Just a thought.
[/ QUOTE ]
You'd see the exact same thing that's happening right now, where the voters are essentially divided into three groups:
1) The Five-Starrers, who vote something at the top of its rating, either as a favor to their friends/SG mates or simply because they liked it.
2) The Grief Voters, who will vote something at its lowest rating because they can, or because they're trying to push their own arc to the top.
3) The Honest Voters, who try their best to be objective, but, as such a small minority, will be unable to swing the vote much one way or another.
A star rating is honestly a really bad idea for an online game like this.
It would be nice if we were allowed to publish anonymously instead of attaching our global name to our work.
Establishing greater minimum activity levels before allowing a rating is not an idea I support, for two reasons:
1) First impressions count, and it's no crime to rate something based on them. If the initial contact dialogue is offensive gibberish, I see no problem giving the arc a low rating without even setting foot in the mission. I agree that it's a little preemptive for my tastes, but my tastes are not and should not be the absolute law of how others may express their opinions.
2) The impact on ratings griefing would be minimal. If someone is going to go to the trouble of searching for your arcs, starting them, quitting them, and 0-starring them, they're going to go to the trouble of searching for your arcs, starting them, entering the mission, killing one mob, quitting them, and 0-starring them. No matter how much effort you force griefers to expend (5 minutes? 10 minutes? 3 mobs? 8 mobs?), they will still do it.
The combination of 1 & 2 is what kills the idea for me. The more effort you force from griefers to low-rate something, the more you kill the honest expression of people who honestly hate the beginning of an arc and want to rate it accordingly.
In your case, P_P, the best advice I can really give is to seriously consider changing your global name in-game. As long as your infamous forum identity has enemies willing to grief you in-game, anything with that name attached to it in-game is going to get griefed. No matter what new rating system the devs come up with. I know you don't want to, and I agree you shouldn't have to, but realistically it would be a very good way to protect yourself. Please think about it.
Again, a lot of this could be avoided if we could publish anonymously, and I'd think it would be an easy thing to add to MA. Maybe campaigning for that would help solve this specific problem.
I also wholly agree with the poster above who suggested that you report each and every instance of harassment.
[ QUOTE ]
Lots of things they could do. Question is, do they really care?
[/ QUOTE ]
This
[ QUOTE ]
[ QUOTE ]
3) Show statistics that show the number of people who started the mission vs those who completed it.
[/ QUOTE ]
I like this minus one detail: show death count instead of start count. The reason is that people can still easily go in and quit as a form of griefing to lower the start/end ratios. Just show how many ended it as a mean of displaying true popularity (some one that didnt liked it may not finish it nor recomend it) and a kill ratio would just help provide a difficulty idea (not if the arc is good or not).
Actually, an arc with high death ratios and high complete count may point at an amazing story that no one would stop playing just because they kept dying.
[/ QUOTE ]
Honestly, who cares how many times people die? Showing a death count would first off, skew the ratings against people whose friends and SG-mates aren't powergamers. Let's face it, often our friends are the first ones we get to play our arcs. Also, an arc with few plays that an incompetent PuG just happens to stumble across and finish, would suddenly rack up a zillion deaths, giving the impression that it's difficult, even if it isn't. And then I would feel bad, if I run your arc on a squishy with the difficulty cranked up and die a lot.
So really, showing the number of deaths is meaningless.
Eva Destruction AR/Fire/Munitions Blaster
Darkfire Avenger DM/SD/Body Scrapper
Arc ID#161629 Freaks, Geeks, and Men in Black
Arc ID#431270 Until the End of the World
[ QUOTE ]
So really, showing the number of deaths is meaningless.
[/ QUOTE ]
Strongly disagree. I would love to see a faceplant ratio. Yes, outlying cases could skew that number (as with EVERY numerical tracking system), but I for one would find it highly informational.
So, to answer your previous question:[ QUOTE ]
Honestly, who cares how many times people die?
[/ QUOTE ]
Me, for one.
Note to self: Run ICF_Zombra's arc with Fire/Storm corr on difficulty level 4, playing like an idiot and not caring if I faceplant, ie my usual playstyle.
If I die a lot because I'm careless, do I rate you down for it? No. Will other people see my slew of deaths and decide your arc is difficult before even trying it? Probably. The same applies to people ignoring the "challenging" or "designed for teams" warnings in the arc description, or people running high-level arcs with lower-level characters.
Eva Destruction AR/Fire/Munitions Blaster
Darkfire Avenger DM/SD/Body Scrapper
Arc ID#161629 Freaks, Geeks, and Men in Black
Arc ID#431270 Until the End of the World
[ QUOTE ]
If there is a rating system -- any player rating system -- in place, there will be high raters and low raters. Nothing one can do about that. What do you propose they do to change it?
Not everybody who gives us low ratings is griefing. They may pretty well hate our arcs, while others may love them.
However, if you get comments saying that they low-starred you because they don't like you on the forums, report them. That is harassment as well as griefing. It's not allowed by the rules.
But there is nothing programmatically that can be done about this. Programs can't gage the intent behind the button-pushing, they can only recognize what buttons are pushed and record it accordingly.
[/ QUOTE ]
Set a flag somewhere to only allow rating if any of the following criteria has been fulfilled:
* At least one mission in the arc has been completed.
* The player has died at least once.
* The player has spent X amount of time inside a mission.
That should cover all the bases. It might not stop all the griefers but it would definitely slow them down.
Winner of Players' Choice Best Villainous Arc 2010: Fear and Loathing on Striga; ID #350522
OK, check this out: you know how there's this perception on the forums that "many plays, 4 stars" are the best arcs to go for? That's there for a reason: because the player base is coming to understand what ratings really mean. The same thing will/would happen with other types of information that become available.
No one is going to look at a report that says that 1 person played through my arc and died a billion times and assume that that's it, that arc is definitely too hard. One person doing ANYTHING is not going to be good information for anyone - just like, right now, an arc with a single rating of 5 stars, or 0 stars, or any number of stars, is not considered highly reliable information.
If thirty people play through my arc and die a billion times each, that may give players some actual good information. More importantly, it will give me some good information and will inform my decisions to edit existing arcs and build better new ones.
More information = good.
What I would like to see is a system in place where every X weeks/days the highest and lowest rating on an arc are thrown out. That is barring a total redo of the system to 10 stars or best of all 30 stars (3 categories).
WN
Check out one of my most recent arcs:
457506 - A Very Special Episode - An abandoned TV, a missing kid's TV show host and more
416951 - The Ms. Manners Task Force - More wacky villains, Wannabes. things in poor taste
or one of my other arcs including two 2010 Player's Choice Winners and an2009 Official AE Awards Nominee for Best Original Story
There are some decent and terrible ideas in this thread, but I just wanted to insert a couple things real quickly.
First of all, as devoted forum dwellers, you have to know that Issue 15 is partially going to be about revamping the way we find MA stuff, right? They've said as much. It's in news releases. It's all over this here interwebs. So those of you saying " they won't fix it they don't like us " just need to suck it up and encourage them to get I15 into testing ASAP. If what you want isn't in there or you don't feel they've done enough, whine then.
[ QUOTE ]
And yet, the reason I no longer even go to the AE buildings anymore doesn't seem to be on their list of stuff to fix. People who 0 or 1 star peoples arc for absolutely no reason other than to be mean. It has happened to me with all of my arcs so far.
People say, "Just make them for your friends". or "Don't worry about the stars".
It is pretty damn hard to have strangers play my arcs when people come along and low rate my stuff.
I don't ask my friends to play my stuff. I want HONEST ratings. I am sure if I asked them to rate it 5 stars they would. BUT that defeats the purpose of me working hard on my arcs. It's like cheating in a final exam.
Anyways, I could go on and on but I will cut this short and say... Something should be done about ratings griefers.
I haven't step foot inside the AE building for a really long time. I turned off my comments mail from the story arcs because people really were sending me tells like "i hate you" or "i 1 starred you because i hate you on the forums".
[/ QUOTE ]
I agree they should do something about the ratings griefers, but holy crap do you need to grow some thicker skin. Or maybe just some skin even.
Every time I've seen you post, it's a "someone done me wrong woe is me" situation or thread. I can remember the first time I saw you post a thread about the MA and the first thing I thought was "Wow, I should go vote her 0 because I really dislike her on the forums."
Of course, the difference with me is that I only thought it and didn't do it. I prefer the honest ratings too and only like to hand out ratings based on what I played. Your arcs honestly didn't sound all that interesting to me based on what I personally enjoy, so I didn't play them or vote for them one way or the other.
If you've turned off your comments, then they can't get to you anyway. Just ignore what your arcs are doing if you're going to get yourself in a twist about it and enjoy all the great ones out there. Heck, enjoy some of the terrible ones too. Finding something really bad can be like watching a Godzilla movie you get to punish the film makers for at the end. Go collect your tickets every so often, play them if you enjoy them, and maybe go outside once in a while. The weather's beautiful right now.
I see they are removing badges and fiddling with all kinds of stuff. I worked hard for my badges I got. I NEVER farmed anything in the MA. They removed farming for alot of people, tweaked rikti dolls which I still have no idea what those are.
And yet, the reason I no longer even go to the AE buildings anymore doesn't seem to be on their list of stuff to fix. People who 0 or 1 star peoples arc for absolutely no reason other than to be mean. It has happened to me with all of my arcs so far.
People say, "Just make them for your friends". or "Don't worry about the stars".
It is pretty damn hard to have strangers play my arcs when people come along and low rate my stuff.
I don't ask my friends to play my stuff. I want HONEST ratings. I am sure if I asked them to rate it 5 stars they would. BUT that defeats the purpose of me working hard on my arcs. It's like cheating in a final exam.
Anyways, I could go on and on but I will cut this short and say... Something should be done about ratings griefers.
I haven't step foot inside the AE building for a really long time. I turned off my comments mail from the story arcs because people really were sending me tells like "i hate you" or "i 1 starred you because i hate you on the forums".