So, how do you feel about the rating system?
By
Alari_Azure,
Posted on: 2009-05-10 18:28 in Mission Architect
It seems like you're rolling two issues into one here. The first is the one-star grieffing, which needs to be stopped by the Devs; being able to know who's doing the damage isn't going to stop most from doing so, nor are any actions that could be taken by the player community.
The second issue is that of rater anonymity, which I feel should be maintained. Authors want folks to honestly rate their arcs, for a variety of reasons. Those who want to give feedback can, and in so doing make their identity known, but you shouldn't force a rater to reveal such.
[ QUOTE ]
It seems like you're rolling two issues into one here. The first is the one-star grieffing, which needs to be stopped by the Devs; being able to know who's doing the damage isn't going to stop most from doing so, nor are any actions that could be taken by the player community.
[/ QUOTE ]
Yes I agree that "Star Grieffing", is a problem, but I do think that knowing who is the one behind that rating, could help stop some of it.
Mainly because if they have a arc, they surely would not want the same thing done to theirs.
I have yet to be star griefed on mine, and the majority of the players have left me great feedback, as well as 4-5 star ratings.
But I have no doubt that it is being done to others, and the only solution would be putting a face (Or global) to the griefer IMO, but YMMV.
[ QUOTE ]
Im wondering how people feel about the fairness of the rating system for the arcs?
[/ QUOTE ]
The rating system is totally bogus.
It is completely based on popularity and "You give me a 5, and I'll give you a 5" (the old "manus manum lavat" in action - sorry, M8)
It has nothing to do with how good an arc is nor does it give players the ability to find good arcs.
[ QUOTE ]
Do you think that people who rate your arcs (a 0-5), should have their global name attached so you know who gave you what?
[/ QUOTE ]
Definitely not.
It would be a great way for some to make sure that others are giving them a 5 if they gave them a 5 - so I could see why some would want it for that reason.
[ QUOTE ]
Or is it fair the way it is right now?
[/ QUOTE ]
It isn't fair in the least.
It definitely isn't balanced.
[ QUOTE ]
The reason I ask this is, a friend of mine who has a seemingly well liked arc (Kept a consistent 4 stars through about 40 plays or so), got a "0" rating while he was online.
[/ QUOTE ]
Someone could have just backed out of the interface without clicking it. If the game is catching not voting as 0 stars there is an issue. the rating system is from 1-5 not 0-5. 1 should be the lowest rating an arc can get.
It appears to be a bug. Bug it in game if you see it happen. I have not seen evidence of this myself.
[ QUOTE ]
He knew this because he saw that it had been rated and went to claim tickets, and there were none. (now granted something could have happened, but there was no feedback left regardless).
[/ QUOTE ]
He should have /bug-ed it if he was sure it should have been given stars.
[ QUOTE ]
So do you feel its fair that when a player stars your arc, but does not leave feedback, you don't know who it was?
[/ QUOTE ]
You don't know how many stars someone gives you, unless they tell you in the feedback.
I definitely don't think that you should be given a list of players that have played your arc. It appears to me to be a method of keeping track of the "I'll 5 star yours, if you'll 5 star mine" crowd. They don't need more tools to help them manipulate the system further.
[ QUOTE ]
Or is it fair like it is now?
[/ QUOTE ]
It isn't fair now.
Your way does nothing to improve the "fairness" of the rating system.
[ QUOTE ]
I personally see both sides of the story, and would like to see what others thoughts are on the rating system as it stands right now.
Discuss....
[/ QUOTE ]
The rating system so off at this point that I think either thumbs up or a thumbs down rating system would still be manipulated too much to be useful.
I don't have the brain power at the moment to dwell on a solution to this too much.
The DEVs need to find a way to do it or hire someone that can figure out a way to do it.
No doubt, plenty of players will think up good ideas to resolve this issue.
Part of the problem is that there are SOOOOO many arcs out there that there is no way to tell what is what. There are 5 starred mission that are garbage and missions that haven't been played that are lost gems.
You can't tell about any of them until you play them. You can't tell what level they were intended for until you play them. It's pretty crazy at this point.
I'm sure that it will all shake down after a while. 
I hate it. It takes way too many 5 stars to counterbalance a single zero-star. It needs redone for balancing and so that a couple of people loading it up and exiting from it doesn't hit it with zero stars and drop you pretty hard.
I do believe it takes 11 5-star votes to counterbalance a single zero stars.
"If you build it, they will run you over with it."-RPG Designers Mantra
Working on: YotZ Legends: Even Heroes Die (First Round Edit)
1 to 5, simple elegant rating system. I like it, removing anonymity would hamstring the variations of responses people might give. I'd rather see comments made anonymous too.
The problem is what the ratings do, that is what opens the door for manipulation and a bit for griefing.
1. Hall of Fame should not be based on ratings. That should be a straight "vote for Hall of Fame." 1000 HoF votes and it's in permanently.
2. The game should never take into account anything but the whole number rounded average score.
3. A search should by default show all arcs of the selected rating; if no other sort criteria is selected the list should be randomized for presentation.
4. Hope the search enhancements in i15 help a lot. Having focused searches will allow people with particular interests to stay in their criteria zone with less cross rating. A challenge player 2 stars an easy story arc, etc.
5. Without 4, it is going to be hard to identify w/ datamining whether it is griefing, manipulation or an interest mismatch.
6. Make zero go away.
p.s. OP, it takes 3 stars or more to generate tickets.
I feel the rating system is easily abused, at least for me. I run PLs in Mercy Island. At the end, all I ask is that they rate my mission 5 stars, some do, some don't. Doing this over and over allows fresh people to get in and rate it. I'm real close to that one.. Legendary whatever star badge and my "arc" is a "farm" mission.
The way I am doing it takes the amount of time I put into it to get the badge. A normal player who just makes an arc would take... a lot of promoting to get to that status.
Also, the stars don't mean anything unless it's over 100 ratings. So if an Arc has 4 stars and 5 plays. That just means he/she ran it with friends and they all gave it 5 stars except for a few randoms who gave it something lower. Doesn't tell you anything and just leaves to griefing with 1 starring people.
Also, Rate Cartels inflate missions further. So, accuracy wise they don't tell you anything exact. Just a broad, meh... it could be good?
If that was the intention then the system is working 
I ignore it.
The more interesting a story is, the likelier it becomes that someone will dislike it. People are just going to like and dislike different things. Trying to please everybody is a recipe for boredom.
The more innovative and strategic you make your gameplay, the likelier it is that the player just sent to the hospital will quit in disgust and give you a rating born of anger and frustration.
Badges won by ratings are nice, but I expect my progress is going to be slow.
<《 New Colchis / Guides / Mission Architect 》>
"At what point do we say, 'You're mucking with our myths'?" - Harlan Ellison
Everybody (should) know that the dice are loaded by now.
From the griefing on the low end to the buddy boy padding on the high end, the whole rating thing is indeed entirely bogus.
A few may still be naive enough to believe otherwise (like I recently read a theory that ppl were being knocked out of the HoF based on "expectations not being met" rather than griefing... yeah riight.)
Lots of others know it's a "game within a game" and play it as such (on the high side for badges, or just to be good to friends and the low side to grief). Some, like me, just refuse to rate at all until this is fixed.
One man's terrorist is another man's freedom (or freem?) fighter; just as one man's exploit is another man's feature.
listos version: online polls are useless, whether in game or not
At first I was concerned about ratings, but over time I realized that the bar for 'hall of fame' is so high, in sheer 'people trying out your arc,' that I honestly couldn't give a spit.
Whether 1 star or 5, pretty much my arcs are only going to be run by friends or random forum folks who happen to notice my sig and think it sounds vaguely interesting.
So when I load up a mission, get into it, decide it's not to my taste, exit, and exit without giving it a rating, it's actually giving it a 0-star rating that counts against the average?!?!?
That is borked. I have no idea how many people I have unintentionally 0-starred.
Keep anonymity on ratings. Too many people would abuse beeing able to pursue the raters. I wish feedback could be made anonymous, too.
The system is too easy to grief. I was a very vocal member of the "It won't be that bad, a few people won't be able to affect the system that much" camp. I was wrong.
[ QUOTE ]
The ability to zero star something has to go...
[/ QUOTE ]
This. This is the first thing that needs to happen. If the scale is 1-5 then 0 needs to not count.
Any time there's any sort of material gain involved in these games there are people who will abuse the feature for their benefit and attack those who are in competition with them.
Giving arcs high ratings provides a very tangible benefit -- tickets. Giving arcs low ratings provides a less tangible detriment to the recipient -- a low rating -- but a tangible benefit to someone who is direct competition with the person receiving that low rating-- a higher relative rating.
I have seen quite a few instances of well-written arcs that are given zero or one stars (you can tell this when they start with a five which changes to a three on the second vote). These arcs function properly, have clearly stated goals that are attainable, are well-written and basically do what the description says.
The possibilities are: 1) someone accidentally rating it zero, or there is a bug that causes zero ratings if you just exit, 2) someone is intentionally running around rating arcs zero for whatever reason, 3) some aspect of the arc really ticked the person off and he zeroed it out of spite.
To my mind, if an arc delivers what the description promises it should get at least a three. The only time it should be zeroed or oned is when it uses some hideous gimmick (the infamous 5 chained AV spawns on a huge outdoor map) that makes it unplayable or excruciatingly frustrating, is socially or morally offensive while lacking any artistic merit, is so poorly written and constructed so as to be unintelligible, is a blatant farm with no story, and the like.
If we wanted honest ratings and comments about the arcs, the comments would be anonymous (to us, but traceable back to the original writer by the devs). There would be no tangible benefits from ratings (no tickets, no badges, no Hall of Fame, nothing except Dev's Choice to allow good arcs to become permanent additions to CoH). There would only be the rating, and nothing else. That still wouldn't make the ratings "fair," but it would remove the motivations for many abuses of the rating system.
Yes, there would still be people zeroing competing arcs out of spite to have a higher rating, voting cartels, vote swapping, etc., but it would just be for the ego boost and not to get tickets or badges.
But now that they've implemented this system, I don't see them retracting it. It should be adjusted in some manner. One change that should be made: if you never actually play the arc your vote should not be recorded.
Right now, if you click the PLAY button and then immediately quit the arc and vote, your vote is recorded. This is patently wrong. Players can just run through their "enemies" list and zero vote dozens of arcs in minutes. Similarly, you can easily run through your friends list and give them 25 tickets per arc every week (at least the last I read that was how it worked).
Your vote should only be recorded if you actually played at least part of the arc. How much you have to play is a good topic for debate. Basically, it should take long enough to make it not worth your while to grief or unfairly reward people. My guess is it should be one mission (considering that some arcs only have one mission).
Will people still grief others and unfairly reward their pals? Sure. But at least it will take longer, and that will make it self-limiting.
I think the ratings system should die in a fire.
But my problem is not the same as yours. Yes, ratings must be anonymous. And they need to give us back anonymous feedback, too. Because let's face it, even if you're a saint and wouldn't go looking for the 0-star rater's arcs to revenge-rate them, others would, and lex talionis is bad.
My problem is that the ratings system includes no way of knowing what other peoples' tastes, or mine, are. I've figured out what gives an average of 5 stars, I think, and I've about concluded that any arc likely to get 5 stars is one that I don't want to play. What I want is an arc that was rated 5 stars by people with similar tastes to mine. Anything short of that is not merely worthless, it's worse than worthless.
I stopped using the ratings and just do arcs recommended by friends, and posted on various forum threads. The system in the MA right now is not really that good.
I like the straight-up HoF idea. No worries about griefing, but just a 'should this be a hall of fame arc?' question, and perhaps something like 1000 votes for that.
Live arcs: 517377 and 517381
Virtue: Quickshot. Swiftwind. Aliuneidis. Gizmodeus. Dasher. Fiver. Inuit Acer. Daniel Darke. Cerebral Flame. El Halcon.
Intel Core2Duo 2.4 Ghz 4 GB RAM**NVIDIA Geforce 9600 GT set to 1280 x 1024**Windows Vista 32 bit
<QR>
I complained before that the 0 to 5 scale is a poor scale - no mid point and it provides a limited variability of responses. A 0 to 10 scale would be much more effective.
Another issue is that having one score to rank an arc is flawed. There should be multiple ratings - difficulty, storyline, farm, whatever - so that players can better choose what they are looking for. Currently there are issues of farmers one- or zero-starring arcs with extensive stories or of RPers one-starring farm arcs.
Another point is that the devs just threw in a rating system, attached badges to it and apparently didn't think about what that would mean. Where badges are involved, of course it is going to mean that competition for them is going to be increased.
So, in short, my feelings about the ratings system are on the negative side. :-)
[ QUOTE ]
So when I load up a mission, get into it, decide it's not to my taste, exit, and exit without giving it a rating, it's actually giving it a 0-star rating that counts against the average?!?!?
That is borked. I have no idea how many people I have unintentionally 0-starred.
[/ QUOTE ]
I tried this on the Training Room (I didn't want to zero-star someone's live arc), and it works the way it should. If you exit without clicking any stars no vote is recorded. So, unless it's broken on live, you haven't slighted anyone unintentionally.
[ QUOTE ]
Im wondering how people feel about the fairness of the rating system for the arcs?
[/ QUOTE ]
Indifferent. I pretty much pay it no mind. I don't even look what an arc is rated before playing. I use a lot of board recommendations and tag searches instead.
Dec out.
Correct. No vote counts as no vote and doesn't impact ratings. You have to click 1 star twice to vote zero and torpedo someone's arc (deservedly or not).
Arc 55669 - Tales of the PPD: One Hell of a Deal (video trailer)
Arc 64511 - The Wrecking Ball
Arc 1745 - The Trouble With Trimbles
Arc 302901 - HappyCorpse
Hopelessly flawed, especially since it's currently the only way to get more publishing slots (which are useless due to how those are implemented). I hope they're able to come up with something better.
And yes, simply quitting or not rating an arc does not produce a 0 star rating. You have to deliberately click one star and then unclick it. And people are, whenever anyone gets on the front page.
My characters at Virtueverse
Faces of the City
[ QUOTE ]
The possibilities are: 1) someone accidentally rating it zero, or there is a bug that causes zero ratings if you just exit, 2) someone is intentionally running around rating arcs zero for whatever reason, 3) some aspect of the arc really ticked the person off and he zeroed it out of spite.
[/ QUOTE ]
My arc got torpedoed by the 0/1-star brigade. Not once, but twice.
It had made it up to 5-stars overall and was on page 3 of the basic listing.
Three 0/1-star rates (it was at least a 2 as there were no tickets waiting for me), no feedback, no progress on the "Players have completed your arc <x> times" badge, and my arc dropped down to 4-stars and was put down to at LEAST page 250.
Second time it had made it back up to page 4. Scenario repeated itself.
The only reason these people are cowardly enough to 0/1-star something is that they know they can get away with it. They know there is absolutely nothing that the owner of the arc can do. They can't ask them to rerate, they can't ask them to actually play it, they can't ask them WHY they 0/1-starred it.
[ QUOTE ]
Im wondering how people feel about the fairness of the rating system for the arcs?
[/ QUOTE ]
Steven King and J.K. Rowling could get together to create the most mind-blowing story ever seen in the MA, and get zero-starred as somebody did not like the power selections for the custom mobs.
The purpose of the rating system is to answer the question, "Is this arc good?". In this task of being able to produce a fair assessment of quality, the rating system fails horribly. All that it produces is the homoginized opinion of an a diverse group of people. Something I consider to have little value in this context. I view the rating system as a joke for this reason: The weight of the person who takes their time to perform a serious critique of the arc is given the same weight as the person who pops in for 10 seconds. Everybody has different tastes, and authors can get penalized if their arc is played by someone whose tastes run contrary.
Saturday night I published an arc that I had made for sg mates. As we were gathering our team it recieved a "0" star...how strange I thought as I had just published it and there was no time to have finished this 3 mish arc in 2 mins since it had been published.
I thought no big deal why not pull it and republish while we gather our team. So I pull and republish.
10 seconds later my hubby noticed that it again went from no stars to 1 vote of zero...ok...strange.
So I pulled and republished the thing again to count how long it would take before our mystery voter would "ding" our practice mish. 14 seconds
So this is getting interesting...I pull it again and republish and 16 seconds later "ding"
I would call this abuse...they could at least have finished the first mish of our practice arc...or not voted at all..It would explain why perfectly reasonable arcs might not get past 4 stars.
So while I have no problems with the rating system I do have problems with folks being able to vote by stepping into a mish and..say 14 seconds later voting zero...for whatever reason. Perhaps there should be some base requirements to vote.
My 2 inf.
MA: Bikini Rescue! 1678
Arshalla Emp/Psi 50
Oracle Hestia Fire/Emp 50
Glass Spider Claws/SR 50
Edica Dual/Will 50
Merry Met Ice/Emp 50
Crouton Eng/Eng 50
Shadowfax Warshade 50
Klotho Mind/Kin 50
Lolth' M/M Dom 50
Ten Ton Tomato Stone/SS Tank 50
Ruprecht Monkey Boy Fire/Kin Cont 50
Balefire Fire/Fire Blaster 50
Im wondering how people feel about the fairness of the rating system for the arcs?
Do you think that people who rate your arcs (a 0-5), should have their global name attached so you know who gave you what?
Or is it fair the way it is right now?
The reason I ask this is, a friend of mine who has a seemingly well liked arc (Kept a consistent 4 stars through about 40 plays or so), got a "0" rating while he was online.
He knew this because he saw that it had been rated and went to claim tickets, and there were none. (now granted something could have happened, but there was no feedback left regardless).
Now that got us into a debate, that I thought might be a great topic for here.
So do you feel its fair that when a player stars your arc, but does not leave feedback, you don't know who it was?
Or is it fair like it is now?
I personally see both sides of the story, and would like to see what others thoughts are on the rating system as it stands right now.
Discuss....