So, how do you feel about the rating system?
By
Alari_Azure,
Posted on: 2009-05-10 18:28 in Mission Architect
[ QUOTE ]
My arc got torpedoed by the 0/1-star brigade. Not once, but twice.
It had made it up to 5-stars overall and was on page 3 of the basic listing.
Three 0/1-star rates (it was at least a 2 as there were no tickets waiting for me), no feedback, no progress on the "Players have completed your arc <x> times" badge, and my arc dropped down to 4-stars and was put down to at LEAST page 250.
Second time it had made it back up to page 4. Scenario repeated itself.
The only reason these people are cowardly enough to 0/1-star something is that they know they can get away with it. They know there is absolutely nothing that the owner of the arc can do. They can't ask them to rerate, they can't ask them to actually play it, they can't ask them WHY they 0/1-starred it.
[/ QUOTE ]
My arcs were ganked the same way, on 2 different arcs. They were both hovering in the top 5 of page 2 when both of them mysteriously had strings of zero-star accidents. They haven't gotten out of the ghetto yet, although 1 of them subsequently got MA patched to near unplayability due to undocumented changes being pushed live.
[ QUOTE ]
One of the problems with the MA is that ratings are too important, and they're really the only way of increasing (or decreasing) the visibility of an arc. Honestly, I'd prefer to see some sort of categorization when I open up the MA, rather than a list of Dev's Choice and some inflated 5-star arcs.
[/ QUOTE ]
^This^
[ QUOTE ]
Except that leaves "excellent but unadvertised" arcs in the same bin as "terrible crap that 600 people have already played and abandoned".
Under that system, I don't want to be the author of the "excellent but unadvertised" arc, and I don't want to be player #601 of the "terrible crap" arc. We need to do something, but I don't think this suggestion is it.
[/ QUOTE ]
All you would need is one person to play through your arc and rate it to get it above the "abandoned crap" arcs.
I understand the reasoning behind allowing players to rate arcs they didn't finish, but I think the downside outweighs the up. Mission creators don't just want ratings, they want plays. Players abandoning the their arcs is already punishment to them, they don't need the extra negative reinforcement of one/zero-starring.
(And given that there's a "trick" to zero-starring, that option doesn't seem to be intentional.)
[ QUOTE ]
Except that leaves "excellent but unadvertised" arcs in the same bin as "terrible crap that 600 people have already played and abandoned".
Under that system, I don't want to be the author of the "excellent but unadvertised" arc, and I don't want to be player #601 of the "terrible crap" arc. We need to do something, but I don't think this suggestion is it.
[/ QUOTE ]
I don't think the above suggestions are the ONLY thing that needs to be done but I think it should be clear to anyone that allowing folks to rate arcs without playing them is inviting ganking. Add in the fact that the ratings are basically meaningless (my 1 star arc could be your 5 star - simply because you love nemesis arcs and I hate them) other than controlling where they show up in a search and potential hall of fame status (which is ajoke) and I don't see why you would continue to defend the status quo.
Seriously - what we really need is a decent search engine with meaningful tags to speed up searching. Missions creators should be able to flag things like a level range for missions, what archetypes it is designed for (if not all), difficulty level, AV/EB/Boss contents, etc.
If we have a decent search engine I won't give a fig what ratings an arc has - I will search for arcs that fit my characters archtype/level/desires and screw the ratings system.
Globals: @Midnight Mystique/@Magik13
By the same token, all a crappy arc needs is one friend/SG mate/significant other/second account to rate it ANYthing, and it's permanently out of the zero-star ghetto, since under the proposed no-finish-no-vote system, there's no way to vote it down again except by suffering through it, which most people won't do.
Arc 55669 - Tales of the PPD: One Hell of a Deal (video trailer)
Arc 64511 - The Wrecking Ball
Arc 1745 - The Trouble With Trimbles
Arc 302901 - HappyCorpse
[ QUOTE ]
By the same token, all a crappy arc needs is one friend/SG mate/significant other/second account to rate it ANYthing, and it's permanently out of the zero-star ghetto, since under the proposed no-finish-no-vote system, there's no way to vote it down again except by suffering through it, which most people won't do.
[/ QUOTE ]
Yeah, I realize the same arguments can be made either way, but I think that's all the more reason to go the less abusable route.
The rating system becomes more meaningful if players have to put a little bit of time into it, and it would better protect the legitimately good arcs from being dragged down by griefers.
[ QUOTE ]
By the same token, all a crappy arc needs is one friend/SG mate/significant other/second account to rate it ANYthing, and it's permanently out of the zero-star ghetto, since under the proposed no-finish-no-vote system, there's no way to vote it down again except by suffering through it, which most people won't do.
[/ QUOTE ]
Your arguement doesn't hold water - if it is easy for me to give a bad arc a low rating it is ALSO easy for someone to give the same bad arc a good rating. At least in the previous proposal we prevent easy ratings ganking and someone at least has to play through the crap arc in order to 'boost' it out of the zero rating level.
Also, what would you propose instead? If you think the current system is ok we have nothing to discuss - if you don't then propose an alternative.
Globals: @Midnight Mystique/@Magik13
[ QUOTE ]
By the same token, all a crappy arc needs is one friend/SG mate/significant other/second account to rate it ANYthing, and it's permanently out of the zero-star ghetto, since under the proposed no-finish-no-vote system, there's no way to vote it down again except by suffering through it, which most people won't do.
[/ QUOTE ]
Haven't read all of the posts here so please forgive me if I repeat something.
I've accepted from the beginning that the rating system is purely subjective. As some others have said "what you like, I might not". But I do feel that it can be a good rough guide for choices.
I actually make a point of trying to play at least one unrated arc a session. The percentage of good-average-bad has been about what you might expect.
I do try to rate everything I play AND send comments to the creator as to likes and dislikes. My hope is that this will encourage people to improve or, possibly, realize that this is not the avenue for creativity that works best for them.
I'm operating under the basic idea that an arc with a number of 4-5 star ratings might just be worth looking at. However, if the description doesn't sound appealing or, worse yet, no description at all, I still won't play it.
Perhaps this was the original thought behind the rating system?
Writer of In-Game fiction: Just Completed: My Summer Vacation. My older things are now being archived at Fanfiction.net http://www.fanfiction.net/~jwbullfrog until I come up with a better solution.
[ QUOTE ]
if it is easy for me to give a bad arc a low rating it is ALSO easy for someone to give the same bad arc a good rating
[/ QUOTE ]
That is correct. Under the current system all arcs are innocent until proven guilty, but if the arc is lousy, people will know it soon enough. The only way to keep a lousy arc on top would be to have an unending stream of shills who are willing to continue to vote the bad arc out of well-deserved anonymity. If we do it no-finish-no-vote, all you need is one shill.
[ QUOTE ]
At least in the previous proposal we prevent easy ratings ganking and someone at least has to play through the crap arc in order to 'boost' it out of the zero rating level.
[/ QUOTE ]
Yes, someone would have to play it. ONCE. You would decrease ratings ganking by making it harder to do, but at the same time you'd camouflage crappy arcs that had found their one shill and decrease the chances that anyone could vote it down by making it harder to do.
I recently played a MA arc in which every mob was a /Ninjitsu Elite Boss. Playing solo, I was ganked by three invisible EBs as soon as I entered the mish. Mind you, this wasn't marked as a "challenge" arc, it was just an unadulterated player-killer. Should I have had to slog through five missions worth of EBs to have my say? What about the cargo ship arc full of bosses that were generated by hitting Random in the costume generator, and in which the Mission Text was "a"? I appreciate the sentiment to try to curb griefing (I've been ratings-griefed myself), but in my opinion forcing players to complete crap arcs to vote is not the way to do it.
Arc 55669 - Tales of the PPD: One Hell of a Deal (video trailer)
Arc 64511 - The Wrecking Ball
Arc 1745 - The Trouble With Trimbles
Arc 302901 - HappyCorpse
[ QUOTE ]
(And given that there's a "trick" to zero-starring, that option doesn't seem to be intentional.)
[/ QUOTE ]
As Positron has specifically pointed out that it can be done, and how, I wouldn't be too sure about that assumption.
I'm with Emberly on this (see posts upthread). The stars just gotsta go. Making it so people have to play through the whole arc in order to rate it won't fix the current system. A couple of the current problems will be diminished, a couple of new problems will be introduced, and a handful of other problems will remain unaffected. Overall, it'll still be broken: not fulfilling its intended purpose.
The Cape Radio: You're not super until you put on the Cape!
DJ Enigma's Puzzle Factory: Co* Parody Commercials
OK, here's an alternate proposal. Three or four categories with a simple yes, no, or abstain.
Good Story Y/N/A
Challenging Y/N/A
Fun Y/N/A
The arc search screen gives an option to assign sort priorities and for each category decide whether you want to sort by number of votes or by ratio.
Possibly more or different categories.
[ QUOTE ]
Is there any indication that these concerns will be addressed? I am still concerned about the 14 seconds after it was published "0" votes that I recieved....repeatedly (please see my earlier post on this thread)
[/ QUOTE ]
Yeah, that stinks. One things the devs promised early on was that they were adding a lot of datamining tools so that they could find and punish people who did this. I haven't heard anything about that since the system went live, though. If they are punishing people for ratings griefing, they are doing it very quietly.
Avatar: "Cheeky Jack O Lantern" by dimarie
[ QUOTE ]
Except that leaves "excellent but unadvertised" arcs in the same bin as "terrible crap that 600 people have already played and abandoned".
Under that system, I don't want to be the author of the "excellent but unadvertised" arc, and I don't want to be player #601 of the "terrible crap" arc. We need to do something, but I don't think this suggestion is it.
[/ QUOTE ]
Not if the number of plays is also displayed. One of those would be played 0 times and rated 0 times. The other would be played 600 times but never rated. That would be a huge red flag to person #601.
Which, sadly, gives a new way to grief: start the mission, then quit. Over and over. It will look like the mission is unplayable, or not worth finishing.
Avatar: "Cheeky Jack O Lantern" by dimarie
[ QUOTE ]
Which, sadly, gives a new way to grief: start the mission, then quit. Over and over. It will look like the mission is unplayable, or not worth finishing.
[/ QUOTE ]
That's because, while trying to tiptoe around it, it doesn't change that it's basically a fancy thumbs up/down system.
Players' Choice Awards: Best Dual-Origin Level Range Arc!
It's a new era, the era of the Mission Architect. Can you save the Universe from...
The Invasion of the Bikini-clad Samurai Vampiresses from Outer Space? - Arc ID 61013
[ QUOTE ]
Under the current system all arcs are innocent until proven guilty, but if the arc is lousy, people will know it soon enough. The only way to keep a lousy arc on top would be to have an unending stream of shills who are willing to continue to vote the bad arc out of well-deserved anonymity. If we do it no-finish-no-vote, all you need is one shill.
[/ QUOTE ]
Except that we can see the number of times it's been rated, and we already know under the current system that the early rates (good or bad) aren't necessarily accurate.
Very, very few players will be able to get dozens of people to play through their terrible arc and 5-star it (unless of course, they don't actually think it's terrible.)
[ QUOTE ]
Allow anonymous *logged* feedback if you complete the arc.
[/ QUOTE ]
I think anonymous canned feedback might be better ie:
() Is that AV really necessary?
() Is that kill-all really necessary?
() This arc felt 1 mission too long
() Typo in mission intro for mission 1
() Typo in mission intro for mission 2
and alot of other common compaints.
That blue thing running around saying "Cookies are sometimes food" is Praetorian Cookie Monster!
Shoot on sight, please.
Qr - the best way to eliminate ratings griefing is for the devs to automate an MA lockout system tied to their datamining. If Johnny Griefer gets tagged by the system for starting X arcs, not finishing X arcs, and low-rating X arcs, with X being set at whatever the devs decide is 'too high', he should be autmaticallt locked out and sent an auto email informing him why, with the usual 'appeal to CS' being available to him.
This system should be widely advertised as being in place, and possibly some invented examples quoted to act as deterrents.
Eco
MArcs:
The Echo, Arc ID 1688 (5mish, easy, drama)
The Audition, Arc ID 221240 (6 mish, complex mech, comedy)
Storming Citadel, Arc ID 379488 (lowbie, 1mish, 10-min timed)
Quote:
|
Originally Posted by Samuel_Tow
[The Incarnate System is] Jack Emmert all over again, only this time it's not "1 hero = 3 white minions" it's "1 hero = 3 white rocks."
|
They need to remove the ability to "downrate" arcs.
Let people either select to "thumbs up" the arc or not rate it at all. There are too many arcs that get low ratings not because of anything wrong with the arc, but because of personal taste ("I don't like carnies, 1 star!" "I don't like office maps, 0 stars!")
Good ones would still rise, but arcs wouldn't be as subject to suffering from nonsensical ratings as they are now.
[ QUOTE ]
Very, very few players will be able to get dozens of people to play through their terrible arc and 5-star it (unless of course, they don't actually think it's terrible.)
[/ QUOTE ]
I can make a 1 mission arc right now where you walk in and you immediately get a mission complete.
The solution you're proposing won't do away with rating circles. We'll just get buried in an avalanche of "very short" arcs. You might change the complexion of the problem a bit, but it'll still be essentially the same problem: The current system is broken. If it's not going to fix the problem, I don't see the point in doing it.
The Cape Radio: You're not super until you put on the Cape!
DJ Enigma's Puzzle Factory: Co* Parody Commercials
[ QUOTE ]
They need to remove the ability to "downrate" arcs.
Let people either select to "thumbs up" the arc or not rate it at all. There are too many arcs that get low ratings not because of anything wrong with the arc, but because of personal taste ("I don't like carnies, 1 star!" "I don't like office maps, 0 stars!")
Good ones would still rise, but arcs wouldn't be as subject to suffering from nonsensical ratings as they are now.
[/ QUOTE ]
I'd rather they got rid of the ratings completely than go with a "fluffy bunny" approach. Most are finding a 5-star too limiting, and you would change it to a binary "good or nothing". "Nothing" would equal "sucks" to most people. It'd still be griefed by people entering the mission and leaving, giving a play and no vote. And it would do nothing to help the 5-star "griefing" in the other direction ("he's my SG mate, so 5-stars").
Dec out.
I think that any proposed change to the rating system comes with this understanding: No matter what is done, people will find a way to grief, exploit, and try to do things which are dishonest to the spirit of any quality indicator system. Some people are just jerks; and the badges, and tickets, only act as an enticement for this behavior. I wouldn't blame the devs for somebody's mother failing to raise them properly.
What I read as the major concern of this discussion is: 'How do I ensure that my arc is only being played by people who will actually give it a fair shot?" The best way I can think of is to hide the arcs from those who won't. A preference system wouldn't be out-of-line (and would be an excellent tool for finding what you want). Give players a checkbox to say what kind of mission they are looking for (i.e. no custom mobs, no AVs, or anything with a 'racy' theme), while giving creators the ability to designate their arcs with those same terms (or use the same information that is given in the basic arc description). This isn't a cure-all, but probably more of a stalling tactic to allow more 'honest' people to play before a 'griefer' happens upon it.
A major benefit I see with this is that it would allow people to avoid content that they do not wish to play (the lesbian hellion issue comes to mind with this), while also focusing on those that they want. This would also give the devs a tool, as they can now ask a person the question, "If you hate arcs with AVs in it so much, why do you keep on playing them?"
This is just an idea. But, I feel that the popularity of the MA lies with people being able to easily find the kinds of arcs they want.
The system has issues, but I think it realisticly is about as good as you can expect. Between personal preference, griefers, etc. any system is gonna have huge flaws.
[ QUOTE ]
I think that any proposed change to the rating system comes with this understanding: No matter what is done, people will find a way to grief, exploit, and try to do things which are dishonest to the spirit of any quality indicator system. Some people are just jerks; and the badges, and tickets, only act as an enticement for this behavior. I wouldn't blame the devs for somebody's mother failing to raise them properly.
What I read as the major concern of this discussion is: 'How do I ensure that my arc is only being played by people who will actually give it a fair shot?" The best way I can think of is to hide the arcs from those who won't. A preference system wouldn't be out-of-line (and would be an excellent tool for finding what you want). Give players a checkbox to say what kind of mission they are looking for (i.e. no custom mobs, no AVs, or anything with a 'racy' theme), while giving creators the ability to designate their arcs with those same terms (or use the same information that is given in the basic arc description). This isn't a cure-all, but probably more of a stalling tactic to allow more 'honest' people to play before a 'griefer' happens upon it.
A major benefit I see with this is that it would allow people to avoid content that they do not wish to play (the lesbian hellion issue comes to mind with this), while also focusing on those that they want. This would also give the devs a tool, as they can now ask a person the question, "If you hate arcs with AVs in it so much, why do you keep on playing them?"
This is just an idea. But, I feel that the popularity of the MA lies with people being able to easily find the kinds of arcs they want.
[/ QUOTE ]
This seems like a solid idea in theory, I like it.
Is there any indication that these concerns will be addressed? I am still concerned about the 14 seconds after it was published "0" votes that I recieved....repeatedly (please see my earlier post on this thread)
MA: Bikini Rescue! 1678
Arshalla Emp/Psi 50
Oracle Hestia Fire/Emp 50
Glass Spider Claws/SR 50
Edica Dual/Will 50
Merry Met Ice/Emp 50
Crouton Eng/Eng 50
Shadowfax Warshade 50
Klotho Mind/Kin 50
Lolth' M/M Dom 50
Ten Ton Tomato Stone/SS Tank 50
Ruprecht Monkey Boy Fire/Kin Cont 50
Balefire Fire/Fire Blaster 50