The rating system is inadequate and other rants
By
AddamsFamily,
Posted on: 2009-04-13 06:48 in Mission Architect
Agreed on many points Whataguy.
With a badge tied to Hall of Fame though, five and four-starring will run rampant until people get their badges.
Questions about the game, either side? /t @Neuronia or @Neuronium, with your queries!
168760: A Death in the Gish. 3 missions, 1-14. Easy to solo.
Infinity Villains
Champion, Pinnacle, Virtue Heroes
On point 2, yes I agree. However, I feel it's a problem with MA because you can't scale the powers to the low levels.
Unlike the critters made by the developers, we don't have the ability to individually select powers or even the pool powers. It's really all or nothing. In fact it is difficult to make a nothing. I am sure there are countless authors who want no secondary power set for their custom minions and find they have no way of doing that.
Wait a sec, you're saying some people aren't good enough to judge your mish, and yet YOU don't give 5 stars? Lrn2jdg noob. 
The more people I meet, the more I'm beginning to root for the zombies.
[ QUOTE ]
Wait a sec, you're saying some people aren't good enough to judge your mish, and yet YOU don't give 5 stars? Lrn2jdg noob.
[/ QUOTE ]
You must have this board confused with the WoW boards, dOOd. Here, we treat each other with respect.
I'm saying that yes, some people should not be allowed to rate missions -- mine or anyone else's. The opinion of the village idiot should not be given the same weight as that of Aristotle. I am not saying that I am Aristotle, only that there needs to be a method to qualify people to rate missions. On Yahoo Answers, for instance, they have a simple "level" system that allows folks to do more stuff the more qualified they are. This is not rocket science.
I have only given 4 stars because I haven't played a mission yet that I thought was deserving of 5. A 5, in this system, is the pinnacle. There is no 6, so 5 had better be reserved for the deserving. When I play a deserving mission, it will get 5 stars.
[ QUOTE ]
You must have this board confused with the WoW boards, dOOd. Here, we treat each other with respect.
[/ QUOTE ]
I wonder which one is worse. WoW or EvE Online forums.
on topic the only Arcs I've rated poorly have been farm missions. If I see that an arc is trying to do something with a story, I give it more stars.
we definitely need to list the genre of missions - could be action or horror, but more importantly could be spoof, or humor. Maybe the guy intended it to be that dumb. But the player should be warned.
A problem right now is simply the low number of ratings most have. A "proper" rating system drops the outliers once you get enough ratings. If an arc has 100 ratings it should drop the top 5 and bottom 5. So if a few people try to skew the ratings it gets fixed.
But if you only have 2 ratings there isn't much that can be done.
No rating system is going to be perfect.
I think the 5 star approach is fine.
Lets keep the rating system simple, eh?
Any system of rating will never be able to make everyone happy and besides, after a while the ratings will even out when more people rate things.
[ QUOTE ]
I wonder which one is worse. WoW or EvE Online forums.
[/ QUOTE ]
WoW, though the potential for backstabbing leading to frothing arguments is greatly increased in EO.
And, on topic: I still don't understand why so many people are expecting that a rating of a certain number guarantees they'll like the result, particularly this early on when so many arcs have been rated by so few people. Not that lots of ratings is any guarantee... Dumb and Dumber grossed some insane number of dollars at the box office and I've heard a lot of people talk about how much they enjoyed it, but I thought it was horrible.
And that's why I don't really pay much attention to ratings.
One drawback of the internet is how it has trained so many people to think that one day is a long time.
You need to keep in mind one thing when dealing with those 5-Star missions that are just garbage: Farmers flock together.
I hit a mission yesterday explicity called a farm by the author. The description said "Standard Rates Apply". The Send Off Dialogue told the player to Rate it 5 Stars.
I other words, I've created a custom group that allows you to maximize your tickets now give me my tickets in return by rating this piece of doo-doo 5 Stars. (1 lieutenant, so all you get is lieutenants in mission, no other work done).
This one had a 5-Star rating and 18 Votes. The same thing is happening with a large number of missions that represent absolutely no effort on the part of the author.
If Brevity is the Soul of Wit, Why are You Reading This?
QR
The fact that there are over 21 pages of 5-star arcs, and 7 of 1-star (as of an hour or so ago), shows how pointless the rating system is. Just to find a 5-star mission to play seems to require looking through a couple dozen obvious farms first (a bit of an exaggeration, but, give it another week).
That being said, there is no functional rating system for a game like this. Earning the right to rate is just as gameable as anything else, and it further creates bad feelings for those who don't participate in whatever it is that grants said right.
@Doctor Gemini
Arc #271637 - Welcome to M.A.G.I. - An alternative first story arc for magic origin heroes. At Hero Registration you heard the jokes about Azuria always losing things. When she loses the entire M.A.G.I. vault, you are chosen to find it.
What is needed is multiple ratings. Role Playing, Difficulty, Soloability, Best Team Size(?), Plays to description (if it says it is a farming arc don't expect a lot of dialog but that doesn't make it a bad farming arc).
I don't know something like that. Problem is everyone is rating missions based on different criteria. If I only like solo and you only like 8 person teams we are going to have completely different experiences on the same mission.
Plus the fact that unless you get 5 stars you may as well not publish is an issue. There is SO much content out there most people are only going to run highly rated arcs. This is causing and will continue to cause a lot of grief for designers.
----------------------------
You can't please everyone, so lets concentrate on me.
[ QUOTE ]
You must have this board confused with the WoW boards, dOOd. Here, we treat each other with respect.
I'm saying that yes, some people should not be allowed to rate missions -- mine or anyone else's. The opinion of the village idiot should not be given the same weight as that of Aristotle. I am not saying that I am Aristotle, only that there needs to be a method to qualify people to rate missions. On Yahoo Answers, for instance, they have a simple "level" system that allows folks to do more stuff the more qualified they are. This is not rocket science.
[/ QUOTE ]
I disagree with you on an inherent level.
Although I must question, how exactly are you treating each other with respect when you deny them their opinions?
Let's Dance!
[ QUOTE ]
I disagree with you on an inherent level.
Although I must question, how exactly are you treating each other with respect when you deny them their opinions?
[/ QUOTE ]
I was referring to respectful forum posting. I reserve the right to disrespect purveyors of ridiculous ratings on undeserving missions.
Everyone can have their opinions. Their opinions just shouldn't be given equal weight.
However, I recognize that a "Rating Qualifier" system could be gamed too. Perhaps I am being unrealistic.
I am primarily interested in being able to search for well-designed story arcs that are, you know, GOOD. As in having a story, cohesive structure, being playable, and with a minimum of spelling and grammatical errors. At the moment, I am unable to easily find such MA missions, due to the ridiculousness of the rating system and sheer quantity of garbage that is published.
In fact, I have recently started combing the "unrated" missions and playing those, with some measure of success. I encourage anyone interested in raising the integrity level of the rating process to play and rate missions like crazy, and to be objective.
I also reiterate that published missions need to be flagged by the designer as Farm/Badge, silly/spoof, "serious," and experimental, so we as players can sift through them.
I think that a significant number of the poorer ratings are due to people not knowing what the difficulty of the arc is before clicking on 'play'; and not knowing how much more difficult custom mobs are. They enter a map, get surprised by the challenge. And after they get out of the hospital, give a one-star in anger.
What I have been doing is listing a level range in the title, and giving challenge ratings along with notice of special threats in the description.
example: Title (Levels: 35-50)
challenge: Solo (7 of 10), Team (5 of 10), Custom EB (7 of 10), multiple ambushes)
75% of the ratings are pure bunk.
First, there's the griefers, badge farmers and petty revenge types, (Still mad at someone over that Bank Mission PuG from 2007? Now's your chance to get even I suppose). Second there's their counterparts -- people in cahoots to give each other good ratings. Finally there's the people who knock down an arc because of some bug that the author had no control over.
Even with the remaining 25%, there's simply no way to get a consensus on what is a good arc.
Some people want the ultimate deathmatch against AVs that spawn amushes of AVs which hit so hard that your computer crashes. Other people want a good story. For the former, they'll rate an arc low because its "too wordy." For the latter, they'll complain its "too hard."
Right now the ratings system is like polling what people think of a hip-hop album when half the voters only like country music.
I've concluded that ratings work when there are lots. Just ignore everything with less than 25 responses and you'll be fine.
We need the system where they are hidden before a threshold to return, along with showing us what we rated in addition to the average.
[ QUOTE ]
The opinion of the village idiot should not be given the same weight as that of Aristotle.
[/ QUOTE ]
I wonder what Aristotle would think of that. I'm certain Plato would disagree with you, he uses dialogue with the less refined to teach.
[ QUOTE ]
I've concluded that ratings work when there are lots. Just ignore everything with less than 25 responses and you'll be fine.
We need the system where they are hidden before a threshold to return, along with showing us what we rated in addition to the average.
[ QUOTE ]
The opinion of the village idiot should not be given the same weight as that of Aristotle.
[/ QUOTE ]
I wonder what Aristotle would think of that. I'm certain Plato would disagree with you, he uses dialogue with the less refined to teach.
[/ QUOTE ]
I like the method you suggest where the ratings are invisible until a threshold is met.
Oh, and I believe you are referring to Socrates, not Plato (who was, essentially, Socrates' biographer). I'm not sure that Socrates stated many opinions per se -- his method of reasoning, rather than his opinions, are his legacy. I'm sure Socrates didn't regard himself as anything special, if my recollection of Plato is correct.
Regarding the opinions of the village idiot and Aristotle: the issue is not the village idiot's freedom of speech. The issue is whether there is a qualitative difference between opinions. On the subject of the theory of special relativity, one would likely value Albert Einstein's opinion higher than that of, say, Bobcat Goldthwait.
It is not oppressive, anti-democratic, or elitist to suggest that opinions have relative value. The tricky part is the assignment of criteria to determine that value. If Yahoo Answers can come up with a formula to allow certain users to have greater or more influential input, it makes sense to me that something analogous could be developed for the MA system.
[ QUOTE ]
It is not oppressive, anti-democratic, or elitist to suggest that opinions have relative value. The tricky part is the assignment of criteria to determine that value. If Yahoo Answers can come up with a formula to allow certain users to have greater or more influential input, it makes sense to me that something analogous could be developed for the MA system.
[/ QUOTE ]
Actually, it completely and totally is and I would rather look for a source like, say, the government of every country where people have the right to vote than a really terrible internet website for my guidance.
[ QUOTE ]
[ QUOTE ]
It is not oppressive, anti-democratic, or elitist to suggest that opinions have relative value. The tricky part is the assignment of criteria to determine that value. If Yahoo Answers can come up with a formula to allow certain users to have greater or more influential input, it makes sense to me that something analogous could be developed for the MA system.
[/ QUOTE ]
Actually, it completely and totally is and I would rather look for a source like, say, the government of every country where people have the right to vote than a really terrible internet website for my guidance.
[/ QUOTE ]
You're conflating universal suffrage with the quality or value of opinions. In democratic nations, citizens have the right to vote, but they don't have the right to have their opinions on fashion, economics, sports, or wombats taken at equal weight.
Are you really saying that two opinions are always equally valid?
Do you find the plethora of 5-star ratings on farm/badger missions to be a meaningful use of the raters' opinions for the entire system? Or perhaps you might agree with me that for non-farmers, those ratings are qualitatively worthless?
[ QUOTE ]
we definitely need to list the genre of missions - could be action or horror, but more importantly could be spoof, or humor. Maybe the guy intended it to be that dumb. But the player should be warned.
A problem right now is simply the low number of ratings most have. A "proper" rating system drops the outliers once you get enough ratings. If an arc has 100 ratings it should drop the top 5 and bottom 5. So if a few people try to skew the ratings it gets fixed.
But if you only have 2 ratings there isn't much that can be done.
[/ QUOTE ]
The flaw that I see is that right now people seem to be more into creating missions than playing them. That's fine and should level out over time. For right now though, yes a lot of us have our missions sitting there with 1, 2, or no ratings at all.
[ QUOTE ]
Lastly: a five-star mission should be unusual. I have not give a "5" to any MA mission I've played. I get the feeling that as long as the mission is functional, there are players out there giving 5 stars for EVERYTHING, which destroys the whole meaning of the system.
[/ QUOTE ]
Your 5 stars aren't the same as everyone elses. This is a very limited medium which by its nature plays down the story. Faulkner or Conrad would have hard times telling compelling stories where every 5 minutes you had to click the glowie or defeat all.
If a mission is well put together, and performs as advertised it gets five stars from me.
Tank Girl well done >>> Another bad Knockoff of (The Matrix, Raymond Chandler, Spartacus etc)
Having played dozens of MA missions, I can confidently state the following:
1. The rating system is a joke. I have played multiple 5-star missions that were nothing close to a real mission, with copious spelling and grammar errors, missing or inadequate text, ridiculous custom enemies, and missions designed specifically for ticket and/or badge farming. I found relatively few actual missions written by people with a grasp of the English language who were trying to actually, you know, write a friggin' story arc. Finding those diamonds among the MA cesspool was a small reward.
I suggest that the MA have some sort of flag where the designer can flag the mission as a "badger," a test, a joke/spoof, or an attempt at at actual real story arc. That way we players can filter through the stuff we're not interested in.
I further suggest that mission raters have to somehow "earn" their ability to rate missions. We need some way to qualify raters.
Lastly: a five-star mission should be unusual. I have not give a "5" to any MA mission I've played. I get the feeling that as long as the mission is functional, there are players out there giving 5 stars for EVERYTHING, which destroys the whole meaning of the system.
2. Custom Enemies are WAY too difficult for lowbies to handle relative to standard PvE gameplay. It is not uncommon in MA missions to face multiple enemy debuffs and foes with high-tier powers (i.e. "hard" and "extreme" settings). Even giving a bad guy the Assault Rifle/Devices powers at "normal" yields a minion that is about equivalent to a mid-20s bad guy. In general, Lowbies cannot handle multiple debuffs and/or multiple status effects. Consider that in normal PvE play, there are very few debuffs and status effects, and bad guys that do serious damage (damned and bone daddies, for instance) are boss-level opponents.
I suggest that anyone using custom enemies put in the description a suggested starting level range that is realistic.
3. Because of #2, I think we need more powersets on the lower end of damage output. We can't make Hellions or Skulls, for instance, because we don't have the wimpy pistol or club attacks. Not all of us want to make enemy groups with hero-equivalent powers.