Mission Architect Ratings Analysis/Suggestions

First, I applaud you, Devs, for creating this wonderful Architect Mission System. It is the finest example of giving the power of creation to the people who play the game that I have seen ever seen in an MMO. I've already experienced many excellent arcs, enhancing my enjoyment of the game considerably. In fact, the Architect System is what likely caused many people (including myself) to renew my account.

I, like many, many others, have created my own stories. All of us, from those who are crafting epic storylines, to those who are simply making efficient methods to progress, put time and effort into our work. We expend a significant amount of creative energy in order to craft what, in our opinion, is the finest possible expression that can be achieved within the constraints of the system. We want to show what we can do. We all want recognition. We want to know that we have done well. And above all, we all want to have fun. That's the point of playing a game, right?

However, there is trouble in paradise. As much effort as we put into our creative works, we hope to have a method by which these works can be properly evaluated and credited by our peers, the rest of the City of Heroes community. The system we have in place is unfortunately, flawed. Some accuse others of unfair rating methods and others choose to intentionally give an arc a poor rating, regardless of quality. There is so much distortion involved with the system that it defeats the entire point. I've seen multiple instances of people who have made excellent arcs, with ratings to match, fall hundreds of pages because of a handful of the dreaded '0-star ratings' for no other reason that the story arc was reaching the front page.

I know this can't be the true intention of the system. People who do good work in the spirit it was intended should be given the reward they deserve.

Therefore, I ask that these alterations to the ratings system be made:

1: Categorized rating system - Please allow us to rate an arc on multiple factors (Perhaps on traits such as Story and Challenge). This will allow us to create a more graduated scale and possibly give us ways to filter for things we as a player are more interested in than others.

2: Require the completion of an arc before it can be rated - I know many have been concerned with the players who, for whatever reasons they have, rate an arc not according to its actual worth but on other, less ethical factors. Requiring the arc to be completed will reduce the convenience factor for these people, thus allowing a more fair assessment of the entire arc instead of your impression of the first 15 seconds.

3: Require comments for ratings under 3 stars - If someone believes that your arc deserves a poor rating, the least you can get in return is some constructive criticism on how to improve it. Additionally, if you feel an arc deserves a poor rating, you should be confident enough in your assessment in order to stand by what you say.

4: Remove the 0 Star Rating - This really does look like a bug with what someone has to do to make this happen, and is one of the most powerful tools that someone can use to ensure an arc never sees the light of day.

5: Append Comments To Arcs - This will allow us to keep track of commentary and determine if there is a common trend to critique, giving us the ability to distinguish between a one-off impression and an actual general commentary. This can be stored on a separate page so as to save loading time on the main search area. As a lead-in to this, perhaps the comments can be sent to mail instead of tells.

6: Unpublish all non-Developer's Choice arcs - This is the only fair way to make the field level again after the revisions are made to the rating system. DC arcs would be DC regardless, and thus wouldn't need to be unpublished. Everyone would be able to republish their arcs immediately, giving them a fresh start on the new rating system.

7: Include a system to contest ratings - Simultaneously, allow us to contest ratings that we feel have been given unfairly. If someone rates an arc poorly and gives no actual reason, we should be given the ability to have that rating removed from our records.

Please consider these options. Please, allow us to make the Mission Architect all it truly can be.

[ QUOTE ]
1: Categorized rating system - Please allow us to rate an arc on multiple factors (Perhaps on traits such as Story and Challenge).

[/ QUOTE ]

The more categories you add, the less likely I am to rate a mission. I'm not taking a job as an arc reviewer, I'm just playing to have fun. I'm not going to bother agonizing over the qualities of various aspects of the arc.

[ QUOTE ]
2: Require the completion of an arc before it can be rated

[/ QUOTE ]

If I can't complete the arc due to being hit with several surprise elite bosses, I certainly want to be able to quit and give a poor rating. In fact, if I can't quit and give a rating, I probably wouldn't bother rating any arc that I can complete. I'd just throw out the rating system altogether.

[ QUOTE ]
3: Require comments for ratings under 3 stars

[/ QUOTE ]

Comment: safdsadfsadf
Requiring a comment is meaningless, since you can't force people to form complete words and sentences.

[ QUOTE ]
4: Remove the 0 Star Rating

[/ QUOTE ]

Okay. There are probably arcs out there I would give 0 stars to, but I think the feature isn't implemented very well.

[ QUOTE ]
5: Append Comments To Arcs

[/ QUOTE ]

This sounds messy. I suspect they moved away from e-mails for a good reason.

[ QUOTE ]
6: Unpublish all non-Developer's Choice arcs

[/ QUOTE ]

Well that would be unpopular. I don't care about that myself, but I think a number of players might be coming for you with a knife if you convinced the devs to wipe out their ratings.

[ QUOTE ]
7: Include a system to contest ratings

[/ QUOTE ]

Creates excessive work for the GMs. Please consider the GM workload when suggesting something like this.

[ QUOTE ]
The more categories you add, the less likely I am to rate a mission. I'm not taking a job as an arc reviewer, I'm just playing to have fun. I'm not going to bother agonizing over the qualities of various aspects of the arc.

[/ QUOTE ]

Fair enough, but I don't really see more than 3 or so categories being available before they become redundant anyway.

[ QUOTE ]
If I can't complete the arc due to being hit with several surprise elite bosses, I certainly want to be able to quit and give a poor rating. In fact, if I can't quit and give a rating, I probably wouldn't bother rating any arc that I can complete. I'd just throw out the rating system altogether.

[/ QUOTE ]

Then the mission will sit there with no ratings and people won't play it. The rating system as it is now is is just as irrelevant.

[ QUOTE ]
Comment: safdsadfsadf
Requiring a comment is meaningless, since you can't force people to form complete words and sentences.

[/ QUOTE ]

If I had a 0-star comment like that, then it would be a legitimate reason for #7... Or perhaps even seeking other recourse.

[ QUOTE ]
Okay. There are probably arcs out there I would give 0 stars to, but I think the feature isn't implemented very well.

[/ QUOTE ]

I agree. And really, the 0-star is a loophole. Shouldn't the minimum be one star?

[ QUOTE ]
This sounds messy. I suspect they moved away from e-mails for a good reason.

[/ QUOTE ]

Possibly, but these are suggestions for a reason, not demands. Maybe there's a more efficient way to do this, but I'll have to think about it.

[ QUOTE ]
Well that would be unpopular. I don't care about that myself, but I think a number of players might be coming for you with a knife if you convinced the devs to wipe out their ratings.

[/ QUOTE ]

What ratings? The ratings as they stand now are completely flawed. Come after me with a knife if you want, sure, but just like revisions to powersets come with a respec, so would a revision of the rating system require a reset. And seriously, if these arcs are honestly good, they'll get right back to where they should be very quickly. As it stands now, I could play some very good 3-star arcs and find many very bad 5-stars.

[ QUOTE ]
Creates excessive work for the GMs. Please consider the GM workload when suggesting something like this.

[/ QUOTE ]

Perfectly understandable. But if a rating is invalid for some reason, shouldn't there be a way to remove it? Frankly, if someone's going to get 30 people to spam me with 0 stars with comments of 'oincaoinfaonfajnf', I'd want them all removed and those players possibly even banned from using the rating system for abuse. A ratings system requires some degree of oversight, otherwise we may as well not have one.

[ QUOTE ]
[ QUOTE ]
The more categories you add, the less likely I am to rate a mission. I'm not taking a job as an arc reviewer, I'm just playing to have fun. I'm not going to bother agonizing over the qualities of various aspects of the arc.

[/ QUOTE ]

Fair enough, but I don't really see more than 3 or so categories being available before they become redundant anyway.

[/ QUOTE ]

Three categories wouldn't be bad, if they were sufficiently distinct.

Just scrap star ratings altogether. If you're someone like Roger Ebert and can objectively look at something, both in terms of content and context (what is the movie trying to do, etc.), then star ratings work fine, but in an online environment like this a star rating is begging for trouble. A simple "thumbs-up" system (where you can thumbs-up something if you like it, or choose to not do anything) would work the best. The arcs that people like will gradually float to the top, and the ones that aren't that good will stay at the bottom.

0 Star really needs to go. It feels like a glitch in the way you have to activate it and it has the ability to tank a valid arc faster than it can be properly be raised back up.

[ QUOTE ]
Just scrap star ratings altogether. If you're someone like Roger Ebert and can objectively look at something, both in terms of content and context (what is the movie trying to do, etc.), then star ratings work fine, but in an online environment like this a star rating is begging for trouble. A simple "thumbs-up" system (where you can thumbs-up something if you like it, or choose to not do anything) would work the best. The arcs that people like will gradually float to the top, and the ones that aren't that good will stay at the bottom.

[/ QUOTE ]

You know, I would be happy with this. It'd be just like karma on Slashdot or comments on Youtube.

I had a similar suggestion on the rate in a couple of categories. I was thinking straight Agree/Disagree/No Opinion.

Example categories: Fun, Challenging, Good Story, High Quality

The key to making this work is if the searching allows for each of these as criteria. The second part is the option to include the Negatives as part of the search and sort criteria. E.g. You could choose to see arcs that have at least 100 Agrees to Challenging; or see only arcs that have > 80% rated as Fun.

For Hall of Fame, you add a simple checkbox for "Nominate for Hall of Fame." X# of nominations and it's in.

Under this scheme, you could grandfather in ratings or even retain them as an additional option.

As Mirai points out, the more complex, the more like work. Requiring comments would fit there, too. And in the A/D/N scheme, comments as a grief control doesn't matter; there wouldn't be much run or need for contesting.

So a rating system with six, possible ratings is a bug, but one that has only five is perfectly legitimate?

How does that make sense?

Or is it just that you want the devs to tweak the math so you can feel like your arc has a higher rating?

[ QUOTE ]
So a rating system with six, possible ratings is a bug, but one that has only five is perfectly legitimate?

How does that make sense?

Or is it just that you want the devs to tweak the math so you can feel like your arc has a higher rating?

[/ QUOTE ]

Can you show me a real star rating system that goes to 0 stars?

If they intended it like this, they wouldn't make you jump through those hoops to make it work that way in the first place. A 1 rating means the same thing, the only reason one would want to use a 0 rating is for maximum damage to the star rating.

[ QUOTE ]
[ QUOTE ]
1: Categorized rating system - Please allow us to rate an arc on multiple factors (Perhaps on traits such as Story and Challenge).

[/ QUOTE ]

The more categories you add, the less likely I am to rate a mission. I'm not taking a job as an arc reviewer, I'm just playing to have fun. I'm not going to bother agonizing over the qualities of various aspects of the arc.

[ QUOTE ]
2: Require the completion of an arc before it can be rated

[/ QUOTE ]

If I can't complete the arc due to being hit with several surprise elite bosses, I certainly want to be able to quit and give a poor rating. In fact, if I can't quit and give a rating, I probably wouldn't bother rating any arc that I can complete. I'd just throw out the rating system altogether.

[ QUOTE ]
3: Require comments for ratings under 3 stars

[/ QUOTE ]

Comment: safdsadfsadf
Requiring a comment is meaningless, since you can't force people to form complete words and sentences.

[ QUOTE ]
4: Remove the 0 Star Rating

[/ QUOTE ]

Okay. There are probably arcs out there I would give 0 stars to, but I think the feature isn't implemented very well.

[ QUOTE ]
5: Append Comments To Arcs

[/ QUOTE ]

This sounds messy. I suspect they moved away from e-mails for a good reason.

[ QUOTE ]
6: Unpublish all non-Developer's Choice arcs

[/ QUOTE ]

Well that would be unpopular. I don't care about that myself, but I think a number of players might be coming for you with a knife if you convinced the devs to wipe out their ratings.

[ QUOTE ]
7: Include a system to contest ratings

[/ QUOTE ]

Creates excessive work for the GMs. Please consider the GM workload when suggesting something like this.

[/ QUOTE ]

Im sorry, but your arguments, to me at least, seem to be the stongest support YET for the OPs ideas.

If you don't care enough about the arc to give it a real rating, then frankly, I don't care what your opinion is. When I look for arcs to play, I don't want the opinions of someone who just felt like "eh, I'll hit this button" but want the opinions of people who actually felt a strong opinion about an arc.

This is why we pay people for movie reviews instead of just going by ticket sales. We want honest, though out opinions.

[ QUOTE ]
6: Unpublish all non-Developer's Choice arcs - This is the only fair way to make the field level again after the revisions are made to the rating system. DC arcs would be DC regardless, and thus wouldn't need to be unpublished. Everyone would be able to republish their arcs immediately, giving them a fresh start on the new rating system.

[/ QUOTE ]

No. The DC arcs started with an unfair advantage at the publish of I14 by being made DC arcs at the start. They're right there on the first page in front of you and based on how some people talk about going through the list based on how "good" they're rated as being, that just makes it more unfair to other stories.

Do you honestly believe it'll be possible for any newly created story to make it to Hall of Fame and/or Developer's Choice without the kind of ratings push the Devs don't want to be done? Believe it or not, not everyone goes onto the Test server, and everyone certainly didn't have very early Beta access to work and tweak their stories before they were transferred over and made live, etc.

Having them as Dev's Choice from the beginning as well as even allowing them to be transferred over from Test to be published right at the beginning, is tied with the star system and tying them to badges for the Dev's greatest mistake regarding the handling of MA.

[ QUOTE ]
[ QUOTE ]
6: Unpublish all non-Developer's Choice arcs - This is the only fair way to make the field level again after the revisions are made to the rating system. DC arcs would be DC regardless, and thus wouldn't need to be unpublished. Everyone would be able to republish their arcs immediately, giving them a fresh start on the new rating system.

[/ QUOTE ]

No. The DC arcs started with an unfair advantage at the publish of I14 by being made DC arcs at the start. They're right there on the first page in front of you and based on how some people talk about going through the list based on how "good" they're rated as being, that just makes it more unfair to other stories.

Do you honestly believe it'll be possible for any newly created story to make it to Hall of Fame and/or Developer's Choice without the kind of ratings push the Devs don't want to be done? Believe it or not, not everyone goes onto the Test server, and everyone certainly didn't have very early Beta access to work and tweak their stories before they were transferred over and made live, etc.

Having them as Dev's Choice from the beginning as well as even allowing them to be transferred over from Test to be published right at the beginning, is tied with the star system and tying them to badges for the Dev's greatest mistake regarding the handling of MA.

[/ QUOTE ]

Ahhhhh, I was unaware of this. Hmm, that's a difficult situation then. Perhaps truly starting everyone from ground zero would be for the best? I'm not sure.

[ QUOTE ]
[ QUOTE ]
If I can't complete the arc due to being hit with several surprise elite bosses, I certainly want to be able to quit and give a poor rating. In fact, if I can't quit and give a rating, I probably wouldn't bother rating any arc that I can complete. I'd just throw out the rating system altogether.

[/ QUOTE ]

Then the mission will sit there with no ratings and people won't play it. The rating system as it is now is is just as irrelevant.

[/ QUOTE ]
Unfortunately, as a side effect someone who actually intended to publish a good mission would not be receiving any feedback, some of which might be useful to make their arc better and more enjoyable; as much as it makes it possible to grief players, being able to rate an arc without finishing it is something that's needed.

[ QUOTE ]
[ QUOTE ]
Comment: safdsadfsadf
Requiring a comment is meaningless, since you can't force people to form complete words and sentences.

[/ QUOTE ]
If I had a 0-star comment like that, then it would be a legitimate reason for #7... Or perhaps even seeking other recourse.

[/ QUOTE ]
However, there are other problems as well. Folks may simply not rate an arc, or rate such an arc as three or better for no other reason than to avoid having to give comments. Forcing people to something in an online game is one of the fastest ways of creating an unused feature. Even more so if they have to identify themselves as a result of what they are being forced to do.

[ QUOTE ]
[ QUOTE ]
This sounds messy. I suspect they moved away from e-mails for a good reason.

[/ QUOTE ]

Possibly, but these are suggestions for a reason, not demands. Maybe there's a more efficient way to do this, but I'll have to think about it.

[/ QUOTE ]
Give an author the option to create a text document for the purpose of recording such comments locally.

[ QUOTE ]
[ QUOTE ]
Well that would be unpopular. I don't care about that myself, but I think a number of players might be coming for you with a knife if you convinced the devs to wipe out their ratings.

[/ QUOTE ]

What ratings? The ratings as they stand now are completely flawed. Come after me with a knife if you want, sure, but just like revisions to powersets come with a respec, so would a revision of the rating system require a reset. And seriously, if these arcs are honestly good, they'll get right back to where they should be very quickly. As it stands now, I could play some very good 3-star arcs and find many very bad 5-stars.

[/ QUOTE ]
Damaging a player's efforts in such a manner are a bad idea. Look how many folks got up in arms with Positron's recent MA use statements. With the removal of certain maps and MA resources. No, if someone thinks they have been hurt by the existing system, they have the option of republishing their arcs when a new system is implemented. And if the new system does its job, those arcs that benefited illegitimately from the current system will eventually be brought in line.

[ QUOTE ]
[ QUOTE ]
Creates excessive work for the GMs. Please consider the GM workload when suggesting something like this.

[/ QUOTE ]

Perfectly understandable. But if a rating is invalid for some reason, shouldn't there be a way to remove it? Frankly, if someone's going to get 30 people to spam me with 0 stars with comments of 'oincaoinfaonfajnf', I'd want them all removed and those players possibly even banned from using the rating system for abuse. A ratings system requires some degree of oversight, otherwise we may as well not have one.

[/ QUOTE ]
You're simply never gong to see this level of oversight in an online game. There are more pressing concerns, especiallt when players have the option of republishing to clear the slate and start over. And a successful rating system shouldn't suffer from such grieffing anyway.

[ QUOTE ]
Ahhhhh, I was unaware of this. Hmm, that's a difficult situation then. Perhaps truly starting everyone from ground zero would be for the best? I'm not sure.

[/ QUOTE ]

Content was needed at the get go, and I understand and appreciate that. I personally think the initial stories that showed up on the MA when I14 went live for the first few minutes should have been done by the devs and NC staff. Anything that was Dev's Choice on Test should have been taken note of and after a short time of I14 being live if the players had published the stories live then they'd be given recognition.

I know I'm waffling a little bit. I don't want to upset people and say their work was all for naught, but at the same time I don't think it's fair to future creators, as any time that's passed since I14 that your story wasn't already done and tweaked and published is creating a disadvantage for you in regards to tickets, badges and notoriety. Especially in a system that rates a one 5-star voted story above a 500+ vote story averaging 4 stars.

Do Dev's Choice story authors get tickets for the votes they get? Because if they do then that either needs to stop or the Dev's Choice stories need to be moved off of the front page and into say a separate section just for Dev's Choice stories that's still easy to navigate to but doesn't slap them right in the average MA user's face as it currently is.

[ QUOTE ]
Unfortunately, as a side effect someone who actually intended to publish a good mission would not be receiving any feedback, some of which might be useful to make their arc better and more enjoyable; as much as it makes it possible to grief players, being able to rate an arc without finishing it is something that's needed.

[/ QUOTE ]

In my opinion, a rating system that allows someone to rate content they're not even experiencing is not a rating system at all.

[ QUOTE ]
However, there are other problems as well. Folks may simply not rate an arc, or rate such an arc as three or better for no other reason than to avoid having to give comments. Forcing people to something in an online game is one of the fastest ways of creating an unused feature. Even more so if they have to identify themselves as a result of what they are being forced to do.

[/ QUOTE ]

If you have a better solution handy on how to solve the problems with the rating system, I would be happy to hear them. I don't think I have all the answers, this is just the result of me giving it a little thought.

[ QUOTE ]
Give an author the option to create a text document for the purpose of recording such comments locally.

[/ QUOTE ]

Now this? This is a good idea.

[ QUOTE ]
Damaging a player's efforts in such a manner are a bad idea. Look how many folks got up in arms with Positron's recent MA use statements. With the removal of certain maps and MA resources. No, if someone thinks they have been hurt by the existing system, they have the option of republishing their arcs when a new system is implemented. And if the new system does its job, those arcs that benefited illegitimately from the current system will eventually be brought in line.

[/ QUOTE ]

Could you explain to me how that would be fair?

[ QUOTE ]
You're simply never gong to see this level of oversight in an online game. There are more pressing concerns, especiallt when players have the option of republishing to clear the slate and start over. And a successful rating system shouldn't suffer from such grieffing anyway.

[/ QUOTE ]

In a world where the playerbase is responsible, moral, and ethical, much like the heroes we pretend to be, yes.

I would love for the system to be made functionally immune to griefing. That's why I want to help bring this up to the attention of the Devs and playerbase once more. I want to have a helpful discussion on the issue so we can help design a system that everyone can be mostly (because completely is, of course, impossible) satisfied with.

Any suggestions you have to assist with this would be welcome.

[ QUOTE ]
Can you show me a real star rating system that goes to 0 stars?

If they intended it like this, they wouldn't make you jump through those hoops to make it work that way in the first place. A 1 rating means the same thing, the only reason one would want to use a 0 rating is for maximum damage to the star rating.

[/ QUOTE ]

So, just to clarify, by "maximum damage" you mean to say that the number isn't as high as you wish it were for some arbitrary reason?

Or do you just not understand math?

[ QUOTE ]
In my opinion, a rating system that allows someone to rate content they're not even experiencing is not a rating system at all.

[/ QUOTE ]
Perhaps, but a previous poster did state a time when such rating is called for, when an arc cannot be completed due to too many surprise EBs. I can elaborate on a couple of others. One mission I played was placed on a tech map, but destructible object spawns were so thick they blocked doorways. There was literally no way to bypass the objects and complete the mission. On another, every unit of a custom enemy group utilized heavy End drain. The mission was slow and a headache, but doable until I get to the end and there's a surprise EB, downgraded from AV level. Possessing the same End-drain abilities as the rest of the enemy group, he and his followers dropped me to zero End after only one attack. Four times. I couldn't even kill off a minion in the group before I was out of juice.

[ QUOTE ]
If you have a better solution handy on how to solve the problems with the rating system, I would be happy to hear them. I don't think I have all the answers, this is just the result of me giving it a little thought.

[/ QUOTE ]
Unfortunately, I do not. I was just stating my opinion on your suggestion; I think it would do more harm than good.

[ QUOTE ]
[ QUOTE ]
Damaging a player's efforts in such a manner are a bad idea. Look how many folks got up in arms with Positron's recent MA use statements. With the removal of certain maps and MA resources. No, if someone thinks they have been hurt by the existing system, they have the option of republishing their arcs when a new system is implemented. And if the new system does its job, those arcs that benefited illegitimately from the current system will eventually be brought in line.

[/ QUOTE ]

Could you explain to me how that would be fair?

[/ QUOTE ]
Fairness in fixing what's already been done to misuse the system is too much to ask for. Item rewards have already been received, badges have already been issued; the impact any such action will have on those that misused the system is negligible. The ones it would truly harm are those that legitimately rose above the rest, despite the obstacles. They're the ones that would be feeling the sting of such action, that would be most likely to leave the game as a result of such action. And they're the ones we want to keep around.

I think the best we can hope for is to leave things as is for now, but design a system that will repair the damage done to the system naturally, over time.

[ QUOTE ]
Perhaps, but a previous poster did state a time when such rating is called for, when an arc cannot be completed due to too many surprise EBs. I can elaborate on a couple of others. One mission I played was placed on a tech map, but destructible object spawns were so thick they blocked doorways. There was literally no way to bypass the objects and complete the mission. On another, every unit of a custom enemy group utilized heavy End drain. The mission was slow and a headache, but doable until I get to the end and there's a surprise EB, downgraded from AV level. Possessing the same End-drain abilities as the rest of the enemy group, he and his followers dropped me to zero End after only one attack. Four times. I couldn't even kill off a minion in the group before I was out of juice.

[/ QUOTE ]

Perhaps something to keep track of what percentage of people quit the arc without completing it? I think you should be able to send comments regardless, which means you could talk to the author about the problems with their story, but the fact that no one would be able to rate the arc should speak for itself, don't you think?

As it stands now, those arcs, all with serious flaws, could have lots of 5-star ratings even though no one actually finished them. Or possibly even played them.

[ QUOTE ]
Fairness in fixing what's already been done to misuse the system is too much to ask for. Item rewards have already been received, badges have already been issued; the impact any such action will have on those that misused the system is negligible. The ones it would truly harm are those that legitimately rose above the rest, despite the obstacles. They're the ones that would be feeling the sting of such action, that would be most likely to leave the game as a result of such action. And they're the ones we want to keep around.

[/ QUOTE ]

If they were so good that they could beat out the 0-star crew, then there's no reason why they wouldn't rise to the top again even more quickly without the interference from rating system abuse. The arc isn't changing, the rating system would be.

[ QUOTE ]
I think the best we can hope for is to leave things as is for now, but design a system that will repair the damage done to the system naturally, over time.

[/ QUOTE ]

The system as it stands now is, in my opinion, so flawed as to defeat half the point of using it. Right now, all people can do once they've been visited by the ratings demons is to go on a mass advertising spree in order to get people to find their arc at the bottom of page 293 or so. People can get their friends or supergroups to play their arcs just fine, but what if you want more than that? The main thing everyone uses to figure out what arcs are worth looking at is the rating system and few people that I have ever spoken with have the patience to dig down past page 10 or so.

I don't expect the system will change overnight. But I do believe the system needs to be seriously overhauled if they want it to actually do what it was meant to do.

[ QUOTE ]
Perhaps something to keep track of what percentage of people quit the arc without completing it? I think you should be able to send comments regardless, which means you could talk to the author about the problems with their story, ...

[/ QUOTE ]
Unfortunately, not onlt does that require players to go out of their way to do something extra, it's making them do so when they're upset that they couldn't complete an arc. In such a situation, I probably wouldn't leave any feedback. And I love making my thoughts known. :P

[ QUOTE ]
... but the fact that no one would be able to rate the arc should speak for itself, don't you think?

[/ QUOTE ]
There would be no way of determining if no one's been able to complete the mission, if no one's tried running the mission, or if no one has simply cared to rate the arc.

[ QUOTE ]
As it stands now, those arcs, all with serious flaws, could have lots of 5-star ratings even though no one actually finished them. Or possibly even played them.

[/ QUOTE ]
The system definitely needs an overhaul, we both agree on that. But limiting the options for providing feedback will only hurt things in the long run

[ QUOTE ]
[ QUOTE ]
Fairness in fixing what's already been done to misuse the system is too much to ask for. Item rewards have already been received, badges have already been issued; the impact any such action will have on those that misused the system is negligible. The ones it would truly harm are those that legitimately rose above the rest, despite the obstacles. They're the ones that would be feeling the sting of such action, that would be most likely to leave the game as a result of such action. And they're the ones we want to keep around.

[/ QUOTE ]

If they were so good that they could beat out the 0-star crew, then there's no reason why they wouldn't rise to the top again even more quickly without the interference from rating system abuse. The arc isn't changing, the rating system would be.

[/ QUOTE ]
They would be able to regain their standing, and they wouldn't. Most that have already rated their work are unlikely to play through it again, and as such these authors have a smaller prospective audience from which to build their story's standing. Add to that the fact that these authors had to put real effort to overcome the broken system, not just writing the arc but promoting, bug fixing, and generally getting the word out. In essence, all their hard work would be thrown out, and when a player's effort and work is tossed out the window by developers, online titles begin to die. Star Wars Galaxies is the best example of this.

[ QUOTE ]
2: Require the completion of an arc before it can be rated -

[/ QUOTE ]
Problem. I zone in. Four AVs kill me right after I come around the first corner. I should be able to 1-star this thread and quit. Under your proposed system I can no longer do that.

I would be in favor of a system that disallows EBs and AVs in the first mission, and then requires you to finish the first mission before you can rate the story-arc.

[ QUOTE ]
3: Require comments for ratings under 3 stars -

[/ QUOTE ]
Once again: 349hgfdubjhvbnelriht4ioh!

However, changing the interface so that it prompts people to send a comment if they rate an arc at three stars or less without requiring it would be a good thing. Especially if the comments are fixed to be annonymous again and the prompt reminds people of that.

[ QUOTE ]
5: Append Comments To Arcs

[/ QUOTE ]
All published arcs should have a tab that lists all feedback on that arc sorted by date/time. Only the author can see this feedback unless he choses to make it public. This way comments are stored, annonymity is preserved, and we still don't have to use that broken email system - which should either be fixed or just shut off already.

[ QUOTE ]
6: Unpublish all non-Developer's Choice arcs

[/ QUOTE ]
Everytime a patch affects the MA an internet-email should be automatically sent to the person who authored the arc. The email would inform them that a recent patch has affected story-arcs. They now have 7 days to review, fix, and republish their story-arc or it will automatically lose "Devs Choice" status.

[ QUOTE ]
7: Include a system to contest ratings

[/ QUOTE ]
Completely disagree. You're not being griefed by ratings griefers. Seriously. Just accept the fact that you're not Stephen King already and move on.

[ QUOTE ]
Unfortunately, not onlt does that require players to go out of their way to do something extra, it's making them do so when they're upset that they couldn't complete an arc. In such a situation, I probably wouldn't leave any feedback. And I love making my thoughts known. :P

[/ QUOTE ]

This seems to be a common theme. Are people so afraid of actually talking to someone else and bringing up a problem that they aren't willing to type in a few words? What is the point of feedback if no one is willing to actually step up and spend the 30 seconds it takes to tell someone what they really think and give them an honest opinion?

If people care so little that they won't even deign to acknowledge the people who put time and effort into making these arcs, then we as a community have already lost.

I, for one, do not accept this.

[ QUOTE ]
There would be no way of determining if no one's been able to complete the mission, if no one's tried running the mission, or if no one has simply cared to rate the arc.

[/ QUOTE ]

Track how many people have started the arc as well, then? Really, the devs should be tracking this data anyway. Make it visible.

[ QUOTE ]
The system definitely needs an overhaul, we both agree on that. But limiting the options for providing feedback will only hurt things in the long run

[/ QUOTE ]

You ALWAYS have the option of telling them directly. After all, it lists the global handle of the arc's author in the information! If you want to let someone know a problem with their arc, tell them!

[ QUOTE ]
They would be able to regain their standing, and they wouldn't. Most that have already rated their work are unlikely to play through it again, and as such these authors have a smaller prospective audience from which to build their story's standing. Add to that the fact that these authors had to put real effort to overcome the broken system, not just writing the arc but promoting, bug fixing, and generally getting the word out. In essence, all their hard work would be thrown out, and when a player's effort and work is tossed out the window by developers, online titles begin to die. Star Wars Galaxies is the best example of this.

[/ QUOTE ]

Everyone else would be in the same boat. And really, why WOULDN'T you play a good arc again? Especially with new friends or teammates who haven't seen it? I've run through some arcs many times over. Being able to go back and do stuff again was the theme of an entire Issue.

No, their work is not being thrown out. Rating something is not work. Especially if the rating system is this flawed.

And as a former player of SWG, that title didn't die because player's work was thrown out. It died because SOE has no concept on how to properly balance, insisted on keeping broken systems in the game, and overall has no concept on how to make a good Star Wars game.

Bioware, now their product is something I'm looking forward to...

[ QUOTE ]
Problem. I zone in. Four AVs kill me right after I come around the first corner. I should be able to 1-star this thread and quit. Under your proposed system I can no longer do that.

I would be in favor of a system that disallows EBs and AVs in the first mission, and then requires you to finish the first mission before you can rate the story-arc.

[/ QUOTE ]

As I stated before, the fact that no one can finish the arc would speak for itself, especially if they make 'times started' and 'times completed' visible statistics.

[ QUOTE ]
Once again: 349hgfdubjhvbnelriht4ioh!

However, changing the interface so that it prompts people to send a comment if they rate an arc at three stars or less without requiring it would be a good thing. Especially if the comments are fixed to be annonymous again and the prompt reminds people of that.

[/ QUOTE ]

Once again: If someone can't bother to speak English when they give a comment, it's obviously not a legitimate rating.

I am also against comments being anonymous. Take responsibility for your actions. Anonymity only further allows people to abuse the system and prevents me from having dialogues with people who have actual constructive criticism to make.

[ QUOTE ]
All published arcs should have a tab that lists all feedback on that arc sorted by date/time. Only the author can see this feedback unless he choses to make it public. This way comments are stored, annonymity is preserved, and we still don't have to use that broken email system - which should either be fixed or just shut off already.

[/ QUOTE ]

I like this!

[ QUOTE ]
Everytime a patch affects the MA an internet-email should be automatically sent to the person who authored the arc. The email would inform them that a recent patch has affected story-arcs. They now have 7 days to review, fix, and republish their story-arc or it will automatically lose "Devs Choice" status.

[/ QUOTE ]

Interesting proposal.

[ QUOTE ]
Completely disagree. You're not being griefed by ratings griefers. Seriously. Just accept the fact that you're not Stephen King already and move on.

[/ QUOTE ]

Actually, I don't like Stephen King.

And while I have been affected by this, I am not nearly the only one. If it was just me, I would just 'get over it', as you so succinctly put it.

Besides, how would you know?

[ QUOTE ]
[ QUOTE ]
Unfortunately, not onlt does that require players to go out of their way to do something extra, it's making them do so when they're upset that they couldn't complete an arc. In such a situation, I probably wouldn't leave any feedback. And I love making my thoughts known. :P

[/ QUOTE ]

This seems to be a common theme. Are people so afraid of actually talking to someone else and bringing up a problem that they aren't willing to type in a few words? What is the point of feedback if no one is willing to actually step up and spend the 30 seconds it takes to tell someone what they really think and give them an honest opinion?

If people care so little that they won't even deign to acknowledge the people who put time and effort into making these arcs, then we as a community have already lost.

I, for one, do not accept this.

[/ QUOTE ]
It's not a matter of not wanting to provide feedback. It's a matter of being so fed up with an arc and what it put them through that one doesn't want to provide comment at all. One just wants to be done with it and move on.

[ QUOTE ]
[ QUOTE ]
The system definitely needs an overhaul, we both agree on that. But limiting the options for providing feedback will only hurt things in the long run

[/ QUOTE ]

You ALWAYS have the option of telling them directly. After all, it lists the global handle of the arc's author in the information! If you want to let someone know a problem with their arc, tell them!

[/ QUOTE ]

Unfortunately, some folks don't want to take the risk of negative backlash due to even constructive criticism of an author's work. There's a somewhat valid fear of payback, and if the only way of providing feedback is to do so directly, many simply won't do so.

[ QUOTE ]
Everyone else would be in the same boat. And really, why WOULDN'T you play a good arc again? Especially with new friends or teammates who haven't seen it? I've run through some arcs many times over. Being able to go back and do stuff again was the theme of an entire Issue.

[/ QUOTE ]
I've run fun arcs again a second time, especially in a group. But I've never rated it a second time. Nor do I rate an arc if I run through with a group on the first go.

[ QUOTE ]
No, their work is not being thrown out. Rating something is not work. Especially if the rating system is this flawed.

[/ QUOTE ]
Rating something isn't work (unless you go through it with fine-toothed comb). Promoting an arc to get people to play and rate it, however, is. And that's what would be destroyed. Good authors have been promoting their work, sometimes going so far as to put cash into such promotions, to get to where they are. Removing their stars would be negating the effort and dollars put into their work, and several talented contributors would likely be lost. And I wouldn't blame them; they would be getting punished for other exploitation of the system.

[ QUOTE ]
And as a former player of SWG, that title didn't die because player's work was thrown out. It died because SOE has no concept on how to properly balance, insisted on keeping broken systems in the game, and overall has no concept on how to make a good Star Wars game.

[/ QUOTE ]
You make it sound as if you don't think I played the game. I did, and I was there as well. And I can honestly tell you everyone I knew that left the game claimed they did so because the work and effort they had put into their characters and the game were erased in one fell swoop by the developers. Claims that SOE didn't know how to make a good Star Wars game fall short of the mark, as the game was doing extremely well before they changed things. Claims that SOE has no concept on how to properly balance things fall short as well, as the company still has hit titles.

As an aside, I too have high hopes for The Old Republic. Here's to hoping Bioware doesn't botch things.

The ratings system is totally meaningless. They've tied the ratings that you get to badges, which means the ratings no longer signify the opinion of the arc. I would like to see the ratings disconnected from the badges entirely. Sure, add categories (perhaps optional, so those that don't want to take the time don't have to) to fine tune the ratings, so that we know what they actually mean; but the single most important step to take here is to take away the incentive and possibility of griefing and the padding of ratings, which is what is making the ratings meaningless.

To satisfy the badge hunters, I propose that badges instead be given for finding arcs of quality and recommending them for Dev's Choice. Not all recommended arcs could be Dev's Choice, of course, but if the dev reviewer agrees that it's a 4 or 5 star arc, the person recommending it would get stars, or points, towards a badge. The badge qualifications would have to be reworked, of course, for the ones you get for getting good ratings, but in the interest of fixing the entire system, I really think they should be. And, for the record, I AM a badge collector, and I've collected several of these badges, and would happily give them up if the rating system could work better.

The main benefit of changing the badges earned from getting stars for your arcs to earning stars for finding other people's arcs of quality, is the type of activity each of those badges encourages. We've already seen that tying badges to the stars you personally earn leads to a competitive atmosphere, where arcs that earn high ratings are griefed, and arcs languish, unplayed and unrated, because there is limited incentive to play someone else's arc when you're trying to get people to play yours. On the other hand, giving people an incentive to sort thru the arcs that are out there means that there is no way or incentive to grief, and more new stories will be discovered and appreciated, and isn't that what the idea of the Mission Architect was to begin with? Sharing stories and new content?

For a more detailed look at my proposal, please see the topic I started; it's just a page past this one, entitled "suggestion for changing the ratings/badges in MA". Of course, this is just the spark of the idea, I'm sure it could be improved upon. I really hope the devs will realize that the ratings system needs to be totally revamped.