Originally Posted by PoliceWoman
The best and most shining example of doing this is, in my opinion, Escalation. In my original review of Escalation I gave it 4 stars. Femfury took my feedback, along with the feedback of many other forum reviewers, and used it to improve the story arc. Now I can't praise Escalation highly enough. Escalation deservingly became a Dev Choice arc in November. In my opinion, this was due to the tireless efforts of its author to make it a better story arc.
|
Star Rating = System Failure
On the other side of your shining example. If Escalation hadn't gotten a Dev Choice, it would be a 5-star exemplary arc languishing in the 4-star dead zone getting one play every 2-4 weeks.
|
Honestly, the rating system isn't the problem. It's the SORTING system. There needs to be some way to search by number of ratings rather than just the score itself (For example, searching for all arcs that have a 4 star rating and >50 ratings). Likewise there should be an option to specifically search for "New" arcs - arcs which have less than 10 ratings, or perhaps arcs published within the last week, or maybe an option to search for both or either of those (The more options the better, really).
I think the arc display window itself could also stand to be redesigned - at the moment the 1st and 2nd pages are clogged up with DC, HoF, and GA arcs, and there's no way to change the number of arcs displayed per page. I think the interface would benefit a lot from an overhaul, maybe showing "Honoured" arcs on a separate tab (With some sort of indicator on the tab title like a (!) if a new one has been added recently), and maybe some other sorting options.
The best and most shining example of doing this is, in my opinion, Escalation. In my original review of Escalation I gave it 4 stars. Femfury took my feedback, along with the feedback of many other forum reviewers, and used it to improve the story arc. Now I can't praise Escalation highly enough. Escalation deservingly became a Dev Choice arc in November. In my opinion, this was due to the tireless efforts of its author to make it a better story arc.
It's anecdotal, I'll admit. But I think it's an example of how I think the feedback loop should work. |
Actually, there's an idea. Maybe instead of ratings being based on a straight average, ratings could be weighted based on age, so more recent ratings would contribute more to the average? I still think it would also be good if we could click on an arc's rating and get a breakdown of exactly what scores it's gotten.
Captain Skylark Shadowfancy and the Tomorrownauts of Today. Arc ID: 337333 - Signal:Noise, where is everybody? Arc ID: 341194
@The Cheshire Cat - Isn't it enough to know I ruined a pony making a gift for you?
12 second horror stories - a writing experiment.
I am all for new ways to sort through the thousands of arcs to find one I think would be fun.
On the other side of your shining example. If Escalation hadn't gotten a Dev Choice, it would be a 5-star exemplary arc languishing in the 4-star dead zone getting one play every 2-4 weeks.
|
I agree with Cheshire Cat that a better sorting algorithm for story arcs, maybe some improvements to the MMI for searching & sorting through AE arcs, would go a loooong way to making it easier for players to find arcs and good arcs to get players to try them. In the meantime, doing little promotional things like forum contests, QPQ reviews, advertising on MA Arc Finder channel, playing on the MA Arc Superteam, and other community activities, all help for getting people to try your story arc.
I don't think being in the "4 star ghetto" is the kiss of death by any means, though. One of my arcs that was sitting at 4 stars had a 3-player team play through it today. They were no one I knew, no one who frequented any chat channel I'm on (that I know of). I have no idea how they found my arc, but they found it, liked it and left feedback. It made my day.
@PW - Police Woman (50 AR/dev blaster on Liberty)
TALOS - PW war journal - alternate contact tree using MA story arcs
=VICE= "Give me Liberty, or give me debt!"
I think for a lot of people who request reviews this is how they work - I know that the Astoria in D Minor that got Dev's Choice is a VERY different story arc than it was when I first published it. Likewise even though you rated my Tomorrownauts arc at 5 stars, I still found your feedback helpful and made a bunch of changes based on it. Mostly little changes, but I think it's the little things that can make the difference between 4 and 5 stars. I guess one problem with the rating system is that even if an arc is refined a great deal based on feedback from a 3-star rating, the person who made the original rating rarely ever goes back and re-rates the arc.
|
Captain Skylark Shadowfancy and the Tomorrownauts of Today. Arc ID: 337333 - Signal:Noise, where is everybody? Arc ID: 341194
@The Cheshire Cat - Isn't it enough to know I ruined a pony making a gift for you?
12 second horror stories - a writing experiment.
Well, actually re-rating automatically overwrites your old rating (You can only ever have 1-rating per arc, and you can never lower a rating you've given, as per Posi way back when). So that's not really an issue - but it's still true that people rarely ever re-rate.
|
Anyhow, I think that it would be better to give the author some leeway if the errors are not anything serious. Knocking off an entire star over some typos that they would have fixed anyway after receiving a review that they requested from you would be a rather dick move.
I agree with Cheshire Cat that a better sorting algorithm for story arcs, maybe some improvements to the MMI for searching & sorting through AE arcs, would go a loooong way to making it easier for players to find arcs and good arcs to get players to try them. In the meantime, doing little promotional things like forum contests, QPQ reviews, advertising on MA Arc Finder channel, playing on the MA Arc Superteam, and other community activities, all help for getting people to try your story arc.
|
A lot more would be done by fixing the problem at the Search Interface end, since that is what the majority of players will be finding arcs through. There's even a contact at level 5 to tell you about it's existence.
I've found routinely that the actual star system is pretty much useless for the promotion of arcs. If you want people to play your arc, talk about it in a community like, say, this forum.
Get the DC and HoF arcs off the first two pages for starters. Move them to their own tab or something, just stop monopolizing the front page with the same dozen arcs for the last several months. All it does is make MA look stagnant to the more casual players.
If anything, the first page should be a random assortment chosen from the arcs in the system. My idea would be something like: 1 random Guest Author, 1 random HoF, 1 random DC, 4 random 5-star arcs, and 4 random 4-star arcs. |
This would keep everything on the front page fresh and give decent exposure to 4 star arcs, as well.
Laz's other suggestions are top-notch, too (esp weighting by the number of rates), but that would just be icing on the cake to this.
Craft your inventions in AE!!
Play "Crafter's Cafe" - Arc #487283. A 1 mission, NON-COMBAT AE arc with workable invention tables!
Actually, I think this is incorrect if your second rating is is done about a week or so afterwards. I had uprated someone's arc about a month after I first played it since they made some changes and I noticed that the number of ratings increased. It bugged me since I was under the impression that my new rating would be overwriting the old one.
|
Originally Posted by Bubbawheat
On the other side of your shining example. If Escalation hadn't gotten a Dev Choice, it would be a 5-star exemplary arc languishing in the 4-star dead zone getting one play every 2-4 weeks.
|
Originally Posted by PoliceWoman
I'm afraid I disagree. Even before being Dev Choice'd, that arc was a finalist in Projectionist's Contest, in the Annual Mission Architect Awards, and in the Player's Choice Awards. It got a decent amount of recognition even while sitting at 4 stars.
|
Actually, getting nominated for an award is probably your best bet. Very few arcs get nominated, there's far more good arcs (and perhaps better arcs) in the system than there are nominations. It's also easy enough to get nominated. Bubbawheat has a competition going already. If people really want plays, they should be encouraging others to nominate their arcs. No, it's not a perfect meritocracy. But it's the best we got. Sure, people might not want to nominate you. So listen to them, help them like your arc. Of course, you'll also have a better, and more worthy arc
It used to be easy to keep an arc in the 5-star range. But people got pickier, and fewer, and griefing does happen. Now, voters are learning how the system works, so votes tend to be more positively assigned, and things should be improving. But with fewer players per arc, it takes far less effort to grief someone. That last point ought to be relevant to the developers.
I suggested a median calculation in a thread on this topic by MCM, four months ago. I think there were other suggestions also in that thread.
Venture has very carefully produced arcs. They're not everyone's cup of tea, but I liked them a lot when I played them. They're doomed to 3- and 4-stars due to, uh, 'compassion shortage'. I do not believe those ratings have anything to do with the contents of the arcs.
Cheers, airhead
Arc: 379017: Outbroken See all your old friends in the Outbreak Tutorial sequel!
Arc: Coming Soon: The Incarnate Shadow Shard of Fire and Ice Mender Rednem needs you!
Massively.com opinion poll: Please Help Save CoH!
I agree with Wrong Number about most things concerning the stars and search engine. I made a post about it in the other MA forum (Search Engine to the Stars) which contains most of what I've got to say on the subject.
Winner of Players' Choice Best Villainous Arc 2010: Fear and Loathing on Striga; ID #350522
WARNING: LONG, RAMBLING, FULL OF OPINION!
I myself have received arc reviews that rated some of my story arcs 4 stars, 3 stars, even 2 stars. What you take away from such a review is, of course, up to each individual person. I think all feedback is valuable, though; especially negative feedback. |
But PW I ask you what purpose is served by you giving a lower than 5 star rating to the person in game before they tell you that their arc is final? The fact that a person submitted an arc to you means that they care and want to make it better. Could you not perhaps withhold your rating if it is lower than 5 stars to give them a chance to do so? What harm would that do?
@Gypsy Rose
In Pursuit of Liberty - 344916
The Vigilante - 395861
Suppression - 374481 - Winner of The American Legion's February 2011 AE Author Contest
Going forward, outstanding arcs are likely to get dozens of plays, not hundreds.
|
I know I've mentioned it before, but I think the best "variant" on the current system is a pretty simple one: a technique called "sigma clipping". The idea is you calculate both the average rating, and the standard deviation of that average (which is often represented with the greek letter sigma). Then, any subsequent ratings that fall outside of a preset range (often two times sigma or three times sigma) are ignored as "erroneous outliers" when calculating the average.
The system would have to have some artificial boundaries set up to account for the time when the arc has a low number of plays and ratings (for example, artificially set it so that "two times sigma" has a minimum value equal to 5 divided by the number of plays the arc has gotten so far), but this system would eliminate a griefer's ability to disproportionately trash an arc's rating (since a 1 or 2 star rating for an arc that has 33 plays where 21 are 5 star ratings and 12 are 4 star ratings would simply be completely ignored). |
If anything, the first page should be a random assortment chosen from the arcs in the system. My idea would be something like: 1 random Guest Author, 1 random HoF, 1 random DC, 4 random 5-star arcs, and 4 random 4-star arcs.
|
On the other side of your shining example. If Escalation hadn't gotten a Dev Choice, it would be a 5-star exemplary arc languishing in the 4-star dead zone getting one play every 2-4 weeks.
|
If my new arc Poodle Skirt Girl against the Invaders from Mars deserves 2 stars because of its many flaws, it's not doing me a big favor to have one or two people 5-star it despite its 2-star-worthiness.
|
I'm afraid I disagree. Even before being Dev Choice'd, that arc was a finalist in Projectionist's Contest, in the Annual Mission Architect Awards, and in the Player's Choice Awards. It got a decent amount of recognition even while sitting at 4 stars.
|
Actually, I think this is incorrect if your second rating is is done about a week or so afterwards. I had uprated someone's arc about a month after I first played it since they made some changes and I noticed that the number of ratings increased. It bugged me since I was under the impression that my new rating would be overwriting the old one.
|
WN
Check out one of my most recent arcs:
457506 - A Very Special Episode - An abandoned TV, a missing kid's TV show host and more
416951 - The Ms. Manners Task Force - More wacky villains, Wannabes. things in poor taste
or one of my other arcs including two 2010 Player's Choice Winners and an2009 Official AE Awards Nominee for Best Original Story
I really would like to be able to see stats on arcs. If something is 4-star average then I'd like to see how many of those actually were 4-star votes. If you can't give this to everyone then at least allow the author to see them for their own arcs.
|
Most especially, I want the devs to see it, and do something about it. Like take away the option to rate an arc without finishing at least some of it.
I myself have received arc reviews that rated some of my story arcs 4 stars, 3 stars, even 2 stars. What you take away from such a review is, of course, up to each individual person. I think all feedback is valuable, though; especially negative feedback.
For example, Venture recently gave one of my arcs 3 stars. In my opinion, it would be useless for me to rail at him and say stuff like, "You're so mean, why didn't you give me 5 stars? No one will play my arc now!" Aside from being nonconstructive, I'm sure he's built up an immunity to this kind of flame by now. Instead, what I prefer to take away from this experience is, "What can I change in my arc to make the next person give me more than 3 stars?" |
A review is helpful because it gives you feedback. It helps you improve. A 3-star rating tells you nothing. It could be due to a major plot hole, or it could be something completely subjective, or downright stupid.
Well, actually re-rating automatically overwrites your old rating (You can only ever have 1-rating per arc, and you can never lower a rating you've given, as per Posi way back when). So that's not really an issue - but it's still true that people rarely ever re-rate.
|
So marketing is everything, once you're in the 4-star ghetto. It might help to have a lot of friends and a large supergroup to stay 5-star, but for others it's a better bet to be visible on this forum, and pay heed to feedback (even lip-service) to get your 4-starred arc played. I've tried doing this, I've tried not doing this. There's a strong correlation. You might have strong writing, but far more importantly, you need to get played. Forget about getting back to 5-stars though, this forum is not that friendly.
|
Venture has very carefully produced arcs. They're not everyone's cup of tea, but I liked them a lot when I played them. They're doomed to 3- and 4-stars due to, uh, 'compassion shortage'. I do not believe those ratings have anything to do with the contents of the arcs. |
Eva Destruction AR/Fire/Munitions Blaster
Darkfire Avenger DM/SD/Body Scrapper
Arc ID#161629 Freaks, Geeks, and Men in Black
Arc ID#431270 Until the End of the World
I've said before that the star-based ratings system was an epic failure in it's current incarnation. I still stand by that and many of the improvement suggestions made in this thread, other threads pre-launch (that were made in beta) and the DOZENS (literally) of debate threads made in the days when AE was the "it" thing to do, farming or otherwise.
Also keep in mind, as Venture alluded to, it really WON'T make a difference what changes are done. The days of getting 100s of plays quickly are over. Folks need to lower their expectations. Advertising will always trump a perfect system. The devs could make the system 100% air tight against griefing, and some arcs will simply get more plays than others for whatever reason.
NO ONE can make a story that appeals to EVERYONE. HOWEVER, it's more likely that you WILL make a story that SOMEONE doesn't like, at all.
I'd be fine with having to play whatever percentage of an arc to leave a rating, HOWEVER keep in mind that the danger with this is the person doesn't finish the arc if it's a long one, and thus someone who might have given you a 5 star rating simply doesn't get to rate.
I suspect when on Going Rogue comes/with GR coming closer and closer, the AE won't be the first thing on folks minds. Thus ADVERTISING in the various ways stated in this thread will become more and more important in the days/weeks to come. I for one know that once GR is test, I'll basically will be sleeping on the test server. I probably won't touch AE for months come GR.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
I haven't read the entire thread, but here are my two inf..
There was another site that has serious issues with a rating system (1-10). It was abused, and inaccurate, and generally everything was either 9-10, or it was 1-2. This is pretty much the opposite of the intent, but that's how these things ALWAYS turn out.
A 5 star system or a 1-10 system work when you have a small number of preselected judges casting votes. When you open this up to multitudes of people, it immediately breaks.
What that site did was completely got rid of the 1-10 system in favor of "Endorsements".
As far as I know, all previous 9-10 ratings were translated into positive endorsements (thumbs up), and everything else was just wiped, when the system was transferred over.
Now, you have to give either a thumbs up, or a thumbs down, and in order to leave a thumbs down, you HAVE to leave a comment and click a check box on why.
I vote AE adopt this system. It makes a lot more sense to me, and it cuts down on griefing because only people who genuinely have an issue with the arc (as opposed to just griefing) will leave a thumbs down, being forced to comment.
I'm only ladylike when compared to my sister.
I haven't read the entire thread, but here are my two inf..
There was another site that has serious issues with a rating system (1-10). It was abused, and inaccurate, and generally everything was either 9-10, or it was 1-2. This is pretty much the opposite of the intent, but that's how these things ALWAYS turn out. A 5 star system or a 1-10 system work when you have a small number of preselected judges casting votes. When you open this up to multitudes of people, it immediately breaks. What that site did was completely got rid of the 1-10 system in favor of "Endorsements". As far as I know, all previous 9-10 ratings were translated into positive endorsements (thumbs up), and everything else was just wiped, when the system was transferred over. Now, you have to give either a thumbs up, or a thumbs down, and in order to leave a thumbs down, you HAVE to leave a comment and click a check box on why. I vote AE adopt this system. It makes a lot more sense to me, and it cuts down on griefing because only people who genuinely have an issue with the arc (as opposed to just griefing) will leave a thumbs down, being forced to comment. |
What's to stop someone from commenting "lol nub" and then just giving a thumbs down? Not saying it's an overly bad idea. Just playing devil's advocate.
I DO agree that the star system needs to go.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
Hmmm, not so sure about this. I've read folks saying they got things like "lol nub" as a comment attached to 1-2 star ratings.
What's to stop someone from commenting "lol nub" and then just giving a thumbs down? Not saying it's an overly bad idea. Just playing devil's advocate. I DO agree that the star system needs to go. |
I'm only ladylike when compared to my sister.
I usually give like 3-4 stars - but a few get 5s
I think there should be more filters in the search - like one that lets you view only 1, 2, 3, 4 and 5 star arcs for example - so you could filter, say, "Heroic" morality, then filter what star level you wanted, then sort them after publishing date, as well as the number of times they'd been played.
@Golden Girl
City of Heroes comics and artwork
"I didn't like your arc. Didn't like the custom critters" And then gave a thumbs down?
EDIT: Again I'm just being a devil's advocate. Your system is better than what they have now.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
I usually give like 3-4 stars - but a few get 5s
I think there should be more filters in the search - like one that lets you view only 1, 2, 3, 4 and 5 star arcs for example - so you could filter, say, "Heroic" morality, then filter what star level you wanted, then sort them after publishing date, as well as the number of times they'd been played. |
Also FYI, I have played few arcs that I thought truly deserved 5 stars.
Blazara Aura LVL 50 Fire/Psi Dom (with 125% recharge)
Flameboxer Aura LVL 50 SS/Fire Brute
Ice 'Em Aura LVL 50 Ice Tank
Darq Widow Fortune LVL 50 Fortunata (200% rech/Night Widow 192.5% rech)--thanks issue 19!
Shard Warrior - 50 MA/Regen/BM Scrapper
Founding Member and Leader : Shadow-Force
Co-Leader: Council of Heroes
"Whatever evils come this way... we will be there to stop them."
NO ONE can make a story that appeals to EVERYONE. HOWEVER, it's more likely that you WILL make a story that SOMEONE doesn't like, at all.
|
I'd be fine with having to play whatever percentage of an arc to leave a rating, HOWEVER keep in mind that the danger with this is the person doesn't finish the arc if it's a long one, and thus someone who might have given you a 5 star rating simply doesn't get to rate. |
Hmmm, not so sure about this. I've read folks saying they got things like "lol nub" as a comment attached to 1-2 star ratings.
What's to stop someone from commenting "lol nub" and then just giving a thumbs down? Not saying it's an overly bad idea. Just playing devil's advocate. |
I think there should be more filters in the search - like one that lets you view only 1, 2, 3, 4 and 5 star arcs for example - so you could filter, say, "Heroic" morality, then filter what star level you wanted, then sort them after publishing date, as well as the number of times they'd been played.
|
That is a legitimate reason for disliking an arc. Not too helpful to the author, as far as feedback goes, but most in-game feedback isn't anyway.
Eva Destruction AR/Fire/Munitions Blaster
Darkfire Avenger DM/SD/Body Scrapper
Arc ID#161629 Freaks, Geeks, and Men in Black
Arc ID#431270 Until the End of the World