We have our first Hall of Fame...and the winner is
By
AddamsFamily,
Posted on: 2009-05-04 01:20 in Mission Architect
I realized that the voting system was inherently broken when it was explained to me that a four star rating is essentially griefing the arc.
The way Hall of Fame requirements are set up, if you don't get an overwhelming majority of 5 stars, you can't get in, even without people 0-starring you back down. Since there's tons of people who bizarrely refuse to give anything a "perfect" rating ever, very few arcs will ever reach HoF with or without griefers.
Arc #41077 - The Men of State
Arc #48845 - Operation: Dirty Snowball
I was thinking of starting my own thread on this, but lo and behold there's already one here.
I took a quick look this weekend at IMDB. For those not familiar (if there is any such person), it's a giant database of movie information. Most relevant to this thread, it's got a ranking system much like what we have for story arcs.
Now, the IMDB rating system is 1-10 instead of 0-5. However, it's reasonable to compare them at a ratio of 2:1 - that is, a 10 rating in IMDB is a 5 rating in CoX, etc. We can then look at what the millions of IMDB users have chosen to do with their votes.
The average IMDB rating turns out to be somewhere around 7. That's interesting, because it makes the margin for differentials between masterpieces like The Godfather and utter dreck like Lost in Space surprisingly small. In City of Heroes, this would correspond to a 3.5 or so - and that's also interesting because ratings near 3.5 are sometimes lumped in with the 4-stars, which is also where many arcs near 4.5 end up, and sometimes lumped in with the 3-stars, where many arcs near 2.5 end up.
So one problem is that the granularity is all wrong. It doesn't make any sense whatsoever for arcs near 3.5 to be in the same "grouping" as arcs near 4.5, yet that appears to be what happens.
The next issue is the HoF threshold, which, as I understand it, is 4.5. The IMDB equivalent to that rating would be a 9.0. As it turns out, the IMDB will show you what the top-ranked movies of all time are, so it's easy to find out how many satisfy the CoX Hall of Fame criteria.
Know how many there are? Three.
Yep, three movies in the entire recorded history of moviedom satisfy the IMDB equivalent of our HoF. That right there tells me we're doing something wrong. Star Wars wouldn't make our HoF; Casablanca wouldn't make it; Schindler's List wouldn't make it.
I really think the devs should look at what IMDB has chosen to do with their ratings. It's the closest equivalent I can think of to what the MA ratings system is trying to achieve, and they've got a lot of experience with it by now (the site has been around at least 15 years, with a huge participation volume). Their rating system is weighted by number of votes, for example**, and ratings are shown to one decimal point.
Good lessons to learn there, I think.
** IMDB sez: weighted rating (WR) = (v ÷ (v+m)) × R + (m ÷ (v+m)) × C
where:
R = average for the movie (mean) = (Rating)
v = number of votes for the movie = (votes)
m = minimum votes required to be listed in the Top 250 (currently 1300)
C = the mean vote across the whole report (currently 6.7)
And for a while things were cold,
They were scared down in their holes
The forest that once was green
Was colored black by those killing machines
[ QUOTE ]
Actually, it doesn't even take griefing to knock it out. Odds are, any HoF arc is there due to barely having a 4.5 rating. Depending on how it rounds, someone rating the arc 3-star, just because he legitimately thinks it's not a great arc, will knock it down to 4.4. A 2-star will definitely take it out.
The problem may be griefers, but it's also just as likely the flawed system of averaging the ratings.
[/ QUOTE ]
I don't get nearly the number of plays that Footsteps does, so I get to see the phenomenon in slow-mo, and it really does seem as if there is a genuine problem of "Phantom 1-Starrers". I'll get enough 4 and 5 votes to hit page 2 (and even page 1, briefly) when you sort by rating. Then when I sign on a few hours later or the next day, it'll be down to a 4-star average (probably on page 200+), and no tickets waiting to be collected.
I've talked to other people in the same boat, and they've noticed the same thing.
Either there are some people with impossibly high standards who will only play whatever hits the highest pages, or there are some complete and total dinks out there downvoting arcs in order to get theirs into a higher position.
It's a shame, because there are arcs other than Footsteps that were getting the votes and the ratings to hit HoF. Quest For the Coral Horn was one; I think last night that was approaching 700 plays and was on page 1. It's now at 999+ and in 4-star territory. Maybe 300 votes is a bit much to attribute to griefers, and there were legit downvotes, but it's still kind of frustrating.
You know, a simple solution would be to lock out voting on an arc once it hits hall of fame. Person won't lose their HoF status but they also won't get tickets from people who like it. Seems fair to me.
As someone who loves their tickets and knows the guy who made Footsteps (and knows he also loves his tickets), I think I'd rather keep getting tickets. Buying all my salvage for a while there without lifting a finger effort-wise was great. At least until I fell off the front page and have only been getting a few plays a day, anyway.
So people are using arc ratings as a new form of PvP? That's a little sad....
[ QUOTE ]
So people are using arc ratings as a new form of PvP? That's a little sad....
[/ QUOTE ]
Its not that really.
The problem is, to stay at 5 star, you need 4.5 or above average,
In laymans terms, this means anything but a 5 star rating is BAD
Want comedy and lighthearted action? Between levels 1-14? Try Nuclear in 90 - The Fusionette Task Force!
Arc ID 58363!
[ QUOTE ]
Either there are some people with impossibly high standards who will only play whatever hits the highest pages, or there are some complete and total dinks out there downvoting arcs in order to get theirs into a higher position.
[/ QUOTE ]
While it does soothe an author's ego to believe bad votes are the result of some conspiracy to hurt their arc, The fact is, it has nothing to do with griefers nor with people who have impossibly high standards. It has to do with the fact that people have different tastes and look for different things out of the MA.
Just because a few dozen people think a story arc is great does not mean everyone should, or will.
@Doctor Gemini
Arc #271637 - Welcome to M.A.G.I. - An alternative first story arc for magic origin heroes. At Hero Registration you heard the jokes about Azuria always losing things. When she loses the entire M.A.G.I. vault, you are chosen to find it.
I believe I read somewhere in the i15 stuff that they will be adding selectable rewards for devs choice/hall of fame, so you can choose to continue to get tickets, or choose to get normal drops, and possibly a third choice?
[ QUOTE ]
[ QUOTE ]
It has been driving the folks trying to get the "Evaluator/Reviewer/Critic/Judge/Two Thumbs Up" line of badges crazy because even if you manage to start a HoF arc if it gets downvoted before you finish it it doesn't count for the badges. 
[/ QUOTE ]
Not true, but it does make it impossible to farm for the higher of those badges.
[/ QUOTE ]
I haven't bothered to 'farm' for those yet so I was going on what I've been told about it second-hand.
Frankly I'm waiting for a "fix" to this before I beat my head against that wall.
My main point (as you yourself agreed with) was that the current way the system works is in fact making things difficult for badgers. Even if a started HoF arc DOES count for the badges it seems to be very difficult to find one to run in the first place.
Loth 50 Fire/Rad Controller [1392 Badges] [300 non-AE Souvenirs]
Ryver 50 Ele� Blaster [1392 Badges]
Silandra 50 Peacebringer [1138 Badges] [No Redside Badges]
--{=====> Virtue ♀
[ QUOTE ]
So people are using arc ratings as a new form of PvP? That's a little sad....
[/ QUOTE ]
Sadly I started seeing the term "badge PvP" thrown around sometime back during the I14 beta.
Loth 50 Fire/Rad Controller [1392 Badges] [300 non-AE Souvenirs]
Ryver 50 Ele� Blaster [1392 Badges]
Silandra 50 Peacebringer [1138 Badges] [No Redside Badges]
--{=====> Virtue ♀
[ QUOTE ]
I really think the devs should look at what IMDB has chosen to do with their ratings. It's the closest equivalent I can think of to what the MA ratings system is trying to achieve, and they've got a lot of experience with it by now (the site has been around at least 15 years, with a huge participation volume).
[/ QUOTE ]
Closer to 20 years. It was launched in 1990.
Quote:
|
Originally Posted by eltonio
This is over the top mental slavery.
|
[ QUOTE ]
The fact is, it has nothing to do with griefers nor with people who have impossibly high standards.
[/ QUOTE ]
That's a fact? It looks like an opinion to me, but what do I know?
Meanwhile, maybe you can explain this to me. The second time my arc made it to the front page of the search listings, I got a "someone voted on arcname" from somebody who had already voted on the arc. Since it's a fact that every single rating is legitimate, what happened there? Did somebody who hated the arc before nonetheless play it a second time? Was this vote from the Memento guy?
It must be something like that, because apparantly the fact is that nobody one-stars arcs without even looking at them.
It bounces in and out. It's probably been Hall of Fame more than a dozen times at this point, but the longest period of time was only a day. The shortest period was two minutes.
Glad you had fun with it, by the way.
[u]Issues with the Fourth Slot[u]
To clarify the issues with the extra slot: once you get an arc into the Hall of Fame, you gain an additional publishing slot. Once the arc leaves the Hall of Fame you lose this extra slot.
<ul type="square">[*]This means that, even if you have had Hall of Fame at some point, if you don't have it right now you cannot put up a fourth arc.[*]If you put up a fourth arc while you have the extra slot, but then you leave the Hall of Fame, you cannot edit any of your arcs. This is because you now have four arcs up in three slots. Editing requires a republish, which requires a free slot, which you do not have even if the arc comes down during the republish.[*]This one's a good one. If you have four arcs up, and even if you are currently in the Hall of Fame, you still cannot edit the Hall of Fame arc. This means that, if you have four arcs up, there exist no circumstances under which you can edit the Hall of Fame arc. It can never be edited again. I presume that this is similar to the previous issue: editing requires a republish, which would (momentarilly) pull down the Hall of Fame arc, which would mean you don't have four slots, which would mean you don't have space to republish. The republish button is simply greyed out on the Hall of Fame arc; pushing it generates a message like "we are currently processing your previous publishing request," even if you have not made any such requests.[/list]I think that's everything. Obviously, these issues are eliminated if you simply do not put up a fourth arc, but the question then is: what's the point of it?
Wow.
That's .... a really nasty bug.
At least I hope it's a bug.
If it's WAI, someone has a sick sense of humor.
"City of Heroes. April 27, 2004 - August 31, 2012. Obliterated not with a weapon of mass destruction, not by an all-powerful supervillain... but by a cold-hearted and cowardly corporate suck-up."
[ QUOTE ]
The system involved with HoF ranking definitely needs to be revised in some way. They probably ought to put a timer on when an arc makes it to the HoF so that regardless of voting it stays as a HoF arc for at least 24 hours or some such.
[/ QUOTE ]
Hmm. To build on this idea (which is a good one as-is):
Set up HoF as a perma-flag on an arc, so once achieved, it retains status on querying; but take the existing HoF up/down code and translate that to a "hot arc" status.
Result: Players who achieve the req's for HoF get it frozen, allowing them to publish a new arc without having to worry about gaining or losing arc availability status; but the HoF code as an indicator for "current popularity" gets rebranded to indicate a hot arc currently in play. One gives a perma-award, the other indicates raw popularity.
Thoughts?
@TURGENEV - Freedom Server / IRON / B.A.N.E / HORDE
Turg Fiction: Ghost in the Machine Acts III & IV coming 2012!
Turg Fiction: IX is now LIVE on Architect Entertainment!
[ QUOTE ]
[ QUOTE ]
The fact is, it has nothing to do with griefers nor with people who have impossibly high standards.
[/ QUOTE ]
That's a fact? It looks like an opinion to me, but what do I know?
Meanwhile, maybe you can explain this to me. The second time my arc made it to the front page of the search listings, I got a "someone voted on arcname" from somebody who had already voted on the arc. Since it's a fact that every single rating is legitimate, what happened there? Did somebody who hated the arc before nonetheless play it a second time? Was this vote from the Memento guy?
It must be something like that, because apparantly the fact is that nobody one-stars arcs without even looking at them.
[/ QUOTE ]
Actually, that's an artifact of bad editing. I had originally made some comments about conspiracy theories and that sentence was much longer. I gutted the middle and neglected to take the first part out.
Still, people claiming griefing do have a higher burden of proof, and I'm not seeing anything that even comes close to circumstantial evidence.
Your second paragraph is fairly nonsensical to me, and only helps my point about paranoia. I don't know who Momento guy is, but if he 1-starred your arc the first time then 1-starring it again does not change it's overall rating. You can change a rating you've given, you can't add a second one to an arc.
And I hate to break it to you but there are people, for various reasons, who do replay arcs, even those they hated. I know I often turn off the "don't show arcs you've rated" button because there are some good ones I'd play again with another toon. I've never seen anything on the interface that tells me what I rated an arc.
As many as I've played so far, the odds are good that I would not realize whether it was a good or bad one from the description. I'll remember them when I get into them, and if they've changed for the better I will re-rate them.
I don't know what crystal ball you've found that allows you to know that this person, or anyone, 1-starred an arc they didn't even look at, so I don't even know what you're talking about there.
@Doctor Gemini
Arc #271637 - Welcome to M.A.G.I. - An alternative first story arc for magic origin heroes. At Hero Registration you heard the jokes about Azuria always losing things. When she loses the entire M.A.G.I. vault, you are chosen to find it.
Im against HoF being locked in permanantly.
Getting up to the HoF is one achievement, basically a marketing one, maintaining that status over the logn run is the true test of worthiness for the HoF.
I am however in favour of the HoF flag being locked for a day or a week at a time, to avoid the up/down issues.
@Catwhoorg "Rule of Three - Finale" Arc# 1984
@Mr Falkland Islands"A Nation Goes Rogue" Arc# 2369 "Toasters and Pop Tarts" Arc#116617
I pretty much agree with you. That and they won't need to change HoF at all if they quit averaging the ratings and weigh them instead.
@Doctor Gemini
Arc #271637 - Welcome to M.A.G.I. - An alternative first story arc for magic origin heroes. At Hero Registration you heard the jokes about Azuria always losing things. When she loses the entire M.A.G.I. vault, you are chosen to find it.
What's your highest rated arc? You ever been on the first page with hundreds of votes? Know anyone who has? If not, then you have no idea what you're talking about. When you have x number of plays and y number of tickets, it isn't hard to figure out the average rating. When "someone voted on arcname" and you get no tickets, it isn't hard to figure out what's going on. When groups are openly admitting to 1 or 0 starring arcs just to get them off the front page without ever having played them, it doesn't take a rocket scientist to figure it out.
My guess is that you don't have anything with more than a few dozen votes, so you haven't seen it. But hey, since I didn't write everything down as I saw it happening with my own arc (just did the math in my head, said wtf and went on my way since there's nothing I can do about it) I guess it's just maaaaaaagic.
[ QUOTE ][*]This one's a good one. If you have four arcs up, and even if you are currently in the Hall of Fame, you still cannot edit the Hall of Fame arc. This means that, if you have four arcs up, there exist no circumstances under which you can edit the Hall of Fame arc. It can never be edited again. I presume that this is similar to the previous issue: editing requires a republish, which would (momentarilly) pull down the Hall of Fame arc, which would mean you don't have four slots, which would mean you don't have space to republish. The republish button is simply greyed out on the Hall of Fame arc; pushing it generates a message like "we are currently processing your previous publishing request," even if you have not made any such requests.
[/ QUOTE ]
I thought that part was explicitly WAI? I know DC arcs work that way. The devs were trying to eliminate the loophole of completely rewriting an arc (possibly introducing an exploit) while retaining HoF/DC status and associated ratings.
Given how the MA has turned out to operate in practice, I would say this is a case of wildly mis-directed paranoia, but there you have it. The devs could at least reverse this decision at some point with no ill effects.
And for a while things were cold,
They were scared down in their holes
The forest that once was green
Was colored black by those killing machines
[ QUOTE ]
What's your highest rated arc? You ever been on the first page with hundreds of votes? Know anyone who has? If not, then you have no idea what you're talking about. When you have x number of plays and y number of tickets, it isn't hard to figure out the average rating. When "someone voted on arcname" and you get no tickets, it isn't hard to figure out what's going on. When groups are openly admitting to 1 or 0 starring arcs just to get them off the front page without ever having played them, it doesn't take a rocket scientist to figure it out.
My guess is that you don't have anything with more than a few dozen votes, so you haven't seen it. But hey, since I didn't write everything down as I saw it happening with my own arc (just did the math in my head, said wtf and went on my way since there's nothing I can do about it) I guess it's just maaaaaaagic.
[/ QUOTE ]
And, again, what you're saying is "No one could possibly 1-star my arc because they don't like it, therefore it's griefing."
That does not constitute proof, or even a reasonable assumption.
but whatever floats your boat, guy.
@Doctor Gemini
Arc #271637 - Welcome to M.A.G.I. - An alternative first story arc for magic origin heroes. At Hero Registration you heard the jokes about Azuria always losing things. When she loses the entire M.A.G.I. vault, you are chosen to find it.
[ QUOTE ]
[ QUOTE ]
What's your highest rated arc? You ever been on the first page with hundreds of votes? Know anyone who has? If not, then you have no idea what you're talking about. When you have x number of plays and y number of tickets, it isn't hard to figure out the average rating. When "someone voted on arcname" and you get no tickets, it isn't hard to figure out what's going on. When groups are openly admitting to 1 or 0 starring arcs just to get them off the front page without ever having played them, it doesn't take a rocket scientist to figure it out.
My guess is that you don't have anything with more than a few dozen votes, so you haven't seen it. But hey, since I didn't write everything down as I saw it happening with my own arc (just did the math in my head, said wtf and went on my way since there's nothing I can do about it) I guess it's just maaaaaaagic.
[/ QUOTE ]
And, again, what you're saying is "No one could possibly 1-star my arc because they don't like it, therefore it's griefing."
That does not constitute proof, or even a reasonable assumption.
but whatever floats your boat, guy.
[/ QUOTE ]
You're missing his point. If someone played his arc through, he'd get tickets for the play-through. If that person then voted a 1 or 0-star for his arc then that'd be a difference in opinion.
But when someone doesn't even play the arc, but still votes a 1 or 0-star then that's griefing. And the author can tell when his arc is or is not getting played by the number of tickets he can collect from the MA guy.
Statesman said let there be heroes, and there were heroes.
Lord Recluse said let there be villains, and there were villains.
NCsoft said let there be nothing, and there was nothing.
[ QUOTE ]
[ QUOTE ][*]This one's a good one. If you have four arcs up, and even if you are currently in the Hall of Fame, you still cannot edit the Hall of Fame arc. This means that, if you have four arcs up, there exist no circumstances under which you can edit the Hall of Fame arc. It can never be edited again. I presume that this is similar to the previous issue: editing requires a republish, which would (momentarilly) pull down the Hall of Fame arc, which would mean you don't have four slots, which would mean you don't have space to republish. The republish button is simply greyed out on the Hall of Fame arc; pushing it generates a message like "we are currently processing your previous publishing request," even if you have not made any such requests.
[/ QUOTE ]
I thought that part was explicitly WAI? I know DC arcs work that way. The devs were trying to eliminate the loophole of completely rewriting an arc (possibly introducing an exploit) while retaining HoF/DC status and associated ratings.
Given how the MA has turned out to operate in practice, I would say this is a case of wildly mis-directed paranoia, but there you have it. The devs could at least reverse this decision at some point with no ill effects.
[/ QUOTE ]
I probably didn't make this clear enough because I was focusing specifically on the bugs associated with the fourth slot rather than first clarifying that everything works correctly when you're only using three slots.
So to make it explicit:
It could be edited when I only had three arcs up, and it was Hall of Fame.
It could be edited when I only had three arcs up, and it was no longer Hall of Fame.
It can no longer be edited if I have four arcs up and it is in the Hall of Fame.
It can no longer be edited if I have four arcs up, even if it isn't in the Hall of Fame any more.
So it can never be edited if I have four arcs up, regardless of its current status.
As far as I am aware DC arcs can be edited, but doing so removes DC status. Hall of Fame arcs can be edited if you have three arcs up, but never again if you have four up, even if it isn't even in the Hall of Fame any more. I do not think that is working as intended.
If you double-click 1-star it rates the arc 0-stars.
@Doctor Gemini
Arc #271637 - Welcome to M.A.G.I. - An alternative first story arc for magic origin heroes. At Hero Registration you heard the jokes about Azuria always losing things. When she loses the entire M.A.G.I. vault, you are chosen to find it.