Search engine to the stars


Clave_Dark_5

 

Posted

Several things come together to make the MA seem unfair when it comes to star ratings and database searches based on those ratings.

Issues:


  1. The 5-star bias in the search engine combined with the way ratings are averaged means that anything but a 5-star rating is a death sentence for a story arc as far as the search engine is concerned. The names of the ratings don't reflect this. 3 stars is ”good”, 4 stars is ”excellent”. These names seem to indicate that they are perfectly acceptable ratings for a good arc. In reality they are severe penalties which cause arcs to drop from the all-important 5-star rank sooner.
    It is far too easy to down-rate an arc, for griefing purposes or to push down arcs which dare to appear above your own arc in the search result lists. Or even by accident, considering how a 3 or 4 star rating might seem like a good score when in reality it isn't.
    A single 1-star rating turns an arc with seven 5-star ratings into a 4-star arc. Once an arc has dropped to 4 stars it is unlikely to show up among the first few hundred or thousand results in a random search and thus will rarely if ever see random plays again, further cementing its 4-star rating since nobody will play and rate it, up or down.
  2. It is possible to rate arcs without completing them. This is a boon to griefers and a bane to authors who want their arcs to receive fair ratings.
  3. The clutter of abandoned arcs, farms, tests and other debris in the database makes searching for arcs a chore. Finding anything actually worth playing is nearly impossible. Even if you find something that isn't an abandoned farm it is still likely to have been left untouched after the issue 16 exploit fixes drove away many players and creators. This means that even if an arc isn't a farm designed to abuse some kind of exploit it still might not provide any kind of proportional reward. This does not attract normal players who are both looking for a nice story and nice rewards to go along with it. A few people might like stories enough to not care about rewards. Those people are a very tiny minority, I'm sure. By not removing arcs which haven't been updated to take these changes into account the MA database has become filled with mostly useless waste.


These are just some of the problems that I've noticed and discussed with other authors for the past few weeks and months. Below are some of my ideas for how to fix them, but I'm sure there are other solutions.


  1. Make it more difficult to down-rate an arc by making 1 and 2 star ratings count as 3 stars for the purpose of calculating averages. Every five 1-star ratings and every three 2-star ratings will actually count as 1 and 2 stars, in order to make sure that those arcs which really deserve such an abysmal score gets their just rewards.

    Optionally just round fractions UP. That alone would go a long way to make down-rating an arc more difficult.
  2. Make it so you can't rate an arc if you don't actually complete it. I can't understand why we have this option at all.
  3. Add a flag to the database to mark all arcs which haven't been republished after a certain date. In this case we'd pick the issue 16 launch date since that was when the most recent drastic changes to the MA were introduced. Arcs with that flag don't automatically show up on searches in the MA. To remove the flag an author would simply have to open the arc in the editor and republish it. This way we would make sure that only arcs that are actively kept up to date are immediately visible in the MA. By checking a box a player could include arcs with the flag in their searches, if they wanted. This option should be off by default to make sure that casual players don't get their searches cluttered up by junk arcs.


Winner of Players' Choice Best Villainous Arc 2010: Fear and Loathing on Striga; ID #350522

 

Posted

Quote:
Originally Posted by FredrikSvanberg View Post
[*]Make it so you can't rate an arc if you don't actually complete it. I can't understand why we have this option at all.
There was a comment from Positron when MA first went live, to the effect that you shouldn't have to complete a truly bad arc to give it the 1-star rating it deserves. Of course with the way the search engine works, those bad arcs would sit at 0 stars and 0 plays forever if you had to finish an arc to rate it. It works both ways, it means the author's friends would have to actually beat the mob of Extreme/Extreme AVs to 5-star it.

Those are good ideas, although I'm not sure about rounding up. Four stars is four stars, and the search engine should be modified to reflect that a legitimately 4-starred arc is still worth playing.


Eva Destruction AR/Fire/Munitions Blaster
Darkfire Avenger DM/SD/Body Scrapper

Arc ID#161629 Freaks, Geeks, and Men in Black
Arc ID#431270 Until the End of the World

 

Posted

I do agree that the search engine could serve to be a lot better, especially since being 5-starred by one person is more meaningful than being 4 starred by a hundred, meaning that the 5-star category is constantly roiling over and being renewed with new **** that's rated highly by idiots with low standards, while the 4-star category goes underexposed.

I suppose in the spirit of confession, though, my only arc is at 4-stars and every criticism I've heard of it as an arc sounds like it's coming from someone whose opinion I wouldn't trust very heartily. Seeing some of the stuff that is 5-starred makes me wonder why the system exists at all, since it's clearly not helping me find things I like.


 

Posted

Quote:
Originally Posted by Talen_Lee View Post
I suppose in the spirit of confession, though, my only arc is at 4-stars and every criticism I've heard of it as an arc sounds like it's coming from someone whose opinion I wouldn't trust very heartily. Seeing some of the stuff that is 5-starred makes me wonder why the system exists at all, since it's clearly not helping me find things I like.
The problem with any rating system is that if you don't like the same stuff as everyone else it doesn't help you very much.

However, even with that caveat, the current system needs work.


Eva Destruction AR/Fire/Munitions Blaster
Darkfire Avenger DM/SD/Body Scrapper

Arc ID#161629 Freaks, Geeks, and Men in Black
Arc ID#431270 Until the End of the World

 

Posted

Like I said in another thread, when sorting results the interface needs to take the quantity of ratings into account rather than just the average. Is it really fair that an arc with a 4.42 average after 200 ratings is listed dozens of pages behind an arc with a 4.43 average after only 10 ratings? Being able to hold onto a 4-star average after hundreds of plays (since people are very fickle and subjective with their votes) says a lot more about the quality of an arc than one that is already on it's way down into the pit of despair.


 

Posted

I agree with the problem. I'm not sure about the solution. Tweaking the numbers doesn't seem likely to matter. Stopping people from 1 starring an unplayable arc isn't helpful. Marking things as stale might be reasonable, though I'm not sure a flag based on republishing is a great idea.

I think the whole search and rating thing is fundamentally problematic and needs a serious overhaul. I know it would be a ton of work, but I think it would be great to have features like:

* bookmarks, so I can easily find favorite missions
* Ability to share my list of favorites, possibly along with my comments on those favorites.
* "people who liked this also liked..." feature
* the initial list of missions should always be random
* it should be easier to find missions which are truly level appropriate

These ideas and many others have been posted many times, starting in beta before the MA was even released. I sincerely hope that it is something they are working on, but I have no grounds for hope.


Avatar: "Cheeky Jack O Lantern" by dimarie

 

Posted

Quote:
Originally Posted by peterpeter View Post
I agree with the problem. I'm not sure about the solution. Tweaking the numbers doesn't seem likely to matter.
It would make 1-star griefing harder to accomplish. It would require a whole supergroup of griefers to make a real impact, and then they would be easy to track down and ban/punish.

Quote:
Stopping people from 1 starring an unplayable arc isn't helpful.
There is already a method for reporting an arc as unplayable. First, tell the author. Second, report the arc as unplayable with the Report button. I don't see why we shouldn't be allowed to use that for this purpose.

Quote:
Marking things as stale might be reasonable, though I'm not sure a flag based on republishing is a great idea.
What else should it be? How else would you make sure that an author has to actively keep his or her arc updated, if not by checking if the arc is actually republished, or published after a certain date?

Quote:
I think the whole search and rating thing is fundamentally problematic and needs a serious overhaul. I know it would be a ton of work, but I think it would be great to have features like:

* bookmarks, so I can easily find favorite missions
* Ability to share my list of favorites, possibly along with my comments on those favorites.
* "people who liked this also liked..." feature
* the initial list of missions should always be random
* it should be easier to find missions which are truly level appropriate

These ideas and many others have been posted many times, starting in beta before the MA was even released. I sincerely hope that it is something they are working on, but I have no grounds for hope.
Agreed with all your ideas there. Just because they have been suggested before is no reason to not keep pestering the devs about them until something happens. How would they know that we're displeased if we don't keep telling them until they fix it?


Winner of Players' Choice Best Villainous Arc 2010: Fear and Loathing on Striga; ID #350522

 

Posted

Quote:
Originally Posted by peterpeter View Post
I think the whole search and rating thing is fundamentally problematic and needs a serious overhaul. I know it would be a ton of work, but I think it would be great to have features like:
* bookmarks, so I can easily find favorite missions
* Ability to share my list of favorites, possibly along with my comments on those favorites.
* "people who liked this also liked..." feature
* the initial list of missions should always be random
* it should be easier to find missions which are truly level appropriate

These ideas and many others have been posted many times, starting in beta before the MA was even released. I sincerely hope that it is something they are working on, but I have no grounds for hope.
All of these are issues the search engines on the web have solved and the same solutions should work. Most of the code required is available online too. Now, it would need tob e converted and tweaked to unrecognizable form for sure, but at least the method of getting past the problem already has been worked on.

"Trust" as defined by search engines (age, who is linking, etc) alone solves many of the issues.


"Hmm, I guess I'm not as omniscient as I thought" -Gavin Runeblade.
I can be found, outside of paragon city here.
Thank you everyone at Paragon and on Virtue. When the lights go out in November, you'll find me on Razor Bunny.

 

Posted

Having read a couple of these kinds of threads now, I think there are a couple of problems that they don't seem to bring up too much.

Firstly, not a lot of people are playing MA since the ragequitting because of the "nerfs" to the exploits. That left a lot of misguided and mislead people with the idea that there are no rewards to be had (samurai farms not withstanding; maybe they'll end up being a good thing for us after all).

If we had, say, 75% of the CoX population playing an MA arc once a week, I think the better stuff would get a chance to shine due to word of mouth etc., even without the "five stars or it's dead" problem - which is a real problem, especially with such a thin population playing MA arcs.

Secondly, there's just so bloody much to look at to begin with. Not only do five-star arcs show up first, but a bajillion of them do. Why even bother sorting through 4-star arcs when you could play 5-star all day and never run out? This I think is one of the main reasons that 4 stars=dead arc.

Solutions? Figuring out how to give decent rewards while deterring crazy loophole exploitation would be a good start and then making sure that every player knows about it. So much misinformation gets spread around by b'cast know-nothings that it's akin to learning about sex from fellow sixth-graders. I think adding back in a few badges to get player's attentions and announcing the changes with a pop-up when you log in would deal with a lot of that problem.

Changing the rating system might help, although I'm not knowledgeable enough to toss my opinion behind one or another of the suggestions.

As for that last problem... it's matter of scale, and the only thing I can think of that could change it would be a mass of people actually playing MA arcs on a regular basis again.

This reminds of conversations that newspapers must have about how to get people to buy newspapers... :/


 

Posted

Quote:
Originally Posted by peterpeter View Post
* bookmarks, so I can easily find favorite missions
* Ability to share my list of favorites, possibly along with my comments on those favorites.
* "people who liked this also liked..." feature
* the initial list of missions should always be random
* it should be easier to find missions which are truly level appropriate
These would all be very welcome, especially the random one. I wish there was a way to click a “random arc” button and have one pop up for you. This might help people “discover” some hidden gems.

And by “people,” I mean me and people like me. And by “hidden gems,” I mean my arcs and arcs like my arcs.


 

Posted

Quote:
Originally Posted by FredrikSvanberg View Post
But there is a "random" button already. Clearly the search interface isn't as obvious as it has to be. Of course it would be nice if the random arcs that popped up weren't 95% farms or abandoned unplayable arcs or test arcs, as well. I.e. the database should be cleaned out every now and then, somehow.
It doesn't help that the Random button is hidden in a submenu behind the vague "More" button that is tucked over in a corner. The text box for Search shouldn't be hidden by default either.

The Random button has other issues as well, such as that if I click it 20 times often the same arc will show up at least 5 of those times. Sometimes multiple times in a row even.