PUGing through Incarnate Trials 04/05/11 to 10/20/11
By
Arilou,
Posted on: 2011-10-21 19:10 in CoH and CoV General Discussions
My very rough estimate of what I'd seen of late was 1/3 commons, 1/3 uncommons, 2/9 rares and 1/9 very rares. Looks like what I see about matches what you've seen.
I have to say .... I don't complain too often about CoH "stuff", but ......
I have this one toon that I've been playing on Exalted. He's level 50(+2), and I started doing ITrials about 3 weeks ago or so with him. I usually do an average of about 2-4 ITrials per day, and I still haven't gotten a VR drop. Its one of the few things that is really starting to tick me off ........
Wow, I think I've probably played less than 50 total trials, maybe a handful more, but over 400, that seems like a ton.
Quote:
I've read reports of people doing 400 runs April to June.
|
Originally Posted by Bubbawheat
Wow, I think I've probably played less than 50 total trials, maybe a handful more, but over 400, that seems like a ton.
|
Triumph: White Succubus: 50 Ill/Emp/PF Snow Globe: 50 Ice/FF/Ice Strobe: 50 PB Shi Otomi: 50 Ninja/Ninjistu/GW Stalker My other characters
Quote:
Really glad I can't datamine myself.
|
Originally Posted by Snow Globe
I've read reports of people doing 400 runs April to June.
|
<:[ shark goes nom nom nom ]:>
[QUOTE=theOcho;3409811]As to the REAL reason I'll be leaving, I'm afraid it is indeed because Tamaki Revolution dc'd on me during a RSF.[/QUOTE]
Quote:
|
Originally Posted by Bubbawheat
Wow, I think I've probably played less than 50 total trials, maybe a handful more, but over 400, that seems like a ton.
|
Not really Bubbawheat. the first trials went live, as Snow shows, on April 5th thats more than 28 weeks ago or about 200 days ago as of Saturday October 22nd. If a player only did one BAF and one LAM a day and played every day they'd have been on 400 trials. Now consider that a LOT of us did much much more than just 1 each a day until we got Judgement, Interface, Lore and Destiny open ad slotted. Granted now all my incarnates have tier 3 or tier 4 enhancers in every slot so I am down to doing just enough trials to get empyreans and not even on every character every day.. I have 18 incarnates. I haven't kept track but I'd venture a guess and say i have done well over 600 trials. LOL and on Virtue I know for a fact i am no where near the top of the list in trials completed.. There are players that do all 4 trials on a character, alt and start all over again .. every single day. heck for the past week I think I did maybe 4 trials.. been playing with me Street Justice Brute and my Beam Weapon/Time manip corruptor.
�We�re always the good guys. In D&D, we�re lawful good. In City of Heroes we�re the heroes. In Grand Theft Auto we pay the prostitutes promptly and never hit them with a bat.� � Leonard
�Those women are prostitutes? You said they were raising money for stem cell research!� � Sheldon
Quote:
Yeah, I know you have done more than me. |
Originally Posted by TamakiRevolution
Really glad I can't datamine myself.
|
Triumph: White Succubus: 50 Ill/Emp/PF Snow Globe: 50 Ice/FF/Ice Strobe: 50 PB Shi Otomi: 50 Ninja/Ninjistu/GW Stalker My other characters
On the other hand of course, I'm nowhere near tired of doing trials either.
Quote:
I'm not there yet either. |
Originally Posted by Bubbawheat
On the other hand of course, I'm nowhere near tired of doing trials either.
|
Triumph: White Succubus: 50 Ill/Emp/PF Snow Globe: 50 Ice/FF/Ice Strobe: 50 PB Shi Otomi: 50 Ninja/Ninjistu/GW Stalker My other characters
From that list, I think I've done trials with 3 of your characters.
That is some really impressive numbers. I remember all of those uncommons at the start, ugh.
Nice work on this.
Hmmm, while I think the overall numbers are probably decent, the so-called drop in rares is probably not statistically significant.
My gut feeling was about % chance of a VR and 20% of a rare, and that seems consistent with this data, though.
"Men strunt �r strunt och snus �r snus
om ock i gyllne dosor.
Och rosor i ett sprucket krus
�r st�ndigt alltid rosor."
i used to do about 6 trials a day for 3-4 weeks after they were relased then i slowed up to maybe 1-2 trials a day then down to 1-2 trials per week, right now im almost at 1-3 trials per month
i got sick of the trials because of the crappy reward system, i need VRs on like 3 toons but whenever i do a trial with them i get bloody common or uncommon and crafting a VR outside of trials is extremely expensive, and the amount of emp merits for a single very rare is highway robbery and would be more worthwhile to just save another 20 emp merits for a freaking purple
the trials themselves also get extremely repetitive but since i slowed down on doing them they are bit more enjoyable although the reward system as mentioned above is definitely a reason i dont run them as much
For those nay-sayers who insist all the components have a fixed drop rate, feel free to explain this abundance of "uncommon" being more common than "common"
Quote:
I'd say that a 6%-10% drop is significant.
|
Originally Posted by Arilou
Hmmm, while I think the overall numbers are probably decent, the so-called drop in rares is probably not statistically significant.
|
Triumph: White Succubus: 50 Ill/Emp/PF Snow Globe: 50 Ice/FF/Ice Strobe: 50 PB Shi Otomi: 50 Ninja/Ninjistu/GW Stalker My other characters
Quote:
Not really, you only have about 100 instances in your various data sets. That's a pretty low number and gives a pretty decent chance for simple randomness to play a part. |
Originally Posted by Snow Globe
I'd say that a 6%-10% drop is significant.
|
Quote:
| For those nay-sayers who insist all the components have a fixed drop rate, feel free to explain this abundance of "uncommon" being more common than "common" |
"Men strunt �r strunt och snus �r snus
om ock i gyllne dosor.
Och rosor i ett sprucket krus
�r st�ndigt alltid rosor."
Quote:
Let that sink in for a bit.
|
Originally Posted by Arilou
Uncommon has an equal or larger drop chance than commons?
|
Quote:
That's actually rather easy.|
Originally Posted by GuyPerfect
For those nay-sayers who insist all the components have a fixed drop rate, feel free to explain this abundance of "uncommon" being more common than "common"
|
Quality does not relate to rarity of drop rate.
As for why that occurs, it is likely that the plan/intention is for incarnate threads to generate the vast majority of common salvage.
Let's Dance!
Quote:
I'm trying to wrap my head around the formula myself, but based on what other people have calculated, you have around 50% chance of that result simply being a case of random chance. |
Originally Posted by Snow Globe
I'd say that a 6%-10% drop is significant.
|
To be able to have a 95% certainity of the drop (which is standard statistical certainity, used for most kinds of polls) you'd have to run a bit over a 1100 trials, and get the same result. (eg. you had to run 1100 trials in your third period and 1100 trials in your second and 1100 trials in your fourth period, to have a useful basis of comparison)
50% certainity is just too low to make any kind of conclusions. It means your results are as likely to be the results of random chance as from a change in the drop rate.
"Men strunt �r strunt och snus �r snus
om ock i gyllne dosor.
Och rosor i ett sprucket krus
�r st�ndigt alltid rosor."
Quote:
It turns out that if you're trying to tell if the underlying probabilities have changed, you don't necessarily need that much data at all - it depends on how different the results are.|
Originally Posted by Arilou
I'm trying to wrap my head around the formula myself, but based on what other people have calculated, you have around 50% chance of that result simply being a case of random chance.
To be able to have a 95% certainiy of the drop (which is standard statistical certainity, used for most kinds of polls) you'd have to run a bit over a 1100 trials, and get the same result. (eg. you had to run 1100 trials in your third period and 1100 trials in your second and 1100 trials in your fourth period, to have a useful basis of comparison) |
Compare the rare drop rate between the June and September builds:
20110628: 16 rare drops, 103 other component dropsJust eyeballing the figures, it looks like the chance for a rare drop has increased between the last two builds.
20110913: 13 rare drops, 44 other component drops
Starting with the assumption that we don't know anything about the relative drop rates, we can model our confidence in the probabilities of a rare drop by a Beta distribution: for the two builds, P(rare drop in 20110628 build) ~ Beta(16,103) and P(rare drop in 20110913 build) ~ Beta(13,44). The most likely drop rate for 20110628 is 16/(103+16) = 13.4%, and for 20110913 is 13/(44+13) = 22.8%, but given the limited data, how likely is it that the rare drop rate actually increased between the two builds?
The confidence that X > Y when X ~ Beta(a,b) and Y ~ Beta(a',b') is given by a fun double integral of f(x+u;a,b)f(x;a',b').dxdu, with x and u ranging from 0 to 1, and where f(x;a,b) is the pdf of Beta(a,b). Plugging the numbers in and computing gives:
P(rare drop in 20110913 more likely than rare drop in 20110628) = 94%.Based on the data given, there's roughly a 19 in 20 chance that the underlying drop rate for rares has actually changed (increased) between the two builds. Alternately, you could say there's only a 1 in 20 chance that these observed changes are just a result of random chance.
Looking at the very rare data, the raw numbers indicate that very rare drops have decreased between the two builds, but because there are fewer drops to work with, the results have lower confidence. With:
20110628: 11 very rare drops, 108 other dropsthe calculation gives:
20110913: 4 very rare drops, 57 other drops
P(very rare drop in 20110913 build less likely than very rare drop in 20110628) = 72%.So there is some evidence indicating that the very rare drop rate has decreased, but it's very weak: it easily (about 30% chance) could just be a result of random variation. We'd need more data to be sure.
Quote:
Well, I can tell you that since I did the original post, that I've got:|
Originally Posted by halfflat
So there is some evidence indicating that the very rare drop rate has decreased, but it's very weak: it easily (about 30% chance) could just be a result of random variation. We'd need more data to be sure.
|
0 Fails
0 Threads
1 Common
4 Uncommon
1 Rare
0 Very Rare
I plan on updating these figures on a semi-regular basis.
Triumph: White Succubus: 50 Ill/Emp/PF Snow Globe: 50 Ice/FF/Ice Strobe: 50 PB Shi Otomi: 50 Ninja/Ninjistu/GW Stalker My other characters
Quote:
He's been running four samples though. The chance of one of them being off is indeed 1/20, but the chance of one of four being off is 1/6. (or so)
|
Originally Posted by halfflat
It turns out that if you're trying to tell if the underlying probabilities have changed, you don't necessarily need that much data at all - it depends on how different the results are.
Compare the rare drop rate between the June and September builds: 20110628: 16 rare drops, 103 other component dropsJust eyeballing the figures, it looks like the chance for a rare drop has increased between the last two builds. Starting with the assumption that we don't know anything about the relative drop rates, we can model our confidence in the probabilities of a rare drop by a Beta distribution: for the two builds, P(rare drop in 20110628 build) ~ Beta(16,103) and P(rare drop in 20110913 build) ~ Beta(13,44). The most likely drop rate for 20110628 is 16/(103+16) = 13.4%, and for 20110913 is 13/(44+13) = 22.8%, but given the limited data, how likely is it that the rare drop rate actually increased between the two builds? The confidence that X > Y when X ~ Beta(a,b) and Y ~ Beta(a',b') is given by a fun double integral of f(x+u;a,b)f(x;a',b').dxdu, with x and u ranging from 0 to 1, and where f(x;a,b) is the pdf of Beta(a,b). Plugging the numbers in and computing gives: P(rare drop in 20110913 more likely than rare drop in 20110628) = 94%.Based on the data given, there's roughly a 19 in 20 chance that the underlying drop rate for rares has actually changed (increased) between the two builds. Alternately, you could say there's only a 1 in 20 chance that these observed changes are just a result of random chance. Looking at the very rare data, the raw numbers indicate that very rare drops have decreased between the two builds, but because there are fewer drops to work with, the results have lower confidence. With: 20110628: 11 very rare drops, 108 other dropsthe calculation gives: P(very rare drop in 20110913 build less likely than very rare drop in 20110628) = 72%.So there is some evidence indicating that the very rare drop rate has decreased, but it's very weak: it easily (about 30% chance) could just be a result of random variation. We'd need more data to be sure. |
"Men strunt �r strunt och snus �r snus
om ock i gyllne dosor.
Och rosor i ett sprucket krus
�r st�ndigt alltid rosor."
Quote:
Can you provide me with more sample groups at this time? When Leandro put out his Hundred Trials Later post, he wasn't subjected to the same degree of scrutiny that you are putting out in this thread.|
Originally Posted by Arilou
He's been running four samples though. The chance of one of them being off is indeed 1/20, but the chance of one of four being off is 1/6. (or so)
|
The four sample sets are the major patches that have affected Incarnate Trials. To get more sample sets will take as long as it takes for the developers to introduce major changes. Issue 21.5 (TPN Trial) will be the start of the new sample set. After that, when the MoM trial starts will be yet another sample set. However I don't think I'll be able to run 100 trials between the two, given comments.
Triumph: White Succubus: 50 Ill/Emp/PF Snow Globe: 50 Ice/FF/Ice Strobe: 50 PB Shi Otomi: 50 Ninja/Ninjistu/GW Stalker My other characters
Quote:
No, that's not so.|
Originally Posted by Arilou
He's been running four samples though. The chance of one of them being off is indeed 1/20, but the chance of one of four being off is 1/6. (or so)
|
To use more of the data though, let's consider the hypothesis that the rare drop rate in the 20110628 build was less than the rare drop rate in the builds on either side, which was the same in each (let's ignore build 20110405 on the grounds that we know they changed the drop rates between that and the next build.)
In builds 20110426 and 20110913, in total, there were 37 rare drops and 141 non-rare drops. Comparing with the 16 rare, 103 other drops from 20110628, we get
P(rare drop in 20110913 or 20110426 more likely than rare drop in 20110628) = 95%If anything, the evidence is stronger.
All of this though is under the assumption that drop rates are constant throughout each build, and constant across trials and archetypes. This might not be so.
Quote:
But the chance of one sample in three being off by 2 sigma is much larger than 5%. |
Originally Posted by halfflat
No, that's not so.
To use more of the data though, let's consider the hypothesis that the rare drop rate in the 20110628 build was less than the rare drop rate in the builds on either side, which was the same in each (let's ignore build 20110405 on the grounds that we know they changed the drop rates between that and the next build.) In builds 20110426 and 20110913, in total, there were 37 rare drops and 141 non-rare drops. Comparing with the 16 rare, 103 other drops from 20110628, we get P(rare drop in 20110913 or 20110426 more likely than rare drop in 20110628) = 95%If anything, the evidence is stronger. All of this though is under the assumption that drop rates are constant throughout each build, and constant across trials and archetypes. This might not be so. |
I do agree that the fact that we're assuming that drop rates are completely random (and not based on any factors within the trials, performance, AT etc.) is a major uncertainity factor.
Quote:
| When Leandro put out his Hundred Trials Later post, he wasn't subjected to the same degree of scrutiny that you are putting out in this thread. |
Quote:
|
I see. So what he actually has is: 04/05/11: 17 / 82 = 20.7% rare 04/26/11: 24 / 121 = 19.8% rare 06/28/11: 16 / 119 = 13.4% rare 09/13/11: 13 / 57 = 22.8% rare What he is saying is sounds about right -- the third result is unusually low. By eye, it looks like a 2-sigma (i.e. 1-in-20; sigma is jargon for standard deviation) effect. However, keep in mind that these aren't actually that rare. If you take multiple sets of these samples, chances are you will run into them for a particular sample sooner or later. With 4 samples, the probability that 1 will be off by 2 sigma is about 17% or 1-in-6. |
"Men strunt �r strunt och snus �r snus
om ock i gyllne dosor.
Och rosor i ett sprucket krus
�r st�ndigt alltid rosor."
Introduction
Warning, picture heavy as I really don't want to spend the time converting these to text. I should note that all these trials took place on the Triumph Server. Usually between 4pm and midnight pacific time on week nights, and 6pm and midnight pacific on weekends.
Part 1: Charts
For those that will have their heads swimming at the tables that follow, here is a simple graph breaking down the results.
Here is the overall percentages for the common/uncommon/rare/very rare tables.
Here is the percentages for the common/uncommon/rare/very rare tables for the 04/05/2011 patch (Issue 20). If you thought that you were getting a lot of uncommons during the first few weeks of Issue 20, you were not mistaken.
Here is the percentages for the common/uncommon/rare/very rare tables for the 04/26/2011 patch. This patch was supposed to fix the uncommon percentage by increasing the chances for higher level rewards. Instead it increased commons while decreasing uncommons.
Here is the percentages for the common/uncommon/rare/very rare tables for the 06/28/2011 patch (Issue 20.5). This patch introduced the Keyes trial. This patch significantly reduced rare drops.
Here is the percentages for the common/uncommon/rare/very rare tables for the 09/13/2011 patch (Issue 21 Head Start). This patch introduced the Underground Trial to the game, as well as brought some players back.
Part 2: Spreadsheets
Here is a straight breakdown of fails/threads/common/uncommon/rare/very rare by trial by major patch change. I think the numbers pretty much speak for themselves.
Here are the previous tables, turned into percentages.
This table is for those that are mainly interested in the distribution of trial rewards, not including fails and threads.
Breaking down the numbers further, here are the results of the various Archetypes by patch. Yes, I tend to play controllers more than anything else.
Here is the character breakdown. Yes, White Succubus is my badge character and tends to be played the most, so it isn't suprising that she has the most trial experience.
As you can see, each character has widely different luck with the trials. This should hopefully give players a starting point for a discussion about overall trial rewards, difficulty/popularity of some trials over others, and provide a better picture of what is happening with PUG Incarnate Trials as opposed to having a select group to do the trials with.
Triumph: White Succubus: 50 Ill/Emp/PF Snow Globe: 50 Ice/FF/Ice Strobe: 50 PB Shi Otomi: 50 Ninja/Ninjistu/GW Stalker My other characters