Blaster performance test.

That's interesting, because I would presume most people hospital back more often than not when solo. I cannot easily tell how often missions are taking in your statistics because there seems to be just time entries and not start and stop times, but it might make sense to keep track of how much time is spent returning to a mission when dying. The presumption made on death was that the debt was only part of the problem, and the extra travel time was as much a factor in slowing down blasters as the incurred debt, and also why there was a performance gap below level 10 when debt doesn't exist. I cannot easily tell what the penalty for dying was in the mission it occurred in.

Noted, thanks Arcana. I'll keep track of that going forward.

Sorry you have completely missed it.

The point here is that much of the death penalty is gone.

Heck, below level 10 the only penalty is how much time it takes to run back to your mission from the hospital since there's no debt.

Do you seriously think that the availability of Ninja Run, inspirations at the hospital, travel powers at level 4, and the reductions in death penalties only applied to THB or that they wouldn't skew any and everyone's results ?

Hmm I'm not per se. It's simply not good practice in my eyes to have a person with an obvious bias conduct a test they hope to be meaningful in a scientific fashion. Like I joked, with data it's easy to be misleading without lieing.

I'll give you a few example, I'm not accusing just showing what I mean:

What if your blaster was stocking up insp between each mission and your stalker wasn't.

What if you were using temps on one but not the other. Heck maybe you're just playing differently or measuring differently.

In the end you have something you want the results to prove, which hurts the credibility of whatever results you do produce. And yes, someone eventually might say, 'well thb, i think you're lying because you're obviously biased.'

Anyway, to the point. If this is to prove to you what you already think then more power to you.

But you really can't call it "statistics"

It's a single data point, by a biased player. I'm willing to bet you'll prove to yourself what you already think true, but we'll see.

If you wanted statistical samples you'd need a much larger group, and the people in it need to be controlled to some extent. For example, putting this on the forums limits it to forum readers, which I might infer are better players. So whatever results we'll get are skewed toward how a blaster vs melee plays for the good players vs. average.

TLDR: I guess my point. Saying "This is what I experienced." Is fine I suppose. Claiming it as statistical data to prove your point doesn't really work and really shouldn't in and of itself convince anyone.

I've been using inspirations equally on both as they're needed. No temporary powers on either, no XP boosters on either, veteran powers and ninja run on both. Like I said earlier, I'd welcome it if others wanted to participate in this also playing different sets. Just because I'm expecting to see a certain result doesn't mean I'll necessarily see it. So far the Stalker has technically been better since he hasn't died at all and they've both been taking just about as long to complete the same exact things on the same settings. I'm not stacking the odds either way.

Exactly my point. Post defiance I13 we now have travel powers of all kinds at level 4. Ninja run, hoverboard, coyote run, etc, and that doesn't even include the ability to take regular travel powers at 4 without prerequisite. You also now have the AE, and an endless slew of powerleveling options that weren't available before.

The problem is these changes haven't fixed poor performance they only minimize the penalties for poor performance. If you take it to the extreme case you could reason by plotting the changes to travel time from the hospital to the mission, and blaster earning problems the best way to fix the AT would be to install mission transporter booths in hospitals and further cut down the amount of debt blasters incur for dieing.

As a matter of fact adjusting the amount of debt blasters incur and decreasing their travel time to the mission would be the method, that breaks the fewest things in the AT and requires the least effort to implement. It just happens to be horribly wrong.

I doubt that Slythetic was saying you were lying, I certainly would hope not. However I have doubts for this sort of test, and they may be for the same reason.

An observer can subconciously and with NO ILL INTENT skew the results. In this case you are also the performer which makes it even worse. The fact that you want one side to win means you are likely to give it your all with your preferred build. To make matters worse if you try to account for this you can over compensate and penalizing the side you want to win.

As an example if the phone rings in the middle of a mission you may wait a half ring longer before answering it on your favorite build but not the other. Now that I have mentioned it, if the phone rings you may make sure you give the non favorite the benefit of the doubt. I suggest if it rings answer it on the 4th one so it's fair


Even with these problems I am curious at your results and will be following your test.

I'm pretty sure I didn't say any of that. Now would you like to discuss this completely different thing, or the original topic about why THB's data is at best an experiment, and not statistical data. I'm good either way.


Also: if you go by reward generation and leveling speed the blaster is not doing better: if I'm interpreting THB's data correctly the Blaster is leveling slightly slower (about 2% slower, with fast travel and no debt) from level 3 to level 9.

You are not getting Perma Heat Loss and Benumb at SO levels. Stop doing IO to IO comparisons, which is NOT what the average player will be doing. It's almost like you choose not to read things! I clearly said "Unless IOs are concerned, Dark Miasma is a superior soloing set."

Cold does not compare to Dark at SO levels. Fearsome Stare is far more reliable as an alpha opener no matter what as well. Sleet still gets you opening volley'd. Twilight Grasp is a good buffer to mistakes, which Cold doesn't have. Tar Patch does Sleet's job "Well Enough".

Dark is better at SO levels, and arguably better against enemy groups in general. Cold is above and beyond superior against AV/GM's, and at extreme recharge levels, in general.

Argue it if you want, but this was an SO post, not an IO post >_>

Wtf are you talking about? That guy was talking about high end performance and the capabilities of corruptors. Obviously if that's what you're talking about, you assume high end builds for both. Max potential, Cold beats out Dark. If you want to continue this conversation that's cool with me, but please PM me instead if you don't mind.

I meant MY post, that's all! And nah, let's drop it.

We didn't say anything about theories. You have to be careful when conflating scientific theories and the scientific method with statistical analysis and data gathering. The big reason why the things you list above are bunk is not because they were created by modeling what people observed - that's exactly the way science works. The crazy circle-inside-circles system that Ptolemy built for describing the observable universe was, likewise, wrong, but it was still a good model that served its purpose for a long time. Likewise, Galileo and Copernicus' observations led to them to make even better models. Rinse and repeat. Making models based upon your observations is the way things should go.

The problem with things like Astrology is that you can't test them. The model itself isn't making verifiable predictions that you can test. When I say that Religion is when you alter the Data to fit the Model, that's exactly what I mean: when you perform a test, you then attack the test if it doesn't fit the Model, then tweak and throw out the results until it does fit the Model.

Luminiferous ether, by the way, is a poor example. It was an inaccurate model (as we now know), but it was a scientifically valid one. It was verifiable, people were able to draw conclusions and make predictions based upon it, and the Michelson-Morley experiment (among others) tested it and found it wanting.

No the problem was that epicycles etc never predicted more than the data already showed. Given enough terms you can fit a curve to any dataset.


Think of the situation where you have an older vintage television that has bad over the air reception. This isn't usually a problem because the person watching it can't see. Every so often the picture gets bad enough that it upsets the viewer to the point they feel the need to do something about it. They get up adjust the antenna, bang the TV on the side, maybe assume a particular position relative to the set and after doing one of these things their picture gets better.

You have a 100% correlation between their taking action and the picture improving. So obviously what they were doing was fixing the problem. No, the problem is that the set is old and the capacitors in the tuning system are aging out, their values now vary over time. When their values are still in range the phase locked loops can properly sync to the signal when they aren't too bad Mr. Couch Potato.


This is exactly the same problem that permeates this game. Take task force rewards you want TFs to give an average merit reward/unit time across the player base. What do you do ? You sample the times it took to run the given task force and peg their rewards according to that number. This ignores the fact that the people who speed run tfs can see what you are doing and adjust their behavior accordingly. What is more those same people were overwhelmingly abusing the TFs they could break. The end result ? you get 9 merit Edens, 9 Merit Katies, while the people that abuse the system go off to run down citadel and manticore.




The problem with Ether was that it was thrown out there solely for the purpose of preserving a world view. In the context of this discussion its the equivalent of saying blasters don't work well because of (Snipes being poor choices, Crashing nukes being traps, blaster attacks not being more end efficient). They are obviously less desired powers but how badly hurt can blasters be by things they don't use ?

For the sake of argument lets say we go down the really bad process of saying aha we have sampled blaster performance and they still underperform. It's not as bad as it was before so our last fix to the AT did most of the job lets fix some of the minor problems that remain. We get another round of bad fixes that may or may not have an effect on the AT. Fixing Snipes would be a good example of this. Blaming an AT's problems on a power that people mostly skip and don't use very much when they do take really won't fix the AT. Matter of fact improving snipes, would likely widen the gap between blasters and other ranged ATs that have access to them.

I never claimed that THB's project was anything but an experiment. Maybe you misunderstood that I was talking about how having statistics that show comparable reward earning rates can mask the fact there actually are problems. What's more they tell you nothing about what the problem actually is.

You keep saying that like repetition will make it true. Statistical information can tell you a lot about whether a problem exists, and constrain the likely sources of the problem. Statistics isn't complete, but neither is anything else. Models and experiments themselves are not complete, until large bodies of evidence confirm them or generalizable theories emerge from them.

But most importantly, in this game whether an archetype is performing well or not is itself a statistical question, because an archetype is judged to be performing well if it behaves within certain limits for a majority of players, and the distribution of performance by all players falls within certain limits. There's no Aristotelian version of performance that exists within the archetype and outside the actual results it produces.

If you think it should not be so, you'll have to make your own game and your own design metrics.

Sure it did. People used it to predict things like when and where Venus would appear in the sky. That's why it wasn't until telescopes came around - thus increasing the amount of data available and allowing for early scientists to compare it to the predictions of the Ptolemaic - that alternative theories supplanted it.

This is basically how things like Test Driven Development work; and they do work. You take an iterative process of examining your data, altering the model to fit, then examining new data, rinse and repeat - periodically taking the time to refine the model, applying rules like Okkam's Razor. That is how science works, too, because there's no way for people to magically know the correctness of their model; all we can do is create more predictions, perform more tests, then alter the existing model. Most things work this way, and very rarely does it require you to come up with a completely new model from scratch.

So the data states that it regularly goes bad, the user performs a specific interaction, then it gets better, loop. That's your data, and you can construct a model. Then you test the model by making your main prediction: if the user does nothing, the television will remain with poor reception. You perform the test, you discover that it recovers on its own, so you have to change your model to fit this new data point. Likewise, you can do the opposite and require the user to try multiple things all the time, and record the results. The user would be pretty quickly eliminated from the model.

"peg their rewards according to that number" is the problem here. Statistical analysis did not tell the Devs what to set their TF rewards to; statistical analysis simply told them that a large number of people were receiving far more rewards/time running these TFs than doing other content. The "solution" they chose was based on their own evaluations and a Systems Designer's decision for how to formulate balance.

I'm really not sure what you mean by this. Do you expect scientists to regularly throw everything out and magically invent more accurate models? Working with the existing system is how everything in science starts. Ether wasn't "thrown out there", either: it came from a lot of different places, but mostly from (a) the idea that waves required a medium to pass through, and it had been shown that light was a wave (wave-particle duality wasn't conceived of yet), and (b) the fact that Maxwell's Equations implied that light traveled at c in a vacuum, but the only way that would make sense in Newtonian Mechanics was if it was in relation to some specific reference frame (e.g. Ether). Again, it wasn't until later that the idea that the speed of light was the same in relation to all reference frames was introduced (again, due to testing and making predictions based upon the evidence).

And all of this is outside the realm of statistical analysis. Analysis of the datasets will tell us what people do and don't do, and what results they get based upon those decisions. In fact, it could tell us very interesting things like "how well do people who don't take snipes fare versus people who do" or "of the people who have snipes, does a correlation exist between how often they use it and their performance". That might help us determine if the snipes are actually detrimental to the player. How to fix it, though, is the step after this. That's where you could come up with a hypothesis, like "making snipes better will bring blaster performance up to par", which you could test and see how it results.

I don't think "will accidentally buff Defenders, Corruptors, Dominators, Stalkers, and Scrappers" is as big a concern as you make it. For one, they could only apply the changes to Blasters (after all, Dominators got their snipes revamped into higher damaging powers and nobody else saw it) - though I don't like this, because Blast Sets are pretty poor for everybody, and its only because Corrs and Defs have such awesome Support Sets that they get by. For two, the idea would not be to buff snipes in isolation, but in conjunction with other things that would result in an offensive synergy, making the changes more valuable to Blasters than to others.

Proper analysis recognizes the limits of the analysis tools as well. Statistical analysis of reward earning rates is subject to dilution, but not amplification, across archetypes. That means seeing a problem means its there, not seeing a problem doesn't mean its not there. The statistics are sensitive to certain things, and insensitive to others. A proper analysis would recognize that, and not simply say "statistics are misleading." Everything is misleading when used wrong.

But you can't get "rewards earned" wrong because the server just counts those: getting that wrong would cause people to notice not being awarded things correctly. You can get "time to earn" wrong, but almost always by overshooting and not undershooting that time. You can't count less than the actual time spent running missions and shooting at things, when done across millions of missions and thousands of players.

If Blasters level slower, that means there's a problem. *That* statistic doesn't say why. But if Blasters have more debt, and the debt difference accounts for a significant percentage of the gap, then debt is definitely a problem. And that means dying at a higher rate is definitely a problem, because that's the only real way to get debt. *How* Blasters are dying more often that statistic doesn't say. But if its determined that Blasters are mezzed at the moment of death to a far higher percentage than other archetypes, and spend more time mezzed in general, then *that* statistic points to an element of the problem: blasters die more often in part because they are more vulnerable to being mezzed, and more vulnerable while mezzed. *That* points to an area of focus: to reduce the amount of vulnerability to mez.

If that isn't the *only* reason for higher debt: if factoring it out doesn't eliminate the problem, then that implies that *another* problem is damage mitigation in general, because damage is essentially the only way we die. And that suggests either two requirements - reducing the effects of mez and improving the ability to mitigate damage, or one singular super-requirement: increasing the ability to somehow mitigate both.

All of that is statistically determinable. The statistics don't say nothing of value: they are highly valuable, if you know what their limits are. As all analysis have limits. A good analyst would use all the information at their disposal, in the way most useful. THB's test can't overturn a large statistical analysis, but that doesn't make it useless or I wouldn't bother with it. It can say something about the issue, but what it can say as a single data point would depend on the information collected and what distinctions it highlights. Just as with statistical analysis, some distinctions would not be significant because they would be outside this test's ability to properly distinguish. Others might be more significant because this test more accurately represents them. But that's difficult to predict ahead of what's observed.

Are we looking at the same data? It looks to me like the blaster took less time to complete the same stuff, but with a couple of deaths...

What am I missing?

THB is using timestamps as markers, not durations. If you parse it out, his first run had the Blaster take 58 minutes to go from leaving the tutorial to level 7 with 1 death and the Stalker take 57 minutes with 0 deaths.

On his second run, the Blaster took 54 minutes to get from level 7 to level 9 with 1 death, while the Stalker took 53 minutes with 0 deaths.

Figuring about a minute or so to hosp and get back to the mission after death, that's pretty much identical so far. Aside from the deaths, of course.

The numbers are still small so its difficult to extrapolate from here, and especially because there are only two deaths so far, but if that death rate remained roughly the same throughout leveling its worth noting that on a spawn-adjusted level that's going to equate to about 2 deaths per level in the 20s, and ten deaths per level in the 40s. It probably won't remain constant, but those two deaths in nine levels is actually a fairly high number on an adjusted basis: its about one death every 150 spawns points.

I say that's a high level: its high on an absolute basis, but its probably a very low level compared to the average player. Incidentally when debt kicks in, my guestimate is that separate from travel time each death per level costs about 1-2% in overall performance.

The interesting thing would be to note if the defeat rate goes up slightly in the 20s when mez becomes more common. Although on the red side that does not occur as dramatically as it does on the blue side: the red side content has a higher level of mez and other secondary effects from the beginning.

That's my guess too. Either the Blaster will start accruing debt or he'll have to take pains to avoid dangerous situations, which will slow him down where the Stalker can breeze through more easily.

Personally I'd like to try my hand at this test too, but using a Corruptor or Dominator. Comparing a Fire/Dev Blaster to a Fire/Traps Corruptor would be simply unfair, but a Fire/Ice Blaster and a Fire/Cold Corruptor could be an interesting comparison. (In that case I'd actually expect the Blaster to win the front 25, but the Corruptor would pull ahead in the back. Benumb and Heat Loss, what what.) Sadly I haven't got the spare time for it currently.