"You are the Dev" Issue #1: Fix the Problem or the Perception?

Agree with most of what you've said here, but remember, only a limited number of players do open beta, so to be more granular, 100,000 players>>> 2500* OB testers>>>10 devs.

And you can only test something for so long before it becomes a case of diminishing returns (no pun intended). At some point, you have to Let Fly, (as it were...) and hope things won't completely break the game. And if they do, you can roll it back, etc.

The devs also have to deal with players perceptions of things like feelings, fairness, fun, and balancing these considerations against each other... fixes often are delayed for these reasons.

* - I pulled this number from a dark and strange place. I wouldn't go to that place if I were you.

Willpower had One With The Shield during beta?

I never tested Shield Defense, but I do recall Willpower being overpowered during early beta, especially for Brutes. HPT's resistance was being buffed by damage buffs, including Fury...

They probably intended for us to use them for raiding, when instead most players use them for convenience and a few players use them as a digital Lego set. And in this case, it wasn't game-breaking, so they made a change that effectively changed what bases are supposed to be used for.

And then there's AE. Let's not get into the farming issue here. Over 400k arcs have been published in the last two years. I doubt the devs anticipated anywhere near that volume. I also think they didn't anticipate how much of it, even discounting the farms, would be so obviously and eye-bleedingly bad. I'm not even talking "Twilight" bad, I'm talking "this is just a test to figure out how AE works" bad...with more spelling errors. If they had, Positron never would have made a comment along the lines of "but if you can't rate an arc you haven't completed how can you rate something that is so bad it's uncompleteable?" because as it stands, you don't have to. That bad arc will sit at 0 rates along with tens of thousands of others, where very very few people will ever see it.

I also strongly stated my opinion that Willpower was far more powerful than virtually anyone was giving credit for. I was actually amazed it was buffed on release. I blame Starsman, although that's another story.

This is how it works, is it not?

What you really mean to say, Marcian (PLEASE forgive my presumption, as it is theoretical) is "Please Hammer (players) don't hurt 'em (Devs) while I'm gone."(?)

More or less this.

Originally Posted by Arcanaville Samuel_Tow is the only poster that makes me want to punch him in the head more often when I'm agreeing with him than when I'm disagreeing with him.

I have a partial solution to the problem, but I honestly haven't had an opportunity to discuss it with the devs. I was thinking about it when thinking about how to fix negative stealth. Which I think I might know how to do also.

Semi-cynical response: How much does the success rate go up if we assign fewer of them to balance something? (My limited experience is that at least the later-stage fiddly number balancing sort of thing is in fact best done by a single person, as seemingly orthogonal changes usually aren't as independent as you thought.)

Week #1: Senior dev / producer / etc. monthly letter to community; summarize last month's patch as usual, and then talk about upcoming changes in general terms. Have a paragraph that says something along the lines of "We routinely datamine performance at several levels, as we'd prefer to catch problems before they become obvious to players. No solely numbers-driven review is perfect, however, and our Community Team has been paying attention to some of the recent forum discussions. This month, we're focusing developer attention on several balance issues, both entire sets and specific moves that have come to our attention. This is a good time to remind everyone, however, that due to the diversity of powers and situations in CoH, there will always be some cases where certain builds will outperform others to some degree. Our goal is to keep that degree down to a reasonable level while providing enough real choices to keep the game interesting. We ask that you help us out by providing our Community Team with clear, specific examples in the stickied threads they will be starting on certain specific issues this week; and hope to see you on the Training Zone later this month to give us some good data on the improvements. As usual, the live publish will include a freespec."

W1 Assignments: Devs A, B, & C brainstorm and rapid-prototype ideas for fixing #1, Shepherd's Crook. Dev D work with Community Team lead to set up and monitor designated threads to discuss #2 (CoJ), #3 (Meekling), and a couple of other balance projects high on the to-do list; specifically including dev posts in the threads to focus attention. Someone on CT assigned to work with Devs ABC on research, including searching forum and off-forum HG boards for signs of #1 exploit knowledge spreading.

Week 2: Meeting 1 (early in week), figure out which plan for fixing #1 gives the best combination of balance results while preserving both original intention and powerset feel for those who have taken it. Assign whichever dev came up with the best plan (we'll assume it was A) to flesh it out full time.

Meeting 2a (also early in week), pull together the forum posts and additional datamining to try and figure out *why* the players think #2 (CoJ) is imbalanced. Even if the powers are numerically correct, there is likely to be something at the root of the perception; the power's looks, sounds, situational issues, popular enemies / maps / farms, incorrect documentation, or something.

Assignments: Dev A full time on plan for #1 rebalance. Dev B,C, 0.5D on clever ideas for #2 fixes; may involve other teams if the issue really turns out to be about animation times or some such. D spends some time also checking over Arcanaberg's and remaining community suggestions for fixing #3, focusing on possible numerical fixes that don't require new tech or other resources.

Meeting 2b (late in week): Bring together folks to pick an option for handling #2, assign someone involved with the best idea to deal with it (we'll assume B). Check to see if folks are generally OK with the concept of changes for #3.

Week 3: A working full time on #1, B working full time on #2, D now full time on #3, although hopefully this will be mostly regression testing as the actual changes are quick. C works on New Stuff until late in the week. Test builds internally by mid-week. By the end of the week, A, B, and D are expected to be able to (independently) explain in a few minutes to C what their change *actually* does, and what it *looks* like it does, to someone who knows the powers system well but hasn't been looking at those parts of it all week. Dev D sends NDA'd PM to Arcanaberg and possibly selected other folks who made good suggestions on #3, outlining proposed changes. By end of week, closed beta build goes up so select players can bash on it over weekend; beta channel instructions are explicit that folks should focus on changes to #1, #2, and #3 (and possibly any minor New Stuff that worked its way into the build). If there's a competent and separate QA group, they start their checks.

Week 4: Monday is evaluate closed beta data, final dev checking, and then publish for open beta Tuesday morning unless some parts are simply unworkable. Senior dev post on main boards discussing the opportunity to test out changes to #1, #2, and #3, and focusing . CT brought in again to focus sifting the firehose on forums and training zone channels. A, B, and D responsible for showing up at least once to an announced event on the training zone to interact with players about their temporary focus area directly. Encourage Arcanaberg via PM to work on a detailed post explaining how #2 is "now" balanced compared to similar powers.

Late Thursday, big meeting to decide if things are working, or if *minor* tweaks are both necessary and possible; decision made as to which changes go into Release Candidate build Friday morning. (Hopefully, no changes from the Tuesday AM build, or minor typo stuff.) Check to see if Live servers having any significant problems from people figuring out #1, which may influence how desperate it is to get that into this month's patch.

RC runs Friday - Monday, and unless absolutely disastrous last-minute issues will go live Tuesday morning. Senior person writes monthly letter talking about balance changes, starting off with #2 and mentioning something about "wanting to make the minimum changes necessary to bring the power into line with similar powers" etc. Describe #3 in terms of "We understand that folks have been unhappy with this for a while, but resources are limited", praise the community for being rational and involved. Be honest on #1, "Previous changes did not have the desired effect, and we have had to adjust SC to prevent it from becoming a problem. We have gone to a fair amount of effort to preserve the feel and flow of the set, and welcome detailed feedback on how SC performs on the Live servers." Consider sending Arcanaberg cookies.

This all assumes that the problems are both urgent enough to try solving in a month's timeframe, and tractable enough that it can actually be done. If #1 blows up badly, it might be necessary to pull everyone onto that and push a hotfix; make sure it is announced as a temporary measure that will be refined as soon as possible.

A somewhat more pessimistic take would be to assign 3 devs to #1 in hopes of getting something usable before it blows up on live, and the remaining dev to figure out a smokescreen change to #2 and to make a concerned but apologetic post on the #3 forum topic explaining that they acknowledge the player concerns, would like to fix the problems, but simply don't have the resources to do it right now.

A somewhat saner take would stretch the process out over two months, with the first 2 weeks working on a crash priority quick fix for #1 that can be hotfixed in case it blows up on Live, then a more rational change process with at least a full week in QA+closed beta and a full week of Open, possibly two.

I'd regard the detailed schedule as an "OMG, things are coming apart" plan, we get only a few person-days of dev work on new stuff out of the whole month, which is unlikely to be practical very often. It's got an OK chance of working if "simple" numerical changes can have the desired results, but as the complexity and/or weirdness of the changes needed goes up, the likelihood of success in this sort of timeframe plummets. With really good devs, you can hope that the fix takes much less time and gives you more time to regression test, but you can't count on that. The more I reread this, the more it sounds like a "hotfix" process rather than a "patch" process, despite having a month turnaround; and the lead dev and CT need to be on top of whether they need to pull the red handle and put everything onto a #1 hotfix even faster.