Any QA Left?

Well there's your problem right there.

Does anyone know of an MMO *anywhere* where rigorous regression testing is performed? I've helped with the design and implementation of regression testing and so far I haven't seen or played a single MMO that shows clear evidence of rigorous regression testing of patches.

I sometimes wonder if one reason for that is that most MMO dev teams are opposed to bots in their MMOs, and there is thus a subtle bias against trying to write automation tools necessary to do hardcore regression testing.

I kind of wonder if it's common in the gaming industry period, but we get the most visible examples of the seeming lack of it in online games.

I think part of the issue with MMOs is that they have a kind of captive audience. I seriously doubt they actually think of their customers that way, but at some level they know that people will either stay because they like the game, or not. As long as the bugs aren't so bad that they drive people who like the game away in droves, my guess is they figure we'll deal.

That's an interesting thought.

I also wonder if it's that gaming shops don't seem to have the kind of rigorous or formal atmosphere that enterprise or commercial projects often try to maintain. In my limited exposure to game shops, they usually project a feel more like a team of smart people with good ideas that shoot from the hip, kind of like the internet start-ups in the '90s. In my own experience, that sort of thing can carry you if your product is relatively small or if you don't have to answer to a lot of people for its direction and functionality. The bigger the system or the more customers it has to satisfy, the better served a project seems to be by formal software lifecycles.

So, you're saying that a bug that has creatures in Ouro arcs spawn at thier lowest possible level and makes level 50 TFs critters spawn at 49th level IS FREAKING HARD TO FIND?????
Riiiight.

This right here shows me that you have very little knowledge of the complexity of the issue.

Is it necessarily hard to find a bug like this? No. Can it be? Yes. Very yes.

There's an estimated million lines of code in this game. Probably much, much more nowadays. Trying to find a bug in all of that code can be a pain in the *** to find. And that's assuming that the buggy code is where you might expect it to be. What if it's due to something seemingly unrelated? What if it's a stray bit of code that got misplaced somewhere else? What if the code seems fine, but the bug keeps appearing anyways. What do you do then?

Just assuming that because they know what the symptom is that they can diagnose the problem easily is folly.

Angel Red
"We're heroes. Fighting injustice is the thing we do." ~Rebel Scum

There's a difference between finding the bug and finding out what's causing it. You've already found the bugs, now tell me why they happen. Once you've told me why they happen, show me how to make them stop happening, and be sure your fix doesn't break something else. As Aett_Thorn said, any program of appreciable complexity, such as City of Heroes, has a ridiculous amount of code, and virtually any single line, any one command, even a misplaced letter, could cause the problems you've described. To fix the problem, the cause must be found, corrected, and tested, because any change of code is going to affect the entire program, even parts it shouldn't affect. Because software is weird like that.

This is the plight of Quality Assurance.

That depends what the initial purpose of the patch was for, and whether or not it was tested in-depth to see how it affected the other areas of the game and/or in other possible circumstances.

It's obvious that thousands of players are going to find problems a lot easier than a couple dozen people on the test server. That, and sometimes thing happen to features once they go live as well.

If it was from the last patch or two, then it was part of the Halloween event. And the spawning bugs, they are not a showstopper. Plus, with the event still going, they don't want to patch it and have that go all bums up.

I would think there is a good chance that we will have a patch to test within the next couple of days, that if all is well, will go live after they kill the event.

The developers actually have a stated policy about patching during special events. They don't do it.

Case in point can be seen on the Veteran Reactivations which shortly follow new issues. Despite some show-stopping bugs in the code, the developers didn't actually patch some of the bugs until after the special event was over.

The basic problem is that many of the special events (Halloween Event, Christmas Event, Valentines Event) require special coding and often have their own branch in the system development. As we've seen, scripting problems with one segment of the game, such as the interaction with the auction house window, can have an impact on the rest of the game, such as the chat colors, that wasn't ever intended.

It's really a case of the demon you know versus the devil you don't know.

Yes, some of the bugs introduced are annoying... but as long as the game client is loading and operational for the majority of users, the devs aren't going to go messing about under the hood.

As an example, I'll point at Champions Online... which is going through a lot of this. If you check out the patch notes page : http://champions-online.com/patch_notes : you'll find a /chunk/ of patches going up. This is because the CO developers are still mucking about under the hood and changing how the system works, which keeps causing problems for people playing the game.

Now, you seem to think that the bugs in the most recent patch for our game are show-stoppers. That they are just so bad the developers shouldn't have published. However, before that, the developers had a massive amount of feedback coming in about how some things were broken: http://www.cityofheroes.com/news/patch_notes/

So they made the best changes they could to make the game playable for the most amount of people possible.

It is a judgment call on whether or not to release and updated patch that might have a couple of known issues. Given the timing and the content of the two patches though, it's very likely that the developers only made changes in a couple of scripts, the incremental updates that shouldn't have affected anything else.

Only... that didn't work.

Which means there are larger underlying issues. But since the game is mostly working, the developers will simply have to let those issues wait until after the event is over, rather than risk screwing things up further.

It was the trick to the other patch's treat

So what? The "stated policy" clearly isn't a hard and fast rule since they released this patch with a TON of items not related to the Halloween event.

So, apparently, they do do it.

I think, devs rushed that patch live primarily because of Ouro level 1 spawns, that perhaps in their eyes was considered as an exploit.

Thankfully, I will just respec into [Dishrag] when that happens.

Gosh, I thought you knew these things

I think they have a pill for that now.

How is getting a relatively small patch rolled back "disasterous?" Were there massive earthquakes? Did the game spontaneously blink out of existence?

So you have to wait a few more days to walk. Somehow I don't think that warrants questioning QA's ability to please you.