Any QA Left?

Disastrous? Hardly. I was playing along quite nicely and didn't even know what they were taking the servers down for as I had not had a single issue.

Thinks about it,

Thinks again,

Recalls a little bit from the Tao of programming.

" A well written program is its own heaven
A poorly written program is its own hell"

Now admittedly I don't know the details of the inner workings of the game, BUTTTTTTTTTTTTTTTTT its pretty damn easy to notice when variables aren't being properly initialized. That happens all over the place and it does not bode well for the underlying quality of the code. You toss in that we keep getting more and more side effect type issues, its not good.

To some extent it is unfortunately unavoidable. CoH has been running for 5 years and has received many updates during that time most of which have been released on a very tight schedule. Add in the fact that various programmers have come and gone over the years and it's not really surprising that the internal code is a bit messy. Over time any code base that is being maintained will be subject to entropy as new code is added, old code is removed, bugs are patched and programming methodologies change.

As for your statement that it's pretty easy to notice when variables aren't being initialized, in a large code base it really isn't. If you're lucky your compiler catches them but compilers can't always tell if a variable is being correctly initialized so might not catch it.

"That Game Development is complex" is commonly called an excuse. Many customers find excuses unacceptable. To take your example, reloading your customers system is clearly inadequate. A filtering system, third party modifications for IE that provide greater security, or providing a virtual machine with a revertable snapshot are all solutions that seem indicated as actual fixes for his difficulty. Perhaps your customer is someone who will continuously accept excuses, and pay for non solutions, most people won't.

Just as another example wiring a national telecom infrastructure is complex and providing high speed digital communications to all locals is complex. This doesn't stop people from switching their ISPs when they do a bad job or are unreliable.

The above does imply an interesting question though.

Fine, let me simplify the excuse. Software Development is complex. It is complex enough that in relatively short order any code base becomes so complex that "proper" coverage and regression testing becomes economically unfeasible for most code. That is the simple fact of software life.

Back in 1986, when "640KB was enough for anybody", there was a paper on software development called "No Silver Bullet — Essence and Accidents of Software Engineering" by Fred Brooks who was know for "The Mythical Man-Month" the book about software development and project management. The gist of "No Silver Bullet" was "there is no single development, in either technology or management technique, which by itself promises even one order of magnitude [tenfold] improvement within a decade in productivity, in reliability, in simplicity." Short version, software development is extremely hard and unpredictable, there is no silver bullet to kill the werewolf of software development complexity.

Now when you combine that with Nathon Myhrvold's (former CTO of Microsoft in the 90s) first law of software "Software always expands to fit whatever container it is stored in" with Moore's Law implies that in a computer technology capabilities grow 60-100x in a decade it means it's a losing battle, software complexity outstrips a developers ability to produce good quality, timely code at a reasonable cost.

In the 80s one of the first professional jobs I worked on was a military software project, a subsystem for a fighter jet. The best turnaround time between a change request and installable software was 9-15 months. This was because every line of code was tested and retested and retested again. Total code coverage, full range checks on inputs and outputs on every module. Testing at every level of integration. And that software fit in less than 128KB, had less than a hundred inputs and an equal number of outputs, most varied at human speed (switches, rotary knobs) rather than by another computer or electrical device, no dynamically allocated memory.

That level of development simply won't fly for $50-70 entertainment software, especially for an MMO. Take to long between releases, people leave. Charge more so you can test thoroughly and it doesn't sell at all making testing moot.

I could probably continue until I'm blue in the face but I'm sure those who think it's possible to test and produce six sigma software of this size and complexity with a sub $10 million a year budget, the same people who think doctors can diagnose your medical problem with 100% accuracy from a chart of test results or animal rights activists who think product testing can be simulated on computer than use actual rats, bunnies and monkeys will just say that's just an excuse.

It was a stability issue, from what they have said. Various people had mentioned not seeing any issues. So apparently it's kinda localized.

And quite frankly, the test server never gets the same kinda loads the Live servers do UNLESS Open Beta just started, so cool off. Please?

Agreed.

Nor is it the "let's discount all the work other people do that I know next to nothing about, focus on one aspect of that work and then pretend that the players can even answer the 'question' I proposed" Thread.

You are a horrible person.

Do you need a ****** section?

I can't even begin to touch the wrong of this. First Fred Brook's point wasn't that there was "NO WAY", just that there wasn't one single technique. Even in that context, he was speaking of accidental complexity, (Accidental complexity = unintended consequences). He even specifically outlined methods of dealing with essential complexity (Software that deals with intractably difficult to properly analyze or abstract situations) .

So what you are saying here is that the availability of extra resources forces people into producing bad code, and it is nothing that anyone should get upset about when a product you pay for doesn't work.


Let me see here. The low volume servers did not have the mapserv problem, the high pop servers did.

The words of the day are "Load Testing"

This software has been out five years and they have no way to put it under load before they release new versions ?

GOOD AD HOMINEM, nice and indirect. You missed things like fascists that demand unachievable results even though that is hinted at in the body of the post.

Oh and please don't use military software development in the 80s as any kind of paradigm. I have my own fun experiences in that very environment. Producing a working product on time and on budget was always one of many competing objectives and usually not the most important one. What is more the level of bureaucracy in those projects has little if anything to do with software engineering.

ROFL.

Which reality would that be ?
Customers making excuses for companies delivering flawed product ?
Assertions that it is impossible to write good software ?
Assertions that it is impossible to perform basic tests on software before releasing it ?

Its pretty damn obvious that it was decided that live should be the stress test for releases. The terms of the eula make it a no cost option for them.

So, you play on the test server all the time right? Because load testing is so important to you?

There is a huge different between making excuses and giving plausible reasons.

No one is saying that it's impossible to write good software. What they're saying is that with a codebase that is complex as any MMO's is, and with so many different people working on it (and different programmers think differently and code differently), mistakes and unexpected code interactions are bound to happen. Those are not always immediately apparent.

No one is saying that either. You really need to stop blowing things out of proportion...the fact is, with as many powers, systems, combinations, missions, it's literally impossible to test everything every single time a patch goes out without thousands of manhours devoted to it, and even then things are bound to be missed. Yes, basic tests are performed. I'm sure the QA team is very good at basic testing, but it's impossible for them to cover every single angle, and some things just can't be artificially simulated. The QA team doesn't have the ability to test code on every single possible hardware and software combination that can possibly play the game either, so there's a problem right there.

Also, humans are funny in that when you're testing something, you get into certain routines, and certain expections. It's unavoidable, and unconscious. And players tend to think of things that the developers and testers just didn't think of...really, if you want to break a given piece of code, there's no better way to do it than to give it to your player base. If someone doesn't know that something isn't supposed to be able to do something, they'll try it. Things happen. I know that's what the test server is supposed to be for, but there just isn't enough people on there to cover every single possible problem either (though our players sure do try!). And, even then, our amateur testers can fall into the same trap as the professional ones after testing for long enough.

What is "pretty damned obvious" to you doesn't make it correct *shrugs* but it seems you don't want to hear that. I'm not going to assume that you're not open to hearing it, but you certainly give that impression.

LOL its not my product. I have no financial interest in it. I will derive no career advancement from it. If it doesn't work right long enough and often enough and the outages are severe enough, I will simply suspend my sub.

If your car company said to you, "well if handling testing is that important to you why weren't you at the test track", would you just say, "Oh you are so right. Let me fly to detroit and test your product for you on my dime" ? If a drug company said to you "Well if you care so much about that foot growing out of your forehead, why didn't you volunteer to be a test subject" what would your reaction be.

And don't forget to criticize BAB for wasting his time on the walk animations when he should have been busy fixing the task force difficulty settings!

A really good, and recent example of this is the animation bugs. BABs and crew have been fixing animation bugs. When they do, it causes another domino to fall over. After they fix that, something else breaks.

They're on the nth revision of the animation stuff. One of the most recent bugs is shields while flying causing jitters. Not many people run flight with a shield melee toon. What's more, there was a bug that only applied to a couple animations of a weapon set, only while flying and with a shield.

Take all the possible primaries, plus all the possible secondaries, plus all the possible pool powers. That's how many things they'd have to test every time they fix something. An animation bug that ONLY exists when using one of only two bugged powers, and only when using a weapon set, with a shield, AND while flying, is a very specific bug that wouldn't be caught with "general" testing. And it's extremely unreasonable to expect the testing team to test every single primary/secondary/pool/epic combination every time a change is made to animation code.

Originally Posted by PRAF68_EU Dispari has more than enough credability, and certainly doesn't need to borrow any from you.

Not the same thing at all. A given car has set routines that they go through for testing, and a limited number of things it can do unless they're introducing something new. Also, a given model of car (or a specific drug) isn't constantly evolving, changing and getting more complicated. Once the testing for it is done, it's done unless a problem outside of the normal parameters comes up.

Hello? McFly? Sure, they can SIMULATE running under load, but the simple fact is that it's not going to give the same results as ACTUAL load. Why do you think we get product recalls of cars and their components all the time? Testing simply can not accurately reflect the massive variability in real-world conditions. It just can't. That's why there's a test server. Internal testing will never be as effective as having the end users beat up the software. And if the test server doesn't have enough people on it, then there is no way at all to be sure what's going to happen under full load. None at all.

No there is not. There are reasons for everything in the world. When you take a reason and say this should be overlooked because of reason X, the reason becomes an excuse. Whether you choose to accept the excuse or not is entirely a personal issue.

What kills me here is that the OP made a perfectly valid observation, and all of a sudden the amateur red names are trying to jump down his throat and making excuses for NCSoft.

I lold sorry, but using real world in context of a videogame that simulates a fantasy setting is inherently comedy generating.

Anyway you are mischaracterizing my position, the question here is was it done, not if it can catch everything

Then I guess you're deliberately acting the fool. We're not talking about events in the game. We're talking about software development. Sounds like 'real life' to me.