Do the Devs even test their code anymore?
By
all_hell,
Posted on: 2011-12-07 13:27 in Player Questions
What about things like base defense items attacking players? Or the current knockback bug that is affecting one of my characters so severely she is unplayable?
Quote:
So, your position is that the game's programmers are not competent enough |
Originally Posted by ClawsandEffect
You said it yourself: It's not game breaking.
How long do you delay the release of an issue while you search for the cause of something that isn't game breaking? A month? 3 months? A year? As long as necessary, even if it means the issue is never released? As a programmer, can you say with any certainty that the borked code that's causing that bug is even IN the file the map is in? How do you know they haven't already looked everywhere and haven't figured out what's doing it yet? |
to find this bug? Frankly, I'd disagree with that.
Let me flip the question around, if your new car was missing the driver's side
mirror when you got it, how soon would you reasonably expect the dealership
to fix that? Weeks, Months, Never?
In answer to your implication, Yes, I'd prefer the devs to slow the pace down
somewhat to give us a better, more polished, and professional product.
If that timeframe is months, or as you put it never, then I'd begin to question
the competence of their staff or management's prioritization of resource provision
and allocation.
The part you are glossing over completely is that these types of bugs should
not even be IN the build - many of these should have been caught at the programmer's
desk unless a couple things are happening: 1> The coder isn't also the tester
of their own work. 2> The work schedules are SO ridiculous (ie. sweat shop) that
reasonable quality is impossible or 3> They simply don't give a rat's patootie
about what they're pushing out the door.
ALL of those practices are suboptimal if not outright bad.
If I had to guess, I'd lean more towards 2, but that still doesn't make it acceptable
to the end-user.
At least, not to *this* end-user.
Regards,
4
I've been rich, and I've been poor. Rich is definitely better.
Light is faster than sound - that's why some people look smart until they speak.
For every seller who leaves the market dirty stinkin' rich,
there's a buyer who leaves the market dirty stinkin' IOed. - Obitus.
Last time the fog of war issue came up(because it has before, and been fixed, and broken again) it was determined that it came from the number of maps in the game I think. For some reason the people who coded maps in i1 beta made fog of war revolve around the number of maps in the game. When more are added and it passes the "cap" in that code FoW goes haywire until that cap is raised again as I understand it.
10 50's To Date! Check out Titan Sentinel; it got my CoH presence synced online
Quote:
Good discussion. I think everybody has their own individual "threshold of pain" when it comes to living with bugs on the live servers. For example, some people saw getting face planted in one's base by one's own defenses as a significant drawback to game play. Others went "meh, it'll get fixed one day soon" (which of course it did). And a third group celebrated watching base defenses activate (for possibly the first time ever for them) and saw the potential for "danger rooms" coming into being for bases. That is a pretty broad spectrum of a reaction.|
Originally Posted by Zybron1
I have no special inside insight about the process here, but I'm nearly positive that unless it's a bug so severe that it will kill game play entirely, nothing reported in Beta is going to be fixed before it goes live.
Instead, I think Beta is the opportunity for us to QA what's going to go live so they can determine any items that need to be patched at a later date. There are exceptions, of course, like Titan Weapons (there was quite a bit of back and forth feedback and many changes actually happened) but stuff like the inspiration tray moving around is just not important enough for them to prioritize ahead of pushing code to live servers. |
The OP's argument impresses me because: (a) I have no reason to doubt his credentials or assessment in the area of Paragon Studios' performance versus others in the industry. My own personal experiences seem to confirm what he is saying (b) He provided specific examples which thus far have not been disputed (though YMMV on evaluating the seriousness of bugs versus release schedules). and (c) I too am somewhat surprised at the "bit error rate" of bugs, especially recently, and I believe, like I said before, there is room for improvement.
One man's terrorist is another man's freedom (or freem?) fighter; just as one man's exploit is another man's feature.
I must be really lucky. I never had any of these problems.
http://www.virtueverse.net/wiki/Shadow_Mokadara
Fourspeed - This isn't a car. It does not have /mechanical/ reasons for failures. Which are easier to findthan why the maps or UI busted when they were not touched.
It's not financial or safety software. It's a seven year old game, with more years than that on it's code base, with many previous programmers who are no longer with the game who didn't document their work - it's not so much an issue with the current coders as the previous ones.
Orc&Pie No.53230 There is an orc, and somehow, he got a pie. And you are hungry.
www.repeat-offenders.net
Negaduck: I see you found the crumb. I knew you'd never notice the huge flag.
Quote:
It's also called City of Heroes. It's also a mmo. It also features a storyline with super powered characters.
|
Originally Posted by Rajani Isa
Fourspeed - This isn't a car. It does not have /mechanical/ reasons for failures. Which are easier to findthan why the maps or UI busted when they were not touched.
It's not financial or safety software. It's a seven year old game, with more years than that on it's code base, with many previous programmers who are no longer with the game who didn't document their work - it's not so much an issue with the current coders as the previous ones. |
Quote:
+10 Winning Points
|
Originally Posted by Jagged
I think I must be in the wrong game forums because apparently the game most of you guys are playing is Buggycrap. Can I recommend you try City of Heroes instead? This last week I've played it more than I normally do, doing my first Underground Trial, quite a few Keyes and one or two story base arcs. I also created new characters using totally new abilities and newly purchased costume pieces. All without any problem. I have recommended it to most of my friends.
|
Quote:
I would too. But mostly because programmer competence is but one small factor in an issue as large as this. What programmers work on on any given week is rarely even up to them. The product manager for COH has many priorities to juggle, and bug fixes is merely one of them. And it may not even be the highest one. With so many customers clamoring for new content, new power sets, new costume pieces, improved graphics, yadda yadda yadda, I'm sure the production map for this game is a dizzying thing to look at. I do not envy the guy who has to keep this train on the rails. If anyone here is qualified to do a better job, I strongly urge you to submit your resume to Paragon Studios, for all our sakes.|
Originally Posted by FourSpeed
So, your position is that the game's programmers are not competent enough
to find this bug? Frankly, I'd disagree with that. |
- * - * - * - * - * - * - * -
Now, I understand the perspective that says paying customers aren't paying for buggy software. Unfortunately there is virtually no such thing as bug-free software, so the reality of our universe is that we are paying for buggy software, it's just that we're lucky if we simply don't encounter many show-stopping bugs that often. We as customers can stomp our feet and pitch a fit when we don't get what we want, but none of that changes the cold hard realities of large commercial software development. There are going to be bugs. Many of them will be classified as non-critical and will take a back seat to new features/content. Some of them will be maddeningly difficult to track down and solve. Some of them will take years to fix, and some of them will never get fixed. I think everyone has to come to terms with this or else they are going to find themselves forever miserable in this Real World[tm] we're stuck living in.
Furthermore, I think Claws is right in a sense. As a beta tester, one's efforts are paid for in the form of early access to the game. Implicit in the definition of "tester" is "bug reporter". If one isn't prepared to fulfill the role of tester (i.e., the job), then one shouldn't expect--nor be granted--beta access (i.e., the payment). The big difference, btw, between an unpaid beta tester and a paid in-house tester is that the paid testers are often told what to test and don't always enjoy the luxury of just running around playing as they normally would, having a good time without any management from a boss. A lot of the time a tester will spend countless hours testing one particularly tedious aspect of a game; it isn't fun, nor is it intended to be. So while the in-house tester is getting paid in dollars, the outside beta tester is getting paid in fun. Even if it is buggy fun. But the job is the same in both cases: report bugs (preferably with enough detail to actually help the debugging effort).
NOR-RAD - 50 Rad/Rad/Elec Defender - Nikki Stryker - 50 DM/SR/Weap Scrapper - Iron Marauder - 50 Eng/Eng/Pow Blaster
Lion of Might - 50 SS/Inv/Eng Tanker - Darling Nikkee - 50 (+3) StJ/WP/Eng Brute - Ice Giant Kurg - 36 Ice/Storm Controller
Quote:
You are correct - it isn't a car... That is the only viable point in the post.|
Originally Posted by Rajani Isa
Fourspeed - This isn't a car. It does not have /mechanical/ reasons for failures. Which are easier to findthan why the maps or UI busted when they were not touched.
It's not financial or safety software. It's a seven year old game, with more years than that on it's code base, with many previous programmers who are no longer with the game who didn't document their work - it's not so much an issue with the current coders as the previous ones. |
Be it game, flight control, or banking software, it is ALL code. From a conceptual
viewpoint many of the techniques are similar (processing a file, displaying to,
or querying from a device, testing conditionals, variable storage etc.).
In some cases, the actual code can be nearly identical. C for instance is an idustry
standard in a lot of software, including commercial aircraft (non-military) systems.
SQL is not unique to games.
Logical operations with these things are pretty standardized assuming
use of common best practices - A SQL Select is probably a near perfect example.
Age of software - non issue. Many of the computer processes that run in the
financial sector are decades old -- I've worked on several... Why so old?
Because they're well understood, safe and stable, and when millions of dollars
(or people's lives) are on the line, *new* things are far riskier in many applications.
A LOT of code is undocumented, and yes, it IS a pita (sometimes) to debug,
maintain, and enhance. It is no surprise in the software industry that the highest
cost of software is not development - it's maintenance (and people costs to
do said maintenance).
That said, CODE is what we programmers DO.
Undocumented code is NOT a showstopper - it is an annoying irritant.
Any competent programmer meeting the credentials on their resume can take
a piece of undocumented code (in a language they are proficient with) and
A> determine what it does and B> maintain and/or enhance it.
While true that interactions can have bizarre effects (sometimes), you are incorrect
when you say things don't fail for simple reasons. There are always reasons
(the vast majority of which are quite simple and straightforward) and clues to
the cause. Once again, seeing them, tracking them down and fixing them
is a large part of what programmers get paid to do.
Sure, it's not usually because Bolt A broke, but a piece of code that overwrote
a key variable another piece of code expected to use, or a misnamed variable,
or using the wrong variable, are most definitely simple, detectable, and actionable
reasons.
Case in point? The upside down Nemmie Staff animations were due (iirc) to
an improper table value - pretty darn simple... Should (imho) have never gotten
into the live build.
Assuming competent people given appropriate tools and time to do their job,
most of those things are really not all that hard to find, and in places with sensible
standards, many are often avoided in the first place, or caught where they should
be - in proper Unit Testing.
In short, the points you list are simply not viable or acceptable excuses to justify
untested, or substandard quality software in a production environment.
Again, I'm not saying bug free (some things ARE challenging), but not the types
of bugs we've been seeing recently...
I'm sorry, that argument simply doesn't fly wtih me.
Regards,
4
I've been rich, and I've been poor. Rich is definitely better.
Light is faster than sound - that's why some people look smart until they speak.
For every seller who leaves the market dirty stinkin' rich,
there's a buyer who leaves the market dirty stinkin' IOed. - Obitus.
Quote:
Seriously? Are you joking? If I needed a character to test high level content, I used to be able to get one with about 4 mouse clicks.|
Originally Posted by VoodooGirl
Uhhh...
Did you miss the LEVEL BUMP thread? Y'know, the one where you can make a new toon and request to get level bumped to whatever level you want? |
Now I have to:
create a toon
request a level bump
re-pick powers and slots up to 50
buy and craft IO's
I don't think so. I'm not going to spend an hour on menial tasks for the honor of doing free testing for the devs. Previously, I had a stable of toons available for testing things, as well as a supergroup and base. Then they wiped the test server and the character copy tool was either broken previously or at the same time. I've been trying the copy tool every 1-2 days to see if it's fixed yet.
I WANT to test stuff. I WANT to help out. But right now, there are simply too many obstacles. I'm not willing to jump through that many hoops. They need to fix the character copy tool. I see a lot of people complaining that there wasn't enough testing on recent issues. I don't know if that's true. It might be. I sure as hell didn't do any testing. And I'm not going to be doing any testing until the devs get around to making it easier for us to help them out.
Paragon City Search And Rescue
The Mentor Project
Quote:
I have noticed that this is true to wildly different degrees depending on the industry we're talking about. I don't think it is accurate to make the blanket assertion that all (competent) programmers in all application domains are going to be equally rigorous or effective at debugging and testing. I can't speak to what is "typical" in the video game arena, but in the two industries I've worked in as a software engineer, the attitudes towards testing and debugging have been almost diametrically opposed.|
Originally Posted by FourSpeed
Assuming competent people given appropriate tools and time to do their job,
most of those things are really not all that hard to find, and in places with sensible standards, many are often avoided in the first place, or caught where they should be - in proper Unit Testing. |
The programmer culture has, in my 30-year experience, shifted significantly over the decades. Having the Master of Debugging badge seems to be something few programmers (that I've met anyway) aspire to anymore. Unit Testing is an alien concept to most. You describe these skills and knowledge sets as something any and all competent programmers do possess, but if so, the world is simply not composed of as many competent programmers as you imagine.
NOR-RAD - 50 Rad/Rad/Elec Defender - Nikki Stryker - 50 DM/SR/Weap Scrapper - Iron Marauder - 50 Eng/Eng/Pow Blaster
Lion of Might - 50 SS/Inv/Eng Tanker - Darling Nikkee - 50 (+3) StJ/WP/Eng Brute - Ice Giant Kurg - 36 Ice/Storm Controller
Quote:
There is no such thing. |
Originally Posted by Ironblade
...the honor of doing free testing for the devs...
|
NOR-RAD - 50 Rad/Rad/Elec Defender - Nikki Stryker - 50 DM/SR/Weap Scrapper - Iron Marauder - 50 Eng/Eng/Pow Blaster
Lion of Might - 50 SS/Inv/Eng Tanker - Darling Nikkee - 50 (+3) StJ/WP/Eng Brute - Ice Giant Kurg - 36 Ice/Storm Controller
Quote:
The thing is, you have to play this unreleased content on the Beta server. All of that time and effort you put into playing and exploring new content does not carry over. To me, that's even less of an incentive to go on the Beta and Test Servers.
|
Originally Posted by ClawsandEffect
You are allowed to play new content before it is technically released. They are in no way obligated to allow us to do that. In return for that privilege, it is understood that you will be telling the developers about any problems you encounter while doing so.
|
Shortspark: 50 Fire/Fire tanker
Emberblast: 50 Fire/Fire blaster
Jessie Inferno: 50 Fire/SD scrapper
a wizard: 50 Rad/Sonic defender
The Nemesis Plothole: 50 StJ/Reg scrapper
Jessie Inferno: 50 Fire/SD scrapper
a wizard: 50 Rad/Sonic defender
The Nemesis Plothole: 50 StJ/Reg scrapper
Quote:
You forgot to mention that this whole episode is a LITERAL SLAP IN THE FACE! |
Originally Posted by Warkupo
personally, I wont' stand for this injustice any longer. It's time the devs answer to the PLAYERS, not their own SELF SERVING AGENDA. How can they continually be so STUPID and ARROGANT! This game used to be GOOD, but they have DESTROYED IT.
|
Quote:
|
Originally Posted by FourSpeed
I have been an applications programmer (primarily in the corporate financial sector) with companies (who shall remain unnamed, but would be definitely recognized) for ~34 years now (longer, I'd wager than many of our posters have been alive).
|
I, on the other hand, got my first computer about 35 years ago.... According to Wikipedia, the TRS-80 Model 1 came out in 1977, so 34 years. So while I wasn't employed back then, you're not the only long-term programmer.
Quote:
|
It is rather, the kinds of bugs we're seeing lately. The kinds of bugs lately, have been very simple, very obvious, and extremely simple to detect had their been ANY testing at all. |
Quote:
|
Originally Posted by Wing_Leader
I do not envy the guy who has to keep this train on the rails. If anyone here is qualified to do a better job, I strongly urge you to submit your resume to Paragon Studios, for all our sakes.
|
Paragon City Search And Rescue
The Mentor Project
If the Issue 22 beta is another "VIP beta" and is open from day 1 to any schmoe with a VIP account, and goes horribly as a result, I am SO PMing this thread to every redname I can.
Positron: "There are no bugs [in City of Heroes], just varying degrees of features."
Quote:
Agreed.|
Originally Posted by Ironblade
I don't have to be personally able to do a better job in order to have a valid opinion regarding the quality of a product.
|
However, I think that folks who believe they understand all the intricacies of the situation, and think they have viable solutions, with the experience to back it up, would be far more valuable to the process by actually being in charge, than merely by talking about it here. On the other hand, if they don't possess those skills and experience, then I have no confidence in their conclusions about what is or isn't an easy bug to fix, what the production priorities should be, what the development cycle should look like, what resources should be assigned to what tasks, etc.
In other words, any opinion offered is only as insightful and valuable as the direct knowledge, skills, and experience that back it up. Saying "I see crap!" is not nearly as useful as knowing how to clean it up. Personally, I think we need far less of the former, and far more of the latter.
NOR-RAD - 50 Rad/Rad/Elec Defender - Nikki Stryker - 50 DM/SR/Weap Scrapper - Iron Marauder - 50 Eng/Eng/Pow Blaster
Lion of Might - 50 SS/Inv/Eng Tanker - Darling Nikkee - 50 (+3) StJ/WP/Eng Brute - Ice Giant Kurg - 36 Ice/Storm Controller
Quote:
First, I did not say the highlighted piece - there are programmers of varying |
Originally Posted by Wing_Leader
I have noticed that this is true to wildly different degrees depending on the industry we're talking about. I don't think it is accurate to make the blanket assertion that all (competent) programmers in all application domains are going to be equally rigorous or effective at debugging and testing. I can't speak to what is "typical" in the video game arena, but in the two industries I've worked in as a software engineer, the attitudes towards testing and debugging have been almost diametrically opposed.
The programmer culture has, in my 30-year experience, shifted significantly over the decades. Having the Master of Debugging badge seems to be something few programmers (that I've met anyway) aspire to anymore. Unit Testing is an alien concept to most. You describe these skills and knowledge sets as something any and all competent programmers do possess, but if so, the world is simply not composed of as many competent programmers as you imagine. |
skill and experience without a doubt.
What I did say is that if you put C or .NET or SQL on your resume I would *expect*
you to be able to take a piece of undocumented code in those languages and
A> Tell me what it does B> Tell me how you would change it if I wanted to
add some arbitrary feature. That IS competence, plain and simple. I'd further expect
you to be able to test it to verify that it works properly and debug and fix it if
it doesn't.
If you could not do that, those languages should NOT be on your resume.
(Let's not get into the sideline threadjack of folks who falsify their credentials).
In the same regard, I'd expect a competent plumber to be able to find and
fix a leak in my household plumbing or change out a faucet or whatever other
normal things come with the job description, and a competent electrician to
wire up a light switch or find out why a particular circuit breaker keeps popping
without burning my house down or creating a man-made lake in it.
Writing, testing and fixing code IS the job description of a competent programmer.
On another sidenote, I'll add and agree that there are a LOT fewer programmers
who actually understand wth goes on inside code these days (Thank You .NET
et al, where a lot of the inner workings are increasingly inaccessible blackbox
mysteries they don't want a programmer to see). Sadly, the field is becoming
more of a lego block process, but that's also a different discussion altogether.
Again though, that leads to different sorts of bugs (typically) than we're seeing
with non-anchorable UI elements, option switches that don't work and the like.
Regards,
4
I've been rich, and I've been poor. Rich is definitely better.
Light is faster than sound - that's why some people look smart until they speak.
For every seller who leaves the market dirty stinkin' rich,
there's a buyer who leaves the market dirty stinkin' IOed. - Obitus.
Quote:
hehehe... not sure if that is a good thing or not, but I got one of the original |
Originally Posted by Ironblade
Probably. I seem to recall a study indicating that the average player age in this game was 28.
I, on the other hand, got my first computer about 35 years ago.... According to Wikipedia, the TRS-80 Model 1 came out in 1977, so 34 years. So while I wasn't employed back then, you're not the only long-term programmer. |
4K Trash 80's the first week they came out in my town...
I had a lot of fun playing with Z80 assembler on it and had a full printout of
the ROM and all its Basic hooks (custom commands can be fun).
I still have it in storage (Hmmm... wonder if it still works)
Regards,
4
I've been rich, and I've been poor. Rich is definitely better.
Light is faster than sound - that's why some people look smart until they speak.
For every seller who leaves the market dirty stinkin' rich,
there's a buyer who leaves the market dirty stinkin' IOed. - Obitus.
Quote:
There are also the issues of the cost of tighter quality control vs. the expectations of the target consumer. Take, for example, the two domains I'm familiar with:|
Originally Posted by FourSpeed
Sadly, the field is becoming
more of a lego block process, but that's also a different discussion altogether. |
Fault Tolerance is King
When the customer is an airport where smooth, unhindered operations are the highest priority (even over passenger safety), resources go towards systems that reach fault states as infrequently as possible (as opposed to systems that more accurately detect security risks). Guess where debugging priorities go for that application domain. A zero-tolerance policy towards known, possible fault states is expensive but necessary, and "affordable" only because the extremely high price tag of the product has that sort of testing discipline built into it.
Experimental Features are King
On the other hand, when the customers are a company's own in-house artists, fault states are highly acceptible as long as new features are being made available as quickly as possible (a new pyro solver that only works 50% of the time is better than an old pyro solver that works 99% of the time but doesn't look half as good). Do you think programmers in this second case are encouraged to spend time debugging fault states? Or do you suppose their priorities always seem to fall on side of new features? What do you suppose this does to the culture of rigorous quality assurance as a programming philosophy within a company?
Like I said before, I don't know what the testing philosophy is like within today's video game companies, but I have to believe that they take into account the consumer behavior/expectations of their customer base (and not just the vocal minority that log into the forums to kvetch about it). It is quite possible that the greater population of players are far more tolerant of bugs than we think, and this steers production priorities to a great degree.
NOR-RAD - 50 Rad/Rad/Elec Defender - Nikki Stryker - 50 DM/SR/Weap Scrapper - Iron Marauder - 50 Eng/Eng/Pow Blaster
Lion of Might - 50 SS/Inv/Eng Tanker - Darling Nikkee - 50 (+3) StJ/WP/Eng Brute - Ice Giant Kurg - 36 Ice/Storm Controller
Quote:
Actually, I do have enough knowledge and experience to suggest a solution. It's quite simple. Shift some resources from new features to fixing old ones that are broken. As far as I can tell, there are bugs and glitches across all departments - costumes, maps, missions, reward systems, etc, etc. There's something for everyone to fix.
|
Originally Posted by Wing_Leader
In other words, any opinion offered is only as insightful and valuable as the direct knowledge, skills, and experience that back it up. Saying "I see crap!" is not nearly as useful as knowing how to clean it up. Personally, I think we need far less of the former, and far more of the latter.
|
Paragon City Search And Rescue
The Mentor Project
Quote:
Forum goers have assured me that it's quite impossible for the company to make any changes of this nature because the company is already at max rate/capacity for bug fixing. also, forum goers seemed to be concerned that relegating more resources to bug fixing would mean that animators would be asked to work on database issues, so some salt might be advisable.
|
Originally Posted by Ironblade
Actually, I do have enough knowledge and experience to suggest a solution. It's quite simple. Shift some resources from new features to fixing old ones that are broken. As far as I can tell, there are bugs and glitches across all departments - costumes, maps, missions, reward systems, etc, etc. There's something for everyone to fix.
|
Quote:
Out of curiosity, what is your explanation for why Paragon has not implemented this painfully simple and obvious strategy? If it is so simple, surely they would have considered it themselves. If they rejected it, what could the possible reasons have been?
|
Originally Posted by Ironblade
Actually, I do have enough knowledge and experience to suggest a solution. It's quite simple. Shift some resources from new features to fixing old ones that are broken.
|
NOR-RAD - 50 Rad/Rad/Elec Defender - Nikki Stryker - 50 DM/SR/Weap Scrapper - Iron Marauder - 50 Eng/Eng/Pow Blaster
Lion of Might - 50 SS/Inv/Eng Tanker - Darling Nikkee - 50 (+3) StJ/WP/Eng Brute - Ice Giant Kurg - 36 Ice/Storm Controller
As a professional in the software development of financial software myself, I can see where the OP is coming from, but there is more to it than code. I happen work in a building that also has a big-name game making company in it. I get to talk to both kinds of developers and to the people that run the companies.
First, the financial code we create is FAR more 'focused', making it far easier to debug and maintain, IMO. The fact that many of the same technologies used in highly mission critical software, like programming languages and databases, are used to create CoH as well, does not mean they are used in the same way. In fact, many of the underlying technologies were developed to create financial software and have been shoe-horned into creating games. Often, just as it does with financial software, the bugs lie not in the code written by the product developer, but somewhere in the underlying technology.
Second, and more importantly, highly mission critical code has an enormous amount of money and resources thrown into the QA phases as compared to games, where the development resources are placed elsewhere, like art and storytelling, etc. The amount of money spent on QA in the highly critical software, IME, often outweighs by far the amount spent on the actual development.
Games are art, where highly critical software is engineering.
There are two completely different mindsets at work between the two types of software, regardless of the fact they happen to use the same tools, like a welder that builds cars vs one that builds replicas of Transformers.
I am NOT saying this game developers should not unit test like everyone else. I am not saying that QA should not be done. But QA is not where the game industry spends the majority of it's money. The margins they have are small enough as it is, games would not get made if held to the same standards as the PCI complaint software of the modern financial world, or they would cost a whole lot more.
Comparing game code development to highly critical code development is not a reasonable comparison, for a number of reasons; to entertain is art, to track how much money I have in the bank is something else entirely.
IMO, it's like telling an artist his work has to be held to the same scrutiny as a bridge builder.
"The side that is unhappy is not the side that the game was intended to make happy, or promised to make happy, or focused on making happy. The side that is unhappy is the side that is unhappy. That's all." - Arcanaville
"Surprised your guys' arteries haven't clogged with all that hatred yet." - Xzero45
You are allowed to play new content before it is technically released. They are in no way obligated to allow us to do that. In return for that privilege, it is understood that you will be telling the developers about any problems you encounter while doing so.
Playing unreleased content is not a right like some people want to claim. If you want to play unreleased content, hold up your end of the deal and report a bug when you find it. It takes, literally, less than a minute to file a bug report, but you don't want to be bothered because it's "not your job". Even though the devs are letting you play stuff that hasn't been released yet.
The dev team cannot possibly test the entire scope of things that several hundred individual players will play through. If they could, the AE exploits they squashed never would have made it to the live build in the first place. It's a simple fact that the more people you have testing something, the more problems with it you will uncover, because not everyone will do the exact same thing. The devs just don't have the kind of in-house manpower it would require to simulate hundreds of people playing, for that they have to actually get hundreds of people playing.
The devs could decide that they will do 100% of the beta testing in-house, and that they don't care that it will take 6 months instead of 2. By letting us in there, it drastically speeds up the process, and we get stuff faster than we otherwise would. And it also leads to regular players finding things that the devs might never stumble across. Do you really think it will be a dev finding a bug related to farming Battle Maiden for 6 hours? Not bloody likely. It will be a player finding that bug.
The devs will test the new stuff in-house, and that's probably about it. They rely on players testing the other 99% of the game to make sure no new bugs cropped up when they added the code for the new stuff.
Ironically, I remember some of the people in this thread saying they aren't obligated to report anything as being some of the same people complaining that they never got closed beta invites. Why would they give a beta invitation to someone who has freely admitted they have no intention of actually testing anything in that beta?
simple to detect had their been ANY testing at all.
The Fog of War is about as obvious as it gets. There is no way to NOT see
that one in the first 5 minutes of running a character, any character.
To be sure, it is not game-breaking, but the point I'm making is that it should
never have got out of Unit Testing.
How long do you delay the release of an issue while you search for the cause of something that isn't game breaking? A month? 3 months? A year? As long as necessary, even if it means the issue is never released?
As a programmer, can you say with any certainty that the borked code that's causing that bug is even IN the file the map is in? How do you know they haven't already looked everywhere and haven't figured out what's doing it yet?
Some things, like the map bug, just aren't major enough issues to warrant delaying an issue release while you straighten it out. Neither is the inspiration tray moving, or the majority of the other bugs you mentioned.
Now, if there were a bug that causes the deletion of a random character on someone's account when a specifc thing happened, THAT would warrant a delay. But not the Fog of War working incorrectly.
See, it's gems like these that make me check Claws' post history every once in a while to make sure I haven't missed anything good lately.