Do the Devs even test their code anymore?
By
all_hell,
Posted on: 2011-12-07 13:27 in Player Questions
I don't have anything non-inflammatory to say so please enjoy this picture of puppies.
Not sure how many of these were detected on the test/beta shards, but the map issue?
Not a show stopper.
Inspiration Tray issue? Hasn't touched me yet, but I leave mine docked - it only shows up if you disconnect the tray.
I have not noticed any "complete UI reset" on any of my characters.
Did someone run SSA3 while it was on test, redside? Don't know what the issue is (or are you referring to the [now fixed, IIRC] issue with the PGA that wouldn't despawn? Note I haven't run any of the SSAs redside yet, so that could of been an SSA2 issue).
And other than perhaps the SSA issue, none of these are really gamebreakers. I think they've even said some are already fixed. Do you really think they purposely did something to cause the Fog of War to reset? They might not even of touched the maps at all - it's just how coding in a 7+ year old project can go.
And as for the Forum maintenance - the cancel last week was because it was either "Do a schedule forum fix/upgrade" or "Do emergency server upgrade" as they are the same team. Guess what? If CoH had something that required the web team basically shun the other games when one of the others had a forum issue, i'd hope they'd make the LIVE game a priority.
Orc&Pie No.53230 There is an orc, and somehow, he got a pie. And you are hungry.
www.repeat-offenders.net
Negaduck: I see you found the crumb. I knew you'd never notice the huge flag.
My guess is that the devs test changes in an environment that doesn't exactly mirror the training room or the beta server, and then they don't go through the entire QA process over again on the training room once they release it to the users for testing. It seems that once it hits the test server, it's going to go live no matter what, so there's really no point in them doing additional QA there.
Realize that the devs are working with thousands of changes that are targeted for many different releases. They were working on I22 and I23 and I21.5 and bug fixes to I21, all at the same time. There are probably different development and maintenance teams working on the same code base, and sometimes they may even have to change the very same files. Making sure the right changes get into a particular build can be very challenging.
This process of "configuration management" gets even messier when changes in different releases interact with each other in unforeseen ways. If someone makes a change to the I21 code base that depends on a change in the I22 code base, that change may not work properly when it finally gets released in I21.5, which won't include the I22 change it depends on.
Is that the reason for these problems? I can't say for sure, but in my experience with maintaining complex systems that have multiple releases these kinds of bugs are common.
The devs have highly skilled technicians checking the code for errors and bugs.
Just kidding.
The devs have been pushing buggy issue releases/non-Issue game updates for pretty much the entire lifetime of the game, why would you expect anything different?
"You don't lose levels. You don't have equipment to wear out, repair, or lose, or that anyone can steal from you. About the only thing lighter than debt they could do is have an NPC walk by, point and laugh before you can go to the hospital or base." -Memphis_Bill
We will honor the past, and fight to the last, it will be a good way to die...
Quote:
|
Originally Posted by Forbin_Project
The devs have highly skilled technicians checking the code for errors and bugs.
|
I'm not sure if I should be more worried that I am apparently moonlighting as a "skilled technician" without realizing it, or that you have acquired a picture of me at a computer.
Quote:
Yes, but the higher release rate means that the bugs compound on themselves more often; at least before they took so long to push the next set of bugs out that you could get used to the old ones.|
Originally Posted by DarkGob
The devs have been pushing buggy issue releases/non-Issue game updates for pretty much the entire lifetime of the game, why would you expect anything different?
|
But the terminology used so far is wrong, though: it's only a bug if it helps the players - if it hurts/annoys them, it's a "new feature"! There are a couple of longstanding "bugs" that I'm hoping don't get fixed, although I was told that one of them is likely fixed already internally.
Quote:
|
Originally Posted by PleaseRecycle
it has gone from unconscionable to downright appalling that we have no way of measuring our characters' wetness.
|
Quote:
|
Originally Posted by Brillig
It's hard to beat the entertainment value of Whackjob Wednesdays.
|
I think the devs tend to stick with mending game-breaking issues before they fix QOL ones. I remember the bug where NPC's walked on the ceiling in City Hall Atlas was there for at least three years. Yeah it's annoying that some things seem to take an age to fix, but eventually most things do. This MMOG is no worse or better than most others out there.
Get yourself a stiff whiskey, put your feet up and chill. Life's way too short to get all of a dither over trifles FourSpeed. 
There is always a lot to be thankful for, if you take the time to look. For example, I'm sitting here thinking how nice it is that wrinkles don't hurt. ~Author Unknown
Maybe if people who go on the Beta server actually did beta testing and left forum feedback & bug notes instead of just looking at the shinies and then never logging on again, well, then maybe we wouldn't have so many bugs make it Live.
Just a thought, y'know.
Quote:
Oh please. There are numerous instances of bugs that have been reported in Beta and still made it live. And honestly, I don't think the total silence from Paragon on most bug threads is at all helpful. If people feel their feedback is falling on deaf ears, they're going to be less inclined to give feedback in the future.|
Originally Posted by VoodooGirl
Maybe if people who go on the Beta server actually did beta testing and left forum feedback & bug notes instead of just looking at the shinies and then never logging on again, well, then maybe we wouldn't have so many bugs make it Live.
Just a thought, y'know. |
Finally, players are not QA. Beta tests exist in order to get feedback and bug reports, true, but depending on a beta to do your testing for you is the acme of arrogance and futility.
Quote:
Except that many bugs DO get reported, multiple times, with multiple methods (/bug submissions, Beta forum threads, PMs directly to devs), and those bugs still make it to live.|
Originally Posted by VoodooGirl
Maybe if people who go on the Beta server actually did beta testing and left forum feedback & bug notes instead of just looking at the shinies and then never logging on again, well, then maybe we wouldn't have so many bugs make it Live.
Just a thought, y'know. |
Just a thought, y'know.
"You don't lose levels. You don't have equipment to wear out, repair, or lose, or that anyone can steal from you. About the only thing lighter than debt they could do is have an NPC walk by, point and laugh before you can go to the hospital or base." -Memphis_Bill
We will honor the past, and fight to the last, it will be a good way to die...
Quote:
PUPPIES!|
Originally Posted by Frostbiter
I don't have anything non-inflammatory to say so please enjoy this picture of puppies.
|
Aaaw, so cute.
I forget why I was even in this thread.
Character index
Quote:
Don't exaggerate. You either have a short memory, or were not around for the issue 7 release.
|
Originally Posted by FourSpeed
The I21.5 release had more bugs than I can recall in any single release over
the history of the game. |
SadysCHICK ALL the Badges! (I can get. 1396)
Full image by David Nakayama
Arc ID 1435: Performing without Annette
Arc ID 7206: Sadystic Tendencies
Arc ID 3864: The Chronicles of (In)FERNIA!
Two words were drilled into my head in my programming classes.
Unit Testing
Your character does not have capped defense. Depending on your AT the cap is between 175% - 225%. Your defense is not teal in the combat window, it can go higher. STOP SAYING IT IS CAPPED! The correct term is Soft Cap.
I enjoy playing in Mids. I specialize in Melee Characters, other AT's usually bore me.
Quote:
That is just total crap. The first time I used Beta, back when dual blades hit, I copied the files over, because I didnt want to have to download it all again. It worked fine, but when I logged on, I didnt see ANY big notice about doing proper testing and reporting back to the devs. There was no big handy in game survey to fill out. Maybe there is now, I dont know.|
Originally Posted by VoodooGirl
Maybe if people who go on the Beta server actually did beta testing and left forum feedback & bug notes instead of just looking at the shinies and then never logging on again, well, then maybe we wouldn't have so many bugs make it Live.
Just a thought, y'know. |
The reason I don't know? Somehow, the Beta install on my comp broke the Live version..so I had to download it all again. So I said stuff it. Oh you you know another reason why I, and I dare say quite a lot of other people, do not go onto Test and report back?
THE DEVS DONT FIX STUFF.
Yes, that is right! Case in point, the guy who reported the villain base bug, last patch, in Beta. It still went LIVE! SO how does this hold up by your logic, or lack of it? He found the bug, reported it, they did..oh wait, nothing. Now, granted, maybe it was super hard to fix, totally justifying them taking..3? weeks. But seriously, they couldnt just disable base defences, which serve no purpose anyway? wtf?
Or what about the old snake farm bug back in mercy? I saw a thread by a guy who found and reported that, months before..yet again..pushed Live.
If the devs are really using Beta as a way to catch bugs that their testing department misses, big obvious ones like the bases, then I think their testing guys need to wake up and get to actual testing.
Also, in case you havent realised, not everyone has the time to spend hours and hours on Test, playing with the new stuff, which then has to be done AGAIN when it goes Live. I am sure some people enjoy that, just as I bet a lot don't.
Claiming that more bugs would be fixed if more people found and reported them is just silly. Especially in the face of recent evidence. We are not employees of NC, we are not qualified testers, we are PLAYERS, who often have hugely differing ideas on what is even a bug in the first place.
Beta is, and should be, a place where people can look at the current new stuff, fiddle with it a bit, then give some sort of feedback. It should not be somewhere we go, unpaid, unthanked, to TEST a product we pay (sort of) for.
Found the problem.
Quote:
I'd be glad to do it if it were my job. But since it ain't, I'll spend my play time playing instead of doing unpaid QA.
|
Originally Posted by VoodooGirl
Maybe if people who go on the Beta server actually did beta testing and left forum feedback & bug notes instead of just looking at the shinies and then never logging on again, well, then maybe we wouldn't have so many bugs make it Live.
Just a thought, y'know. |
It's all because of people who use line breaks in their posts...
The best comics are still 10�!
My City of Heroes Blog Freedom Feature Article: "Going Rageless?"
If you only read one guide this year, make it this one.
Super Reflexes: the Golden Fox of power sets!
WARNING: I bold names.
Don't worry, I'm here for folks to angrily PM.
There are no words for what this community, and the friends I have made here mean to me. Please know that I care for all of you, yes, even you. If you Twitter, I'm MrThan. If you're Unleashed, I'm dumps. I'll try and get registered on the Titan Forums as well. Peace, and thanks for the best nine years anyone could ever ask for.
Quote:
No it isn't|
Originally Posted by MisterD
That is just total crap.
|
Quote:
| The first time I used Beta, back when dual blades hit, |
Quote:
| I didnt see ANY big notice about doing proper testing and reporting back to the devs. There was no big handy in game survey to fill out. Maybe there is now, I dont know. |
Quote:
| Beta is, and should be, a place where people can look at the current new stuff, fiddle with it a bit, then give some sort of feedback. It should not be somewhere we go, unpaid, unthanked, to TEST a product we pay (sort of) for. |
Every MMO uses Test and Beta severs to get stress testing of new content done and they always invite a group of players to try and break the content and find out where unexpected bugs and other problem are.
Quote:
I know from personal experience with my job, simply because you don't get a response or acknowledgement doesn't mean they aren't aware |
Originally Posted by Brillig
And honestly, I don't think the total silence from Paragon on most bug threads is at all helpful. If people feel their feedback is falling on deaf ears, they're going to be less inclined to give feedback in the future.
|
Would you rather the Devs sit and personally respond to each and every bug/feedback post - or - would you rather they spend their time gathering that information and work on fixing the problems and issues that are brought to their attention?
Think about
(No, really, think about it first for more than a second and then respond.)Quote:
|
Originally Posted by Brillig
Finally, players are not QA.
|
Quote:
|
Originally Posted by DarkGob
Except that many bugs DO get reported, multiple times, with multiple methods (/bug submissions, Beta forum threads, PMs directly to devs), and those bugs still make it to live.
Just a thought, y'know. |
Quote:
|
Originally Posted by Forbin_Project
Every MMO uses Test and Beta severs to get stress testing of new content done and they always invite a group of players to try and break the content and find out where unexpected bugs and other problem are.
|
Quote:
Completely understandable.|
Originally Posted by all_hell
I'd be glad to do it if it were my job. But since it ain't, I'll spend my play time playing instead of doing unpaid QA.
|
On the other hand, if nobody reports bugs during open beta testing, then how are the devs ever supposed to know about them? And without detailed information to go on, how are they supposed to fix them in a timely manner?
There are whole categories of interaction bugs that simply don't arise until the system is tested under the stresses of a full-sized player base pouding away at the game simultaneously (equivalent, at least, to a typical live server environment). When these bugs do occur, they need to be reported, and described in sufficient detail to facilitate proper debugging. Beta testers are certainly under no legal or ethical obligation to participate in this process, but without a good number of them willing to do so anyway, there will remain many bugs that never get fixed, simply because they can't be properly isolated, identified, and characterized.
Part of the implicit understanding between developer and beta tester is that a trade of sorts is occurring: early access to the game in exchange for useful feedback, including meaningful bug reports. A developer who ignores fixable bugs is not holding up their end of the bargain (to say nothing of their obligation to paying customers), while a beta tester who refuses to engage in bug reporting is not holding up theirs. Being a beta tester isn't just a clever way to get free/early access to a game, it is an important part of the debugging cycle, and if you don't want to (or have the time to) be a part of that, then don't be a beta tester, because your contribution as nothing more than a bandwidth load on the server is minimal at best.
NOR-RAD - 50 Rad/Rad/Elec Defender - Nikki Stryker - 50 DM/SR/Weap Scrapper - Iron Marauder - 50 Eng/Eng/Pow Blaster
Lion of Might - 50 SS/Inv/Eng Tanker - Darling Nikkee - 50 (+3) StJ/WP/Eng Brute - Ice Giant Kurg - 36 Ice/Storm Controller
Quote:
Bogus. You have to work to avoid finding out about the test (and now beta) servers as well as how to get on them WITHOUT finding out about the corresponding FEEDBACK and BUG REPORT forums.|
Originally Posted by MisterD
There was no big handy in game survey to fill out. Maybe there is now, I dont know.
|
That's like the people saying if they never agreeed to the EULA by using the -ProjectCOH flag to avoid clicking "I agree" they can't be held to it in game.
Quote:
|
THE DEVS DONT FIX STUFF. Yes, that is right! Case in point, the guy who reported the villain base bug, last patch, in Beta. It still went LIVE! SO how does this hold up by your logic, or lack of it? He found the bug, reported it, they did..oh wait, nothing. Now, granted, maybe it was super hard to fix, totally justifying them taking..3? weeks. But seriously, they couldnt just disable base defences, which serve no purpose anyway? wtf? |
Quote:
| Players and their Base Items have reconciled their differences. As a result, defensive base items will no longer attack friendly players. |
The only other easy-to-throw toggle would be to shut off the base server entirely.
Quote:
| Or what about the old snake farm bug back in mercy? I saw a thread by a guy who found and reported that, months before..yet again..pushed Live. |
Quote:
|
If the devs are really using Beta as a way to catch bugs that their testing department misses, big obvious ones like the bases, then I think their testing guys need to wake up and get to actual testing. Also, in case you havent realised, not everyone has the time to spend hours and hours on Test, playing with the new stuff, which then has to be done AGAIN when it goes Live. I am sure some people enjoy that, just as I bet a lot don't. Claiming that more bugs would be fixed if more people found and reported them is just silly. Especially in the face of recent evidence. We are not employees of NC, we are not qualified testers, we are PLAYERS, who often have hugely differing ideas on what is even a bug in the first place. |
And there are other players that quite simply have twisty minds and find really bizarre ways of breaking things.
Quote:
| Beta is, and should be, a place where people can look at the current new stuff, fiddle with it a bit, then give some sort of feedback. It should not be somewhere we go, unpaid, unthanked, to TEST a product we pay (sort of) for. |
Orc&Pie No.53230 There is an orc, and somehow, he got a pie. And you are hungry.
www.repeat-offenders.net
Negaduck: I see you found the crumb. I knew you'd never notice the huge flag.
Quote:
Maybe if it was even POSSIBLE for me to get on the beta or test servers. The character copy tool has been broken for months.|
Originally Posted by VoodooGirl
Maybe if people who go on the Beta server actually did beta testing and left forum feedback & bug notes instead of just looking at the shinies and then never logging on again, well, then maybe we wouldn't have so many bugs make it Live.
Just a thought, y'know. |
Just a thought, y'know.
Paragon City Search And Rescue
The Mentor Project
Seriously.
Several of last few patches have seen Release Day emergency fixes.
The I21.5 release had more bugs than I can recall in any single release over
the history of the game.
But, I'm not just talking about quantity of bugs. I'm talking about simple, and obvious
stuff that should never even get out the door in the first place.
For instance:
How did they NOT detect the fact that SSA3 couldn't be completed red-side?
How could the guys coding this stuff NOT notice that the primary User
Interface was completely reset ... on every character?
How did they NOT notice that the inspiration tray moves - every single time
you zone on every single character?
How did they NOT notice that the Map went dark on characters who had fully
explored those zones, and that it was clear on characters that had never ever
been there?
How does this crap keep getting released into a live production environment
all the time?
Even worse, how does it continue to happen in cases where these bugs are
seen and reported in Live Beta weeks before actual release (fully discounting
the fact that a lot of this stuff shouldn't have even left the programmers local
build in the first place)?
How in Hades can the Production Release Manager even begin to think these
things are purely minor inconveniences that are still acceptable for live release?
Oh, and lest folks think I'm being overly critical of just the Game Devs, the Web guys
are just as bad, and maybe even worse imho (given the relative complexities
of the game vs their web pages).
The logout bug has been with us for more than a YEAR.
The CSS layout hero side has been borked for more than a month.
The Forum Maintence notice has broken/missing graphics links (at least, it did Monday).
Forum maintenance has been postponed twice in recent weeks and fully rolled
back once.
Do I even need to point out the various billing and accounting issues since Freedom
went live?
In short, during the past year we have seen a serious degradation of quality
in just about every facet of the custormer-facing operation we care to examine closely.
Is this what we're to continue to expect in terms of quality software in the
F2P model?
Any given issue item could be discounted - crap *does* happen in IT (I've seen
a lot of it in my 30+ years in the industry).
But this? This isn't a case of a few odd isolated issues - this is an ongoing trend
that is frequent, highly visible to the customers, and quite honestly, very disruptive
to the game experience. It adversly affects the overall consumer confidence that
we're receiving (and paying for) a high-quality product, and reflects quite poorly
on the game (and by association, the companies involved).
Would someone kindly explain to me why I should continue to pay for this product,
and what exactly is being done to reverse the trend of crap software we've
been enduring lately?
Regards,
4
I've been rich, and I've been poor. Rich is definitely better.
Light is faster than sound - that's why some people look smart until they speak.
For every seller who leaves the market dirty stinkin' rich,
there's a buyer who leaves the market dirty stinkin' IOed. - Obitus.