Do the Devs even test their code anymore?
By
all_hell,
Posted on: 2011-12-07 13:27 in Player Questions
Quote:
By pasta, I hate the "oh, my job gives me special insight" line. Here's a counter for you. I know from being alive that if you want to encourage a behavior, you give positive reinforcements (i.e., fixing bugs and responding to bug reports.) If you want to discourage behavior, you give negative reinforcement. The PS team have been doing far more of the latter.|
Originally Posted by VoodooGirl
I know from personal experience with my job, simply because you don't get a response or acknowledgement doesn't mean they aren't aware
 |
Quote:
|
Originally Posted by VoodooGirl
Would you rather the Devs sit and personally respond to each and every bug/feedback post - or - would you rather they spend their time gathering that information and work on fixing the problems and issues that are brought to their attention?
|
If, by some miracle, all the feedback isn't being ignored, some QA person (not a developer) should be checking the feedback and logging issues into PS' tracking system.
If you're going to say it's unreasonably onerous for that person to post a boilerplate "Logged as bug #736222," or "Duplicate of bug #288237," or "Unable to duplicate, please provide more details," then you've got awfully low expectations.
Me, I play a bunch of other MMOs. Pretty much no dev team excels at communication, but there are some that are way better than this.
Quote:
|
Originally Posted by VoodooGirl
Think about
 (No, really, think about it first for more than a second and then respond.) |
Quote:
|
Originally Posted by VoodooGirl
Of course we aren't, but we are paying customers, and can break things in way that seasoned programs wouldn't think to break them. It behooves Paragon Studios to forego paid beta testers and let the end users - us - beta test it and give our feedback.
|
Sadly, a 30-50 person beta isn't likely to do much good, so they fling open the doors to occasional filers and even people who just want to see the new shinies. Because even those people can unearth bugs.
But you can never lose sight of the fact that none of those people have any obligation to be testing and filing bugs.
And when Paragon consistently falls down at communicating and fixing bugs, it's even more ludicrous to be taking the stance that the problem is due to bad beta testers.
Quote:
I don't beta anymore. I did some before now, but I didn't enjoy it as much as developing my "real" toons. So I stopped and I haven't started back.|
Originally Posted by Wing_Leader
Being a beta tester isn't just a clever way to get free/early access to a game, it is an important part of the debugging cycle, and if you don't want to (or have the time to) be a part of that, then don't be a beta tester, because your contribution as nothing more than a bandwidth load on the server is minimal at best.
|
Though if some powerset I am interested in comes out, I may download the beta client and try it out to see if I want to buy it. Or, I may just wait for Mids' to update. idk. One or the other if something that interests me comes along.
Quote:
How wrong you are.
|
Originally Posted by Brillig
Sadly, a 30-50 person beta isn't likely to do much good...
|
Quote:
Have you even LOOKED at the beta feedback threads?|
Originally Posted by VoodooGirl
Maybe if people who go on the Beta server actually did beta testing and left forum feedback & bug notes instead of just looking at the shinies and then never logging on again, well, then maybe we wouldn't have so many bugs make it Live.
Just a thought, y'know. |
One example: since October, people saying the TPN Trial has too many gimmicks and most of the time nobody knows why they failed because there are too many things going on at once, besides things like civilians one-shotting you. Also, complaints about Maelstrom killing half the league when he felt like it. 1 month of feedback, mostly constructive, and completely ignored. Not even a 'Ok thanks but we'll keep it that way'.
Titan Weapons got a lot of feedback too, which actually helped since Synapse talked to us in the thread and changed some power names, animation for Arc of Destruction, but still, the set went live despite the numerous feedback about the lackluster sounds of the set (does smashing damage, has windy sounds unlike SS or Stone Melee) and the sounds are still a bit out of sync with the animations, these were reported NUMEROUS times.
Most bugs the OP posted were reported BEFORE they went live. If you are writing such nonsense you should go visit the Beta feedback threads, they're all there for you to see. There are tons of posts about bugs like 'I can't believe I reported this so many times and it went live like that' (not talking about things that aren't bugs like the TPN example, because a trial being complicated and gimmicky is not necessarily bad for everyone and if it needs adjusting they probably will do later).
You don't have to believe me, the thread about TW has almost 900 posts on it so no, lots of us didn't go there just to 'oooo lemme play with a level bumped TW toon' - http://boards.cityofheroes.com/showthread.php?t=276097
Even more 'superfluous' things like the Gunslinger costume pack had over 500 replies on beta.
EDIT: Weirdly, you were one of the first ones to post about the TPN Trial, I just saw. I don't know how you can think and post that beta players only go there for the shinies. So... Don't you find it weird that the thread has 100 posts and not a single answer from the devs, not even a 'thanks but we're happy the way it is'? Kinda discourages people from testing it, especially since you have to schedule to form a trial in Beta.
Quote:
Yeah, I'm having a hard time pinning the blame on the actions (or lack thereof) of beta testing players too. All I've ever heard from the devs is about "how much good info" they are receiving from the beta process. But I must admit I've seen multiple bugs reported in multiple formats reported very early on make it to live servers. Seems to me there is lots of room for improvement... and not so much on the player side of the equation.
|
Originally Posted by Brillig
And when Paragon consistently falls down at communicating and fixing bugs, it's even more ludicrous to be taking the stance that the problem is due to bad beta testers.
|
One man's terrorist is another man's freedom (or freem?) fighter; just as one man's exploit is another man's feature.
Nobody 'has' to beta test. It's an option and in my case a 'privilige' to be invited to do so. It's one of the ways the devs find out what is wrong/right with the game before it goes live. Because they already know about any given problem doesn't mean to say there is a guarantee it will be fixed before launch. We are given a list of known problems, which is updated as and when they arise which we all can familiarise ourselves with if we want to take the trouble. If we find a new problem, beta testing or not we should report it, if we want it fixed that is.
I really think some folks need to exercise some patience before getting all hot and bothered about things. I think the blame lies with the instant gratification society we live in these days, nobody wants to wait or even thinks they should wait for anything at all.
As long as a bug/issue isn't game-breaking I'm happy to play the game as it is bugs and all.
There is always a lot to be thankful for, if you take the time to look. For example, I'm sitting here thinking how nice it is that wrinkles don't hurt. ~Author Unknown
Quote:
Or perhaps from the other experiences in life where professionals put out well functioning products for sale and use.|
Originally Posted by Warbaby
I think the blame lies with the instant gratification society we live in these days, nobody wants to wait or even thinks they should wait for anything at all.
|
The rebuttal to that is that mmo games are a different class of objects where bugginess has to be accepted. But the rebuttal to that is that there's still a certain level of non-bugginess that's the standard for mmos.
I have never played another mmo. I have nothing to compare it with other than the other various services I purchase in my life. For most other things, this level of bugginess would be cause to but a different company's product. However, afaict, there's not another CoV for me to buy.
I don't think it's about waiting so much as expecting a certain level of performance (rightly or wrongly). Obviously, ymmv.
Quote:
We don't work for Paragon Studios.
|
Originally Posted by VoodooGirl
Maybe if people who go on the Beta server actually did beta testing and left forum feedback & bug notes instead of just looking at the shinies and then never logging on again, well, then maybe we wouldn't have so many bugs make it Live.
Just a thought, y'know. |
personally, I wont' stand for this injustice any longer. It's time the devs answer to the PLAYERS, not their own SELF SERVING AGENDA. How can they continually be so STUPID and ARROGANT! This game used to be GOOD, but they have DESTROYED IT.
That's why I'm voting for OP to become President of the entire company. It's about time we got a bug fixer who knows how to shout at the problem until it goes away; forget all of that MATH stuff.
Quote:
|
Originally Posted by Xanatos
We don't work for Paragon Studios.
|
Quote:
You mean the people playing with the mechanics and content that the devs have no obligation to allow them early access to? |
Originally Posted by Brillig
But you can never lose sight of the fact that none of those people have any obligation to be testing and filing bugs.
|
Sorry, that's kind of the trade off. If you want to have EARLY access to the new features of the game before they reach the live servers, they expect you to report any bugs you happen to find in return for the privilege of having that access.
Players don't have the right to access content early, it's a privilege we earn with our willingness to report stuff as we find it.
So, actually, you're dead wrong. The simple fact that they are letting us play with new content before it reaches the live servers means we ARE obligated to test that stuff.
The devs could very easily say: "Oh, you don't want to test stuff and report bugs you find? You don't need to be on the Beta server then, you can wait to play that stuff like everyone else. Have a nice day."
And they would have every right to do so.
Quote:
|
Originally Posted by Dechs Kaison See, it's gems like these that make me check Claws' post history every once in a while to make sure I haven't missed anything good lately. |
Quote:
For various reason I don't bother with beta.|
Originally Posted by ClawsandEffect
You mean the people playing with the mechanics and content that the devs have no obligation to allow them early access to?
Sorry, that's kind of the trade off. If you want to have EARLY access to the new features of the game before they reach the live servers, they expect you to report any bugs you happen to find in return for the privilege of having that access. Players don't have the right to access content early, it's a privilege we earn with our willingness to report stuff as we find it. So, actually, you're dead wrong. The simple fact that they are letting us play with new content before it reaches the live servers means we ARE obligated to test that stuff. The devs could very easily say: "Oh, you don't want to test stuff and report bugs you find? You don't need to be on the Beta server then, you can wait to play that stuff like everyone else. Have a nice day." And they would have every right to do so. |
However, there's a new wrinkle involved since we have gone to micro transactions. It's possible that there's a financial benefit to allowing us to try out new stuff before we buy.
So things aren't quite as simple as they used to be.
Quote:
It's not as noticeable when they have 6 months between releasing piles of crap to clean up the mess they made. Now that they weekly take a dump in my computer, I notice more.
|
Originally Posted by DarkGob
The devs have been pushing buggy issue releases/non-Issue game updates for pretty much the entire lifetime of the game, why would you expect anything different?
|
Quote:
When the bugs go live after being reported, it's obvious they did neither.
|
Originally Posted by VoodooGirl
Would you rather the Devs sit and personally respond to each and every bug/feedback post - or - would you rather they spend their time gathering that information and work on fixing the problems and issues that are brought to their attention?
|
I think I must be in the wrong game forums because apparently the game most of you guys are playing is Buggycrap. Can I recommend you try City of Heroes instead? This last week I've played it more than I normally do, doing my first Underground Trial, quite a few Keyes and one or two story base arcs. I also created new characters using totally new abilities and newly purchased costume pieces. All without any problem. I have recommended it to most of my friends.
This is a song about a super hero named Tony. Its called Tony's theme.
Jagged Reged: 23/01/04
Quote:
Would you rather wait 6 months for something new because they haven't tracked down exactly which piece of 7 years worth of code is causing it to act weird? |
Originally Posted by rsclark
When the bugs go live after being reported, it's obvious they did neither.
|
It's not like a red arrow pops up pointing at the piece of code causing the problem when a bug is reported.
A bug gets reported and then they have to manually track down where the bug is coming from, which could be in a piece of code that has no apparent connection to the resulting problem.
It's not as simple as "Oh, a bug! Flip that switch and it'll be fine."
The "civilians walking on the ceiling" bug that existed for 3 years is a good example. Did it occur to you that it may have actually taken them that entire 3 years to FIND the cause of that bug? I imagine that, after 7 years, there is a hell of a lot of code to sift through to find things like that.
And it could be a case of one piece of code reacting strangely when combined with another, but individually they're fine. In fact, I'm sure that happens with quite a few of the persistent bugs.
How do you figure out what's doing it when all the code appears to be working right? You go through, one by one, and combine that code with other code to see if it produces the same result.
Just because they haven't fixed something RIGHT NOW, does not, in any way shape or form, mean they aren't doing anything about it. I'd be willing to bet there are at least a couple people employed at Paragon Studios whose SOLE job is to sift through all that code looking for the bugs that get reported. Their job has to suck royally, especially when you consider that fixing a bug quite frequently creates a new one.
Quote:
|
Originally Posted by Dechs Kaison See, it's gems like these that make me check Claws' post history every once in a while to make sure I haven't missed anything good lately. |
Quote:
Out of the of darkness of chaos comes a voice of sanity - well said!
|
Originally Posted by Jagged
I think I must be in the wrong game forums because apparently the game most of you guys are playing is Buggycrap. Can I recommend you try City of Heroes instead? This last week I've played it more than I normally do, doing my first Underground Trial, quite a few Keyes and one or two story base arcs. I also created new characters using totally new abilities and newly purchased costume pieces. All without any problem. I have recommended it to most of my friends.
|
There is always a lot to be thankful for, if you take the time to look. For example, I'm sitting here thinking how nice it is that wrinkles don't hurt. ~Author Unknown
Quote:
Or, it takes a little longer than you think to actually track the cause of the bug down in the huge pile of code that has continuously grown over the last 7 years.|
Originally Posted by rsclark
When the bugs go live after being reported, it's obvious they did neither.
|
Finding one mistyped variable is not as easy as you'd think.
There I was between a rock and a hard place. Then I thought, "What am I doing on this side of the rock?"
Puppies....
Puppies!
Quote:
|
Originally Posted by VoodooGirl
Maybe if people who go on the Beta server actually did beta testing and left forum feedback & bug notes instead of just looking at the shinies and then never logging on again, well, then maybe we wouldn't have so many bugs make it Live.
Just a thought, y'know. |
Huh?
This is something I've never quite understood. The gaming industry is so ...unlike any other industry.
A business makes a product. But, instead of the makers ensuring that the product is quality work, they put it in this Beta test phase, where unknown gamers are asked to test it. I could understand if the the testers were software developers with knowledge of terms like "look and feel" and "intuitive". (in all fairness, some are quite knowledgeable)
And then, some folks report back errors. Some are fixed, some are not.
The thing is, you can't chide people for not adequately testing things in beta.
It is not any players job to test things out in beta. No one gets paid a thing.
I read the (philisophical)argument over and over. "If you're not going to beta test, then don't complain about the bugs!"
It is flawed from the original premise. I am a customer who pays for goods and services. It is not my job to ensure that these service work properly.
That is the job of game developers. They are paid for this. We are not.
Times are tough, and time is our most precious resource. If I am to spend time on Beta, you will need to pay me for that time. It really is as simple as that.
The counter argument - "Well, you'd better be prepared to pay more each month if you want that kind of quality."
Huh? The market place has plenty of other things I could spend my money on. They either want my business, or they do not. I think they do.
That being said, I think this game tries very hard to put out a quality product. And, I am also thinking that when things do go wrong, they do what they can to fix it. However, even though many of us have some experience in code, or in Software Analysis, or IT, none of us are really qualified to speak on any issue regarding the challenges the dev team face with their code except them. We don't know all the facts and any comments made comparing our own respective experiences with what they're doing is pure speculation.
Myself, I'd like it if they raised the monthly fee to about 25 bucks a month and put that money not into useless costumes or powersets but to increase in game quality of life issues. Like a portal, and/or train to RWZ, Cimerora and the other end of Independance Port.
Revamp task forces that require excessive zoning in and out of the contacts zone.
Make all mission maps within 500 yards of the contact that gives the mission.
Shorten the timer of the temp tp powers back to what it was.
Etc.
"Most people that have no idea what they are doing have no idea that they don't know what they are doing." - John Cleese
@Ukase
Quote:
I have no special inside insight about the process here, but I'm nearly positive that unless it's a bug so severe that it will kill game play entirely, nothing reported in Beta is going to be fixed before it goes live. |
Originally Posted by Fire_Away
Yeah, I'm having a hard time pinning the blame on the actions (or lack thereof) of beta testing players too. All I've ever heard from the devs is about "how much good info" they are receiving from the beta process. But I must admit I've seen multiple bugs reported in multiple formats reported very early on make it to live servers. Seems to me there is lots of room for improvement... and not so much on the player side of the equation.
|
Instead, I think Beta is the opportunity for us to QA what's going to go live so they can determine any items that need to be patched at a later date. There are exceptions, of course, like Titan Weapons (there was quite a bit of back and forth feedback and many changes actually happened) but stuff like the inspiration tray moving around is just not important enough for them to prioritize ahead of pushing code to live servers.
Save Paragon one more time! http://www.cohtitan.com/forum/index....ic,4877.0.html
Petition to end shutting down CoH:
http://www.change.org/petitions/ncso...city-of-heroes
Quote:
If it's a bug that any handful of players might enjoy, it's redflagged as an "exploit" and given highest priority for the fix list. If the players don't gain anything in any evil form as a product of the bug it can be fixed after every exploit and trivial thing deemed exploit, are patched|
Originally Posted by Siolfir
There are a couple of longstanding "bugs" that I'm hoping don't get fixed, although I was told that one of them is likely fixed already internally.
 |
.(Yes I'm mad they finally found and fixed my market-in-base trick
)
10 50's To Date! Check out Titan Sentinel; it got my CoH presence synced online
Some very interesting commentary in here - I appreciate all the responses.
There are a couple points that I want to speak to, and I'm trying to be as non-acerbic
as possible although this issue bothers me a lot on both a customer level and
on a professional level.
I have been an applications programmer (primarily in the corporate financial
sector) with companies (who shall remain unnamed, but would be definitely recognized)
for ~34 years now (longer, I'd wager than many of our posters have been alive).
I'm not saying that to brag, but merely to point out that I'm quite familiar with
software, design/development paradigms, and large company software production.
I think the thing that is bothering me most lately is NOT that there are bugs,
that is truly inevitable in any sufficiently complex application, which CoH definitely
qualifies as.
It is rather, the kinds of bugs we're seeing lately.
The kinds of bugs lately, have been very simple, very obvious, and extremely
simple to detect had their been ANY testing at all.
The Fog of War is about as obvious as it gets. There is no way to NOT see
that one in the first 5 minutes of running a character, any character.
To be sure, it is not game-breaking, but the point I'm making is that it should
never have got out of Unit Testing.
Let's try the age-old car analogy.
There have been many car recalls over the years for some rather dangerous
issues - but how many new cars come off the lot with, say non functional headlamps,
or missing sections of paint, or missing mirrors, or glove boxes, doors and trunks
that don't open or close properly?
Can you drive the car with those problems? Probably.
Would you BUY a new car with those problems? I wouldn't.
If X Motors invited you to test drive the latest model (before it was available
to the public) and you saw these things, would you reasonably expect them
to be fixed before being sold to the public?
This is the caliber of "quality" we're seeing lately, and there's no other way
for me to categorize it as anything other than substandard and shoddy.
It reflects an attitude that is uncaring about either the quality of the product,
or the customer that is using and paying for it.
Getting back to software.
What would your opinion be if these issues were occurring with an ATM, or your
checking account?
What if this software was written for aircraft flight control and management?
I'm on record (in prior post) as saying I'm glad these devs work in the gaming
and entertainment industry, because quite frankly, if the lack of testing and
quality I see here was carried over to other industries, they'd very possibly
cost their companies millions in fines/penalties, or in the aircraft case, wind up
killing somebody.
These bugs aren't "critical", but they're simple, obvious, and should never have
left the coder's desk in the first place, let alone ended up in a live release.
On top of it, they certainly raise the question in my mind regarding what worse
bugs are lurking in the software, given that they can't be bothered to fix the
simple and obvious ones. Not exactly confidence inspiring in my book.
As a paying customer, I am calling for a higher quality product.
As a long-time programmer, I question their procedures.
As a player, I'm grateful to be invited to a beta, and I try to contribute useful
information, but I also have the expectation that it matters, and raised issues
will be addressed in something resembling a timely matter.
Once again, thanks all for your commentary. Obviously, it's a topic I feel strongly
about, and I'm very interested to see how the community views it as well.
Regards,
4
PS> I like cute puppies too!
I've been rich, and I've been poor. Rich is definitely better.
Light is faster than sound - that's why some people look smart until they speak.
For every seller who leaves the market dirty stinkin' rich,
there's a buyer who leaves the market dirty stinkin' IOed. - Obitus.
I love this game, and have played it off and on since release. Overall they have put together a great game and keep things running fairly well.
Having said that, their QA/bug report system is highly ineffectual. Gamebreakers they're pretty consistent with, but mid-range bugs can go unchecked for months (at least), and I'm quite sure that minor bugs/text errors submitted in-game go straight to the circular file. As I'm levelling through old content (I never powerlevel) I sometimes read mission text that I haven't read in a while just to help immerse myself in the setting. To this day I see text errors that I reported years ago.
Just a thought, y'know.
Did you miss the LEVEL BUMP thread? Y'know, the one where you can make a new toon and request to get level bumped to whatever level you want?