the rerturn of the daily maintenance?

Honestly, how much are you willing to bet on that one. Moving downtime *forward* just a couple of hours will actually inconvenience the EU playerbase quite badly, in this case spiting many for the sake of a few.

I have nothing against Australians and New Zealanders, i actually know a few and i find them to be loads of fun to be around. And generally speaking, those Oceanic players actually found the "downtime" a good time to, you know, spend time with the wife/gf/get something to eat*... it was far easier for them to do this as it was "set in stone" as to when the downtime was... and so could more easily schedule around it.

Of course, I am sure that you would still be willing to have the downtime scheduled for when the Americans are asleep and the developers have finished their work day, but still have to stay in the office (on patch day) just to please the few players who said "but you promised us that you wouldnt bring the servers down in OUR peak time this month"

But the downtime has *always* been scheduled at a time where "player concurrency" is always at its lowest, and also at a time where the developers are at hand (at least for CoX... other MMO's may differ) just in case anything happens... especially for patch days.


*actually they did complain when the downtime went on for far far longer than expected, but then again, everyone else also complained about it as well.

EDIT:
If i remember correctly, when CoX was released EU side, a lot of players complaining about the downtime being in their "peak time".... during the daytime hours... (and not in the evening). This was the ideal time to go and eat/drink/do some shopping.

Guess you cannot please everyone all of the time

Sigh no one is suggesting moving the entire shedule foward a couple of hours but staggering so you only get inconvenienced occassionally.

The comment about using the downtime to spend time with families is just plain insulting. If we were that sort of loser who did nothing but play 24/7 we wouldn't be worried about losing the couple of hours in the evening we do set aside for it.

And make up your mind. First you say moving it would greatly inconvience you then you straight away say its a good thing for us to have as we can use it to get a life. So why not have that usefull break yourself then? If its such a good thing that is.

Yet again I will point out that we are not insisting it be done out of business hours. And even if we were it would not be just out of goodwill. They are making money out of us so in essence we would be paying for it.

Why are so many of you against this? Do I take it you would rather we get totally shafted so that you dont even get minorly inconvenienced? Great brotherly spirit there guys.

Who said I was against it? I haven't said Word 1 opining that they shouldn't do it; I'm just speculating, based on my experience in a similar industry, as to one reason why they aren't doing it. As you keep pointing out, you're paying the same as everyone else; but chances are it would cost the company more to run the maintenance windows when it would be most convenient for you than it does now, because that would tend to be at fairly odd hours by North American standards, and operations people don't work odd hours for their base pay rate.

TLDR: Grits are extra.

It is unrealistic to expect a dynamic downtime. I am okay with the servers being down in the AM even if that is the usual time that I play. In fact, I would much rather have it down in the AM the same time every week if it means I dont need to consult a calendar and figure out which week, and what time it is down now. I Play At The Same Time As You and I Dont Feel Shafted Brother In Spirit. As I said earlier, don't take it personally. It aint a NA vrs OZ thing, so stop making it out as such.

It aint like you need to wait 6 months for a new issue to come out after it was released in NA like you have to do with movies. Have a pint of Bundaberg and relax mate.

This is true only on an INDIVIDUAL basis. But NCSoft isn't looking at it on an individual basis; they're looking at a 'customer base'. And, COLLECTIVELY, people in your time slot are NOT paying as much as people in U.S. 'prime time' hours.


While you may disagree with their scheduling, and the reasoning behind it, that doesn't make it an invalid argument.


ahh.... But is that true? Do they stand to lose more subs by regularly hosing people in one time slot than by occasionally hosing a much larger number in another time slot? I think you underestimate how many people would overreact and ragequit if they started putting the shutdowns in prime time even occasionally. Or I could be overestimating. This is one where we (NCSoft) can only find out the hard way.

Yeah, that's been in the EULA since day one.

Agreed we aren't, however since when is it ever good for a company to piss off ANY segment of their customer base if they don't have to? Spreading the load (so to speak) changes that through from one (mostly) pissed off segment to a few mildly inconvenienced segments. Overall better for the business wouldn't you agree?

To me it does for the following reason. Doing things purely because its the cheapest method does not equate to good busiess. Part of being in business often means spending a little money to keep your customers happy and continuing to use you. I'm sure you will agree that any company that does everything the cheapest way possible will not end up lasting long.


You are probably right that there would be some groaning but I seriously doubt they would lose one sub over it. After all at the very worst we are talking about other people having to deal with one extra shutdown every month and no one is going to quit over losing just two hours a month. That being said I accept that no one really knows the answer to that even NCSoft. What I can guarentee though is that they will lose subs if they continue down their current path.

Oh please. Schedules like this can be worked out months in advance and the people doing this work are far from idiots and should have zero problems working around that. Thats a lame excuse.

There is also the information screen in the launcher which can easily keep the player base informed so no one gets confused.


Kinda pointless since for the majority the maintenance/patches etc. never occur when they play so why would they care when it happens. So how is it a benefit? Certainly not to those of us actually affected by it. And for the record except for the thursday night weekly maintenance we almost never have more than a few hours warning for the other shutdowns so it impossible to shedule around them.


Yep, know thats there. And why should that stop us asking to have more consideration shown to us? If you read any EULA of any software it basically absolves the company of virtually anything. Does that absolve any of these companies from providing the same level of service amongst all its customers?

Anyhow I'm not here to justify our concerns to other players. Especially when they have a vested interest in keeping things as they are. I'm just pointing out to the Devs the feelings of some of their playerbase. Obviously I dont know and can't speak for every Australian player but for the ones I have spoken with every single one has the same concerns to varying degrees.

When they switched to the one day a week maintenance schedule we had servers located on the West Coast and servers located on the East Coast. It is entirely possible that the two locations were owned by different companies and that one or both of those companies balked at a varying schedule. Even if they were owned by the same company, it's possible that that company balked at a changing schedule. We don't know all of the details because they didn't tell us all of the details, nor did they have to. They gave us what information we needed to know about the schedule change and that they had considered other options. They didn't even have to tell us that, but they did.

I don't know if the weekly maintenance times have been re-thought since all of the servers are now in one location. I don't know who the company hosting the server farms is, but they may also not be willing to work on a rotating schedule for maintenance since it would impact them scheduling work on other companies servers they maintain, just like the previous server hosting companies.

As for the screen in the launcher being used for posting the schedule, there were many players that never saw that information with the old launcher since it relied on Internet Explorer to display. Some had IE issues on their computers. Some had intentionally disabled or uninstalled IE, so they couldn't see those updates. Many never bothered to read the information presented there. A good recent example of this was the discontinuance of the old launcher. The information was there for quite some time, but several people were clueless when the old launcher stopped working.

You said earlier in the post that you wanted to hear from a Dev about this. I'm not a Dev, but I can tell you what I remember them saying from when this change was made. I never made any claims as to whether or not I agreed with their statements, I just repeated them.



Now, as for me quoting part of the EULA and noting that you might want to reread it, it was mostly due to some of your earlier comments in the thread.

From those statements, it seems that a large part of your complaint is about money.

Now THAT'S clueless.

Pray, do explain.

Please use small words, as my years of experience writing long-uptime scientific-computing and network-service software have possibly left me ill-prepared for a sophisticated argument.

Ok, I am sorry about you being offended, i wasnt intendinging it to come across that way. I was just telling you what i have been told from other Australians in other MMO's and how they cope with DT in their "peak time".

For *ME* personally i am not bothered about Downtime, i have never been bothered about when downtimes happen, they are annoying to me when they happen (especially late at night for me)

But, you are then applying MY opinion across a whole section of the playerbase.

Sidenote:

You know why running a downtime when a load of people are on the server is a bad thing. When it starts back up it gets hammered.

Hard

Which is something that you DO NOT want to do.

The *Current* scheduled downtimes affect me and my GF when we play the game. Yes its during the day for us, but as she is disabled and i am her full time carer, it is something that we work around. We dont really have a schedule to do stuff because it is all dependant on her health, but when we are unable to go out, do shopping, she plays the game. So when the *current* downtime hits, it hits during her play hours.

I believe she posted earlier on in this thread as well... but didnt complain about the downtime, she just accepts it. Its something that she has had to do for a few years, in that the downtime will affect her at somepoint during the day (unless it is between 6 and 8am GMT which are her normal sleeping horus)


The system can be stable, but scheduled maintenance is there to generally also run some server specific tasks, stuff that actually allows the server to be, you know, more stable during the week. Every single server, at some point or another requires a downtime. Database servers (depending on the database) requires sessions to be cleaned up and optimised (look at the market soon after a maintenance down time and then compare it a few days later at the same time... its a world of difference)

If there was a *natural* point in time when all of the game servers had no one logged in, then chances are the developers would try their hardest to arrange the server maintenance for that period of time. There isnt, so they try to inconvenience the *lowest* population level.

My bank generally runs maintenance every saturday night/sunday morning for a few hours every single week. This is to run "scheduled" tasks that can only be ran with with a very limited number of people online....

And how do you know that the server guys are not applying OS specific server patches during the downtime? You do not know, because it isnt a patch related to the game, and so it isnt necessarily linked in game patches.

I dont believe most of us have said that it is fair to "certain players". All most (if not all) of the people have said is that it affects the few number of players *overall*.

Eventually, with a rolling set of downtimes, you will end up inconveniencing the developers as well. Maybe this becomes the time where *scheduled* downtimes dont have a developer entering the building for several hours (if it was held late enough US time zone wise) but then it would mean potentially longer than average downtimes (look at unscheduled downtimes outside of office hours as an example).

I'd be fine with that time interval since I work night shifts!

When they do early maintenance I'm cut out too

I am inclined to troll this one, but I will be honest in that i personally think that if your "years of experience" writing for one style of coding (long uptime scientific computing) is woefully inadequate to writing server code that does what CoX servers do.

How many transactions does your database make? How many users are logged into it at any one point in time? How often are there software upgrades to the base program?

The MMO evolves, it is a constantly changing codebase, and as such, it has its own problems that arises... sometimes those problems dont take long, othertimes they dont arise for several months.

The database side... yes you can leave it alone... but eventually you will *need* to reoptimise it in one form or another (typically with minimal downtime, and depending on the size of the database this can be anything from a few seconds, to 40minutes and longer).

One MMO that i play, its database is 1.7Tb in size.... and they have spent 7 years on working towards getting their daily downtime to 30 minutes or so, but they *still* need the daily downtime to perfom a non incremental backup of the database and clear out the trash.

CoX might only make a *full* backup once a week, but it does incremental backups on a regular basis.

((backing up from incremental is handy for sudden problem crashes, but you still need the full backup to speed it up if the error was actually present for a long period of time and it only appeared after a prolong amount of time)).

Of course, you with your "long uptime" experience would have known all this and it wouldnt have needed explaining in the 1st place.

In your haste, I think you missed the bit about network-service software, too.

I'm not talking about bugs that become apparent, and need to be addressed post haste — that's why there is also unscheduled downtime, annoying as it is. The question is: what is going on such that every week, the whole thing needs to be effectively turned off and on again?

Databases can and are backed up live. (I know my bank, for example, doesn't have weekly scheduled downtime. Does yours?)

Well, yes, that's correct. Your post really did not explain much at all that needed explaining, and was patronizing to boot.

It comes down to an economic decision: doing it 'right' obviously requires more manpower and possibly hardware than doing it the way they are now. They believe, and quite possibly correctly, that the lost revenue from those who give up playing from a disadvantaged timezone is more than offset by the savings they gain from just kicking the servers every week instead.

The whole network-service side i felt was not appropriate, because you didnt actually specify what it covered. Infact, i feel that your experience itself is not *directly* applicable to the game server industry

Databases can indeed be backed up live, either incremental or "whole hog". The weekly downtime i would assume to be the "whole hog" backup, because that way with no-one actually playing the game, you can more easily drop any open connections that there would be the database, ensure that there are no writes happening to it, and then back up the whole hog.

You missed out the part where i said that my banks website *does* have weekly downtimes (typically early hours sunday morning but that is neither here nor there, it is at a time where there are fewest people using the service)

*EDIT IN* Side note: if you have to drop the database connection to a player for any reason, then more often than not its bad for their experience as they get forcibly disconnected from the server in that situation. Unless of course you allow the client to be "trusted" above the server in those situations (Rule for MMO's... never trust the client for stuff, its open to providing false information with bots/injection et al).

Also from talking to one of my contacts in the industry, there are also some maintenance tools that need to be ran as well, and if the server is constantly working, then it just prolongs the agony, and can also make the server unstable whilst it is *active*. Of course, as time progresses, and server OS/maintenance tools improve, eventually the need for downtimes reduces.

Sorry if it came across patronizing, but you *did* ask for the information, as i didnt actually know what your experiance was, it is generally harder to hit a specific level of explanation, so you tend to go for "lowest common denominator". Saying that you are a "long uptime scientific programmer" means that you know a lot about one side of the coin (ie long up time) but not necessarily about the other side (games server maintenance).

Of course, if money is no object, then in the computing world pretty much anything is possible (within the existing technological boundries), but as you yourself admitted as well, they have probably taken the *best* option for themselves, which makes the whole argument moot.

Well, my post was obviously a sarcastic reply to an unreasonable one liner, so I didn't go in to details. I know my background is not unique, but I do have experience in the games industry (network programming, but not MMOs), distributed scientific computing, and in industrial long-running time-critical (and large!) database-backed applications.

I can't see why you would need to have downtime to do a full backup.

Er, yes. Was this in an earlier post? It's not in your reply to my snarky comment.

Well, not entirely moot. It shows they are happy to compromise the user experience of a particular subset of players, in order to save money. It's not equitable, only economic.

GM's work in Austin. You're probably referring to the QA guys like TheNet popping in from time to time.

If 'doing it right' in your world only counts if the data is treated like banking information, then there are a whole lot of key databases on this planet that 'do it wrong'.

There is no fiscal possibility or reason that player data for an MMO should be stored with the same level of CIA as banking data.

Of course if CoH had the kinda of money a bank has to store it's data and then did not, that might be a problem, since changes are they would be charging us a ton more than they do.

One cannot simply 'do it the way banks do it'. It takes, literally, thousands of servers to handle the banking data you reference.

If you are, indeed, in the industry and you do not understand that, then I am very glad you do not work for me.

On top of that, IIRC, the back-end for the CoH data is all in MSSQL servers. Those are notorious for needing regular maintenance, regardless of how stellar the code Paragon creates, so the back end would need to go offline regularly to account for that, to keep our precious data stable, through no fault of the coders.

Banks, if they know what they are doing, are not going to be using MSSQL. Even IF Paragon used a more 'bank like' database, it would certainly destroy any profits they might ever hope to make.

And yes, of course, all of this goes back to the almighty dollar, how could it not?

NCNC and Paragon would be making a very foolish business decision to waste money trying to store player data like banking information, probably one that would not allow our subscription prices to be anywhere close to reasonable.

To conclude, not all customers will EVER be treated equally by ANY company on this planet. It's called Life.

Never even suggested they should. You are arguing with a straw man, so please spare the personal barbs. Just brought up the example to show that it is entirely possible for a live, database-backed application to not need rebooting every few days. And from personal experience, I know you don't have to be a bank to do it.

In many other circumstances, it's called discrimination, and is actually illegal in many countries. (Again, before you go off on a tangent, I'm not claiming NCSoft is acting illegally.) Companies often DO have a legal obligation to act equitably in many important regards, and inasmuch as they are participants in our society, they also have an ethical obligation to act equitably.

I would not want to live in your world, MajorPrankster, where everything appears to be decided by dollars alone.