What are the size limits?
We've been told 100k - but 100k of what?
* the percentage meter claims file size - but which files?
* the tooltip states it's how much 'storage space you have used for your story'.
As a techie, this leaves it unclear. Is it:
* Size on our local drives?
* Size on the MA-arc servers?
* Size of the contents?
* Or of the disk space needed to store the file?
* Is any compression or trimming used?
This was made more confusing due to the reports of arcs showing under 100% being too big to publish, and those at 100% publishing fine.
Whilst creating it, I became aware that my problem-arc published fine and was stored in a 122k file on my local drive but only showed c.90% usage in the MA editor. Of course I was fine with that, as trimming the file before transferring it to the servers could account for that...
Another example, another of my arcs has a storyarc file of 67.1kb. Now I know that 67.1kb is 67.1% of 100kb, but the MA editor tells me that it is using 54.87% of the allowable size. Does this mean that some stuff will be trimmed out or compressed when sent to the server? Does it mean that the MA Editor calculation is wrong? Who knows?
So if you're still uncertain as to what is being limited to 100k, then you understand this as well as I do! We've been told a limit but have no idea what the limit is for or really means or even if it is WAI.
What are the MA files on our local drive?
Note: Following is for Windows - I don't know about the Mac
MA files are contained in the following subdirectories of your CoX installation directory.
/Missions - contains .storyarc files which are complete storyarcs
/Custom_Critter - contains .critter files - each an individual custom mob
/CustomVillainGroup - contains .cvg files - each an individual custom group (not neccesarily villainous)
/costumes is also used for saving/loading costumes, as has been possible for sometime now
If you start writing a relatively simple 1 mission arc, with:
* 2 custom mobs;
* each with their costume stored on disk (so 2 costumes);
* each mob in their own group (so 2 custom villain groups);
* a tiny 1-mission arc which uses the 2 mobs/groups;
* virtually no clues/souvenirs/dialog etc
Looking in the above 4 directories you would probably see a total of just over 64kb being used.
Luckily - that doesn't mean that you're at 64% usage yet. The MA files contain a lot of replication/duplication of data.
* Costume files
Appear to be approximately 8kb in size.
* Custom Mob files
Contain some power data, info/description data and a copy of the custom mobs costume, which is stored in the same format as the costume file and takes c.8kb.
* Custom Villain Group files
Contain copies of the custom mobs that are in the group, including, of course, a copy of the custom mob costume which takes up c. 8kb.
They also contain data on standard mobs, but that seems to come in at well under 100 bytes per standard mob-type.
* Storyarc files
Contain everything needed to run the story arc - which includes copies of the custom villain group data, which contain the custom mob data, which contain the 8kb costume!
Because of this, the above example may have file usage similar to the following:
Costume files - 2 files at 8kb each - total 16kb
Custom Mob files - 2 files again at 8kb (for the costume) - total 16kb
Custom Villain Group files - 2 files again at 8kb - total 16kb
Storyarc file - 1 file at 16kb + description/objective/souvenir/clue data - total 17kb+
Note that it should only be the storyarc file that needs transferring. So it's this file that we need to keep down to under 100kb (though 100kb where and whether it's compressed/trimmed is still anyone's guess).
Also interesting to note is that the standard mob types and number of custom villain groups have virtually no impact upon the story size when compared to the impact caused by custom mobs.
WARNING! With so much data being duplicated/replicated it seems possible that sometimes you'll edit a costume/boss/group in one place but the MA editor may refresh with a different copy and so it looks like your edits aren't effective.
It's not for me to criticise the multi-file structure.. oh hell - of course it is: IMO the devs really should look at employing file linking until publish time. It's easy enough to create multiple copies of a file (either file copy in the O/S or save as in the editor) if you want to, but defaulting to data duplication feels like it's just trying to make bugs more commonplace and harder to trace! 
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
What Size are the Story Elements?
Note that in the following:
- custom groups/bosses etc can add 8k, but only seem to be added to the file once even if they appear many times. So custom mobs may add 8k or may add virtually nothing if they are already used elsewhere...
- standard mobs/maps may vary in size as they tend to be stored as text descriptions - obviously something like 'Minions of Igneous' is slightly longer than 'Arachnos'.
- I may not have noted in all places where custom objective placement, dificulty or spawn trigger is selectable.
I've had a quick look at the sizes of various story elements - these being the arc info, mission info, clues, dialog, objectives overhead etc.
This is what I've found so far:
Empty Arc/Mission Overhead:
Creating a new empty arc and saving it immediately already creates a 530 byte file.
530 bytes is Nothing, and mopst of it is empty data for mission 1, but at the same time looking at the file there are several bits of data that could do with being left out of the file and defaults provided. E.g. the timelimit appears and defaults to zero.
Of course, this is nothing playable - the details still need to be filled in.
As virtually all of this is default/placeholder data for mission 1 I'd expect each subsequent mission to have it's own overhead of not-much-under 530 bytes.
Adding a contact
The above empty arc uses the default hologram contact.
Adding a standard contact, without renaming them takes c.60 bytes, but may change depending upon actual model used.
Renaming the contact adds a further (21 + length of name) bytes.
Adding a custom contact adds c.8k, but at least you save the c.60 bytes that the standard contact took up. 
Standard Enemy and Map
Adding a standard enemy and a standard map for mission 1 ups the file size by about 90 bytes.
The arc still isn't playable as I've not set any objectives (although for testing sizes I did add 1 character descriptions to mandatory fields).
Custom Enemy Group
Adds c.8k per custom character and c.100 bytes per standard mob that are in the custom group.
Objective Costs: Basic Mission Goals:
Mission Goal - Fight A Boss
Using a standard mob as the boss with a single-character name (as name is required) caused a filesize increase of 288 bytes.
Using a custom boss costs c.8k if their group isn't already in your arc.
Adding single character text/dialog for the 14 boss-nav/dialog/info/clue options added a further 300 bytes.
I'd estimate that a custom boss unused elsewhere, with a line or two of dialog/info for each available option could easily make the entire boss objective take up 9-10kb. STandard mobs with dialog I'd guess about 700 to 1000 bytes.
Mission Goal - Collect An Object
Adding a single-character-named object to collect added 175 bytes.
Adding single character text/dialog for the 8 object-nav/interaction/clue options added a further 184 bytes.
I'd estimate that custom interaction/navigation text and a clue could easily make the object collection objective take up 800-1100 bytes.
Mission Goal - Defeat All Enemies
This added a whole 41 bytes.
So small that I didn't even investigate the single text option and the option to change from kill all to just kill all in last room...
Mission Goal - Free A Captive
Using a standard mob as the captive with a single-character name (as name is required) caused a filesize increase of 211 bytes.
Again you can use a custom mob, in which case it could add about 8k to this.
Adding a surrounding villain group only added 22 bytes, but again this can be a custom group which could boost the size by an approximate multiple of 8k.
Adding single character text/dialog for the 11 hostage/captor nav/dialog/info/rescued-clue options added a further 233 bytes.
Setting the 3 emotes options from default added 93 bytes - but the exact amount will vary as, again, emotes are stored as variable-length text strings.
The entire 1-character-text rescue added 573 bytes, so using standard mobs and filling all info/options/emotes/etc I'd guess that it'd be easy to hit 700-1000 bytes. Custom captive and guards could easily add to that though (in 8k-ish multiples).
Objective Costs: Advanced Mission Goals:
Mission Goal - Add An Ally
Using a standard mob with a 1-letter name added 188 bytes.
Adding a surrounding villain group only added 25 bytes, but again this can be a custom group which could boost the size by an approximate multiple of 8k.
Adding single character text/dialog for all 13 ally/enemy nav/dialog/info/rescued-clue options added a further 287 bytes.
Adding non-default emotes (to all 4 options available) added 130 bytes, but again will depend on exact emotes used.
Total 1-character-text with emotes and standard mobs was 629 bytes. But adding real text I'd guess it'd be easy to hit 800bytes - 1.2kb. As always, custom mobs wil hurt more!
Mission Goal - Add An Ambush
Adding an ambush called 'a' to spawn on a valid trigger and using the mission-default enemy used a bare 122 bytes.
There's a single line of dialog that can be entered that is mentioned by the ambush when it spawns - the cost of this should be fairly minimal and wasn't checked. As always using new custom mobs would be the killer here.
Although names for entities that act as triggers (e.g. boss, ally names) can get repeated in trigger names and can also add a fair few bytes... but more on that later.
I'd guess that with reasonable names and trigger/placement options most ambushes would come in around the 200-400 bytes mark.
Mission Goal - Add An Escort
Escorts are mobs that you free and must lead to the exit.
Adding a single-character named standard mob as an escort objective adds just 232 bytes to the file.
However Escort objectives have the most text/clue/dialog options available (at 16) and a grand total of 5 selectable emotes. With the added surrounding enemy type, placement, betrayal, difficulty, pet combat selection etc etc the escort probably has the potential to have the most impact of any single objective type.
Using non-defaults and single-character text the total escort size comes to 864 bytes. Proper text could probably add a kilobyte to that.
Mission Goal - Add A Patrol
Adding a patrol with a single character name and standard mobs added 134 bytes.
Adding single character dialog added 44 characters (that's 2 lines of dialog, which may be up to 100 characters each).
This patrol wasn't set to spawn on a trigger, was default position, default alignment and default difficulty. Altering any of these options increases the size.
With standard mob types, using dialog and setting triggers/alignment/difficulty could easily see a patrol reach around 500 bytes.
Mission Goal - Add A Battle
Adding using a single character name, and 2 standard mob types added 204 bytes to the file size.
There are 4 rows of dialog which can each be up to 100 characters long. Placing just a single character in each of these adds 84 bytes.
It's easy to see that with standard mobs, dialog and spawn-trigger/difficulty/placement/alignment all being set the battle could easily add over 600 bytes.
Mission Goal - Add A Destructible Object
Adds 154 bytes with a single character name, and everything else on default.
Like bosses there are 14 nav/dialog/info/clue and adding single character values added a further 300 bytes.
Adding standard enemy mobs around the object added less than 30 bytes in my test. Adding placement and difficulty information would add more (I'm guessing similar to the patrol).
Mission Goal - Add A Defendable Object
Initially adding this adds 153 bytes.
Seems to be very similar to the Destructible Object objective, with the exception that there is no clue available when the objective is complete.
I'm going to make the lazy assumption that it sizes similar to the destructible object.
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
The JD Arcsize Guesstimator
Hereby known as the JDAG, this table gives the approximate sizes of the various mission arc elements.
All sizings are approximate, and where a range is available it allows for some verbosity of clues/text or variation of options.
I may also be making most of this up and may need to review my table at a future date.
<font class="small">Code:[/color]<hr /><pre>
Size Description
530 b - Empty Arc/mission overhead
60 b - Standard contact (without renaming)
80 b + name - Standard contact (renamed)
8 kb - Custom contact (approx)
500 b - additional mission (2nd+) overhead (approx)
100 b - standard mob- and map-type per mission
(700 to 1000) b - Fight A Boss objective with standard mob boss
and some text and options set
(800 to 1100) b - Collect An Object objective with some text
and and options set
41 b - Defeat All Enemies objective
(700 to 1000) b - Free A Captive objective with standard mob captive,
standard mobs guarding and some text and options set
(800 to 1200) b - Add An Ally objective with standard mob ally,
standard mobs surrounding and some text and options set
(200 to 400) b - Add An Ambush objective with standard mobs,
sensible names and some text and options set
(900 to 1800) b - Add An Escort objective with standard mobs and many text
and options set
(200 to 500) b - Add A Patrol objective with standard mobs,
sensible names and some text and options set
(400 to 600) b - Add A Battle objective with standard mobs,
sensible names and some text and options set
(400 to 700) b - Add A Destructible Object objective with some text
and options set
(400 to 700) b - Add A Defendable Object objective with some text
and options set
8 kb - Every individual Custom mob type (approx)
</pre><hr />
Real Examples (using upper limits as I tend to be verbose.. if you hadn't noticed):
Example 1:
My 1 mission arc 'The foul-mouthed Handyman' has the following JDAG size calculation:
<font class="small">Code:[/color]<hr /><pre>
Mission Overhead = 500 = 0.5 kb
Contact = 100 = 0.1 kb
1 Fight A Boss = 700-1000 bytes = 1.0 kb
2 patrols = 2 x (200 - 500 b) = 1.0 kb
1 glowy = 800 to 1100 b = 1.0 kb
8 custom mobs = 8 x 8kb = 64.0 kb
--------
TOTAL = 67.6 kb
</pre><hr />
Guesstimate at 67.6 kb and the actual file size is 67.1kb (though it's actually 68kb on disk).
Example 2: my 3 mission arc 'Ee-Ai-Ee-Ai-Oh!' which is now meant to be too big...
<font class="small">Code:[/color]<hr /><pre>
Contact = 100 = 0.1 kb
Mission 1 Overhead = 500 = 0.5 kb
1 Fight A Boss = 700-1000 bytes = 1.0 kb
2 patrols = 2 x (200 - 500 b) = 1.0 kb
1 glowy = 800 to 1100 b = 1.0 kb
1 ambush = 200 to 400 b = 0.4 kb
MISSION 1 SUBTOTAL: 3.9 kb
Mission 2 Overhead = 500 = 0.5 kb
4 Allies = 4 x (800 to 1200) = 4.8 kb
1 Free Captive = 700 to 1000 = 1.0 kb
1 Object to Destroy= 400 to 700 = 0.7 kb
2 patrols = 2 x (200 - 500 b) = 1.0 kb
custom mobs (ally) = 4 x 8kb = 32.0 kb
MISSION 2 SUBTOTAL: 40.9 kb
Mission 3 Overhead = 500 = 0.5 kb
1 Fight A Boss = 700-1000 bytes = 1.0 kb
custom boss = 8kb = 8.0 kb
1 Ally = 800 to 1200 = 1.2 kb
1 Battle = 400 to 600 = 0.6 kb
3 ambushes = 3 x (200 to 400) = 1.2 kb
custom mobs (A) = 5 x 8kb = 40.0 kb
custom mobs (B) = 3 x 8kb = 24.0 kb
MISSION 3 SUBTOTAL: 76.5 kb
-------- --------
TOTAL =121.3 kb 121.3 kb
</pre><hr />
Yikes! My JDAG guesstimate is 121.3 kb, which is a relief as I had no idea if my table of estimate sizes was anything like accurate, but the actual file size is 121kb (taking up 124kb of disk).
That's so close I'm pretty darned happy! 
By the way, in the above, the custom mobs in mission 2 were different custom mobs to mission 3 and mission 3 included 3 distinct groups of custom mobs. Also the battle objective in mission 3 is counted only once, even though the objective is repeated several times (by setting the number of battles in the editor).
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
Tips and Tricks
These only have small saving - the big savings, IMO, really need to come from the devs.
But though they are small savings, they all add up - and if your arc is just over the limit then these may help pull it back to a size where it is publishable!
By the way - I am not going to try covering good, succinct writing style. Yes, people get bored easily, but there's a wealth of information available upon writing for the txting generation, and only you know whether the text is needed or not for story progression. Instead I'm going to try sticking to peculiarities of the MA system and how it can be made less wasteful of space!
Which Contact/Mob to Use..?
One of the mantras I've seen bodybuilders use is about energy conservation "Don't stand if you can sit, don't sit if you can lie down".
Something similar goes for mobs and contacts in MA.
Don't use a custom contact if you can use a standard one.
Don't use a specific standard contact if you can use the default hologram.
Heck - we've been used to grabbing missions from papers and radios - why should we all need a funky custom model to stand there and tell us to get into that green thing behind them??
Story - that's why. When filespace is tight, story/comedy/impact/effect is only reason to use a contact at all.
But if the story doesn't need it, then not using a custom mob could save you 8k and not specifying any contact saves at least 60 bytes (more if you renamed the contact).
And the similar rule for mobs is:
Don't use a new custom mob if you can use one that's already in the arc.
Don't use a custom mob if you can use a standard one.
Don't use a standard mob if you don't need the objective at all.
btw I use mob as in mobile-object - a single critter/ally/enemy/anything that moves and isn't a player[/i].
If you're getting short of space, using a new custom mob can add 8k. Re-using a custom mob or a standard mob is much more efficient, but not looking at whether you need a custom mob in this instance and deciding you don't may let you remove the custom mob from the arc altogether. And if you're really struggling it's worth asking whether you need that extra boss/ally/captive/battle at all...
Re-use custom mobs if you can
Custom mobs only get added once to an arc. If you've added a custom mob in mission 1 then using it as is in missions 2, 3, 4 and 5 adds no extra overhead.
However having multiple versions of a mob that looks similar or is similar in the story-sense does not share this saving.
i.e. Having Hexenhammer appear as a boss in mission 1, appear in a different costume in mission 2 and appear in his AV/EB form in mission 3 will require 3 different versions of Hexenhammer to be stored in the arc. It doesn't matter that the powersets and info are the same - to the program they are effectively totally seperate mobs.
Faction Fluff
When you include an existing custom group, all individual mobtypes in that custom group get included in the arc.
If the faction was originally created for another arc you may find that you're including mobs that you don't want or need in your current arc. You may be better off creating a new group using the required mobs from the existing group.
Failable Missions and Failable Text
Failable Objectives are:
- Escort objective (escort dies);
- Defendable Object objective (object destroyed);
- Mission time limit expires.
If you include a failable mission then you need to include mission failed text.
If you don't need to have a mission failable then you can skip the mission failed text and save some space.
Or better yet: Not filling in the mission fail text and the failed mission contact return text puts the following in the arc file:
ContactReturnFail ""
MissionFailDialog ""
If you put a single character (e.g. A) in these two dialog options the story file stores the following:
ContactReturnFail A
MissionFailDialog A
So if you don't have a failable mission, put a single character into the mission fail dialog and the return to contact fail dialog and you'll save 2 whole bytes!
Triggers
1. Trigger Names
The trigger when I tested ambushes was the death of a boss called 'a', and so is recorded in the files as:
Trigger a
Triggering at boss health levels increases this length slightly - e.g. Trigger "a 75_HEALTH" for an ambush at 3/4 hlth.
Also note that the boss name is used - this can be up to 75 characters long. Here I used 'a' but changing it to 75 'a's changes the triggers to:
Trigger aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaa
and
Trigger "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaa 75_HEALTH"
That's a fair bit of space for just a boss name... and it doesn't actually stop there. The boss name is:
* included once at the defeat boss definition;
* included as a mission objective if the defeat is required for completion;
* included in any triggers based on the boss health (at 75/50/25/0 health).
* possibly included where a custom mob definition is stored;
And I don't claim that this list is definitive!
A long boss name may be nice and descriptive, but each character may be repeated several times. So pick a 75 character name and you may be adding 300+ bytes to your file...
Destructible Objects also have several trigger points based on health and if you're going to use them you may want a short name!
Defendable Objects have a single trigger point upon being saved.
Battle objectives have a single trigger point upon being 'complete' - though I'm unsure if that means when one side is defeated or when all combatants are defeated. I presume the former as the latter may be impossible if one side is chosen to be an ally.
Escort objectives have 2 triggers attached. When the escort is rescued and when it is completed. I'm guessing that the former is when you first find/free the escort mob and the latter is when you get them to the exit.
2. Renaming Things - Dependant Triggers don't rename
If you have a trigger being used, e.g. a trigger on the defeat of boss 'a', the trigger is not updated when you rename the underlying entity, e.g. rename the boss 'B'.
Instead it defaults back to no trigger and you'll probably see a few errors suddenly appear!
Objective Names and Mission Requirements
In a similar manner to notes on Triggers, any objectives required for mission completion get relisted in a list of mission completion - so long descriptive names can help add to your filesize.
E.g. MissionRequirements "The unjolly, green tall-troll", Bones
Here "The unjolly, green tall-troll" is the actual name I used for a standard-mob troll boss to defeat in my 'Ee-Ai-Ee-Ai-Oh!' arc. Those 29 characters are repeated 3 times (in the boss name, mission requirements and in an ambush trigger). If I changed the name to a simple "Troll boss" I save 3x19=57 bytes.
Also note that ambush and patrol name sdo not get shown to the player - they are there purely for the designer (and the program) to keep track of individual ambushes/patrols. Even though the player doesn't see the name, it takes up space in the arc. If you're short of space try to name patrols and ambushes with 1 or 2 characters.
E.g. I had patrols called 'Front patrol - medium' and 'middle patrol - funny comments' - I'd be better with calling them 'A' and 'B' or 'P1' and 'P2'.
More Names!
Villain Group Names, custom mob names and even some filenames seem to get repeated in the story files (although the filenames may not be transferred and may not count to your size limit). It can be worth keeping these short.
Even standard villains, maps and costume pieces can be worth looking at, because the names tend to be stored as variable length text strings. So using "Minions of Igneous" in multiple places actually costs more than "Arachnos" and some costume pieces cost more than others... Trial and error may be the way to find savings here.
Speaking of Costumes...
You'll be sick of costumes by the end of the next post. As far as I can see costumes for custom mobs take up by far the most part of a storyarc, and it's partly down to the file format and the way CoX reads/writes the costumes.
But before I start lamenting the costume storage, there are a few minor tricks that I've found that can shave a few bytes off here and there.
1. Black is the new nothing...
If you look at your costume files or your costume data within the critter, group and storyarc files you'll notice that there are lots of entries like this:
<font class="small">Code:[/color]<hr /><pre>
CostumePart ""
{
Geometry Smooth
Texture1 Electron_01
Texture2 !BOOT_Circuitry
DisplayName P2104750136
RegionName "Lower Body"
BodySetName Tight
Color1 0, 0, 0
Color2 212, 0, 0
Color3 0, 0, 0
Color4 212, 0, 0
}
</pre><hr />
These are used bits of costume. But if your costume files are anything like mine, then you'll also have lots of entries like this:
<font class="small">Code:[/color]<hr /><pre>
CostumePart ""
{
Fx none
Geometry none
Texture1 none
Texture2 none
Color1 255, 255, 255
Color2 212, 100, 70
Color3 255, 255, 255
Color4 212, 100, 70
}
</pre><hr />
The lack of Fx, Geometry, Texture, RegionName or BodySetName seems to be indicating that this is a placeholder. It's a space for a possible costume piece, but this costume isn't using it.
You can't just cut these out - the client will get upset and can crash when trying to load a file that you've edited to try remove costume placeholders.
But you see that colour information?
Each colour is defined by it's red, green and blue components in decimal in a comma seperated list.
Yup - in decimal. A slight pause whilst many developers spit on the ground.
One problem with this is the size taken up by different colours depends upon the colour itself. White is 255, 255, 255 whilst black is 0, 0, 0.
But that means that black takes up a whole 6 bytes less than white. And given that each unused costume piece could could have this potential saving on up to four colours...
It's not a lot, but it adds up.
Sadly the easiest way to reduce the costume size seems to be to do it in a file editor (notepad.exe will do fine if wordwrap is off) and re-load the costume.
oh - if you do this be sure to back-up/copy your files first!
2. Scale it back!
At the top of costume files/data there's a list of scales used by your mob to size it.
You'll see scales for individual body parts, like shoulders or legs.
Can anyone tell me what the difference is between ShoulderScale 0.08102 and ShoulderScale 0.08?
That's right - 3 bytes, but there's probably little noticeable on-screen difference for most people.
3. Lose it if ya can!
In many cases selecting no costume piece or no aura or no texture takes up less space than having a costume piece/aura/texture.
Would your custom mobs really lose much to not have a cape? Ok - you won't save much, but they say that gridiron is a game of inches and MA storyarcs are a game of bytes... hopefully enough of these stack up to make 1st down and publishing a possibility.
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
The Biggest Space Savings may be out of our Hands
Well if I'm lucky this will make some sense, some dev or another will read this part, and they'll be struck by what a darn fine idea it is. Or if they're working on it already, hopefully they'll realise just how much it is needed.
I was struck by looking at the files at just how big the custom mobs are - virtually every single one is c.8k.
Why is that?
It's because just about all custom mob data is stored costume data, and the stored costume data format is bulky.
The costume data format is fine for local storage of my characters costumes. Heck - for local storage I don't really care if it contains images and takes up a 8Mb per file - locally I have Gigabytes free.
But when that is transferred lock, stock and barrel into files that need to be transferred across the network and has a 100k limit then 8k per costume is a ridiculously high overhead if it isn't required.
But is it all required?
To me it appears not.
The costume file is pretty much human-readable. It's a text file. If you don't believe me then save a costume, look in the /costumes subdirectory of your CoX installation and open the .costume file in a text editor (notepad.exe will do, but you will want wordwrap off).
You'll notice that it's not all 1s and 0s but it is sort-of readable, and if you look closely (or if you're used to HTML or XML or other computer data files) you'll start to figure out what some bits are doing.
You'll notice that there are 'repeated' chunks of text that look similar to this:
<font class="small">Code:[/color]<hr /><pre>
CostumePart ""
{
Geometry none
Texture1 none
Texture2 none
DisplayName P2793026233
RegionName Head
BodySetName Hats
Color1 143, 143, 143
Color2 87, 87, 87
Color3 143, 143, 143
Color4 87, 87, 87
}
</pre><hr />
These are, as they say at the start, chunks defining a costume parts use in the saved costume. Ones with a DisplayName, RegionName, BodySetName are being used. Ones with a value other than none for Geometry, Texture1, Texture2 or Fx are also being used.
But there are lots and lots of blocks that appear to be acting just as placeholders for unused costume parts. These look something like this (although the values in the colors can change):
<font class="small">Code:[/color]<hr /><pre>
CostumePart ""
{
Fx none
Geometry none
Texture1 none
Texture2 none
Color1 143, 143, 143
Color2 87, 87, 87
Color3 143, 143, 143
Color4 87, 87, 87
}
</pre><hr />
These appear to be placeholders and indeed altering the colour values to 0, 0, 0 (the code for black) and reloading the costume has no discernable visual effect on the costume.
As the actual costume pieces that are used seem to define which body part/costume part they are, the placeholders appear to only exist because the file format has to have a set number of costume pieces.
But how much difference can a few unused costume pieces make?
In the example costume (picked at random) that I pulled the above examples from, removing the apparently-unused would take the filesize from 6.48kb to 3.86kb. That's over a 40% reduction in size.
My too-large arc had a storyarc file size of 121kb. Removing unused costume blocks would reduce this to 85.4kb - a reduction of c.30%.
Ok - I may have got this wrong and it may not all be unused... But even if that is the case then it all appears to be using a default set of values. Surely it the default-value blocks can be removed? If need be have block which defines the defaults to use for the current model.
So what do I want doing?
I ask the devs to consider altering the stored costume format to eliminate unusued or default costumepart blocks.
If custom enemies was really important enough to delay MA into I14, then surely it's important enough to do as well as possible.
The redesign would be vindicated by:
* the data-storage/transfer savings that would be made on existing arcs;
and
* the arcs that would become possible to do with an improved costume data format which are currently impossible purely due to costume overheads.
Ok - So the devs will either ignore this, tell me I'm wrong, my account will mysteriously get banned or they can do this and the MA becomes more powerful, and we become more powerful storytellers alongside it!
Whatever happens, at least I didn't start attacking the verbose naming convention in the files...
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
Final Thoughts ?
Hope that some of this made some sense...
...and may even be useful to you.
If nothing else I think I'll probably run new arc ideas past the JDAG just to get a feel for feasability before I invest too much time in them.
What we can do is small fry
The sheer bloat of costumes for custom mobs dwarfs the size of all the other data. Even if we write single sentences of monosyllabic words for all our clues we probably wont save as much as can be saved by 2 or 3 custom mobs with an improved costume data format as I discuss in the previous post.
It is a real shame that MA was delayed for custom mobs and, whilst they generally work, they do seem to suffer greatly at the hands of an older costume file format.
I'll re-iterate: The costume file/data format is great for local storage (the use it was originally intended for) but it is far too wasteful for use in a shared network-transmitted file that has to fit to a 100k limit.
Custom mobs are a real strength of MA, but using them severely limits what you can do with the story.
I urge the devs to look at improving the way costumes are read/stored/loaded as soon as possible so that MA and our imaginations can soar rather than being limited to small fledgling flights of fancy as they are now.
The MA editor is screwy as heck.
In post 2 I mentioned one of my arcs which was showing as using 54.87% of my available space.
I just emptied all my local MA-related directories and reopened this arc and the MA editor now says it's 39.81%...
Trying the same thing on my now-too-big-to-edit-and-republish arc it now shows it's using up 79.21% of available space - yet it was showing 100% and a red error because it was too big...
As I said: The MA editor is screwy as heck.
I had been looking at downloading the published version as I was going to see just how much space I could save by applying the tips and tricks that I mention in this thread/guide.
I know that I could cut about 33% out of the 121k file if only the costume data format was more efficient, and I know that I won't save as much with small tips and tricks...
But I think I'll save that test for tomorrow - the screwiness of the editor has defeated my will to go on, after working on this thread (and the data behind it) for over 15 hrs. BTW - the time spent on this is possibly a good excuse for any typos/gobbledegook that I've put in this thread/guide!
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
C'est Finis!
...for now at least, though I may do some formatting - my table seems to have lost formatting (under code tags).
Please feel free to read and tear it apart or question my sanity! 
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
[ QUOTE ]
Custom mobs are a real strength of MA, but using them severely limits what you can do with the story.
[/ QUOTE ]
I totally agree. I'd love to see what kind of missions people could come up with if they weren't so entirely limited by the number of custom enemies that can be used. I had a diverse range of enemies designed for my custom villain group before I realised how much memory they took up. My villain group of 20+ individuals got slashed to 12 individuals for a 1 mission arc and had to get slashed again to 8 individuals for me to be able to use them in a 3 mission arc (and I still had to sacrifice some story detail to fit in the 8th member). I think my group works well still but I'd love to see people facing them in their original planned diversity. If they changed nothing else about the design of the MA apart from reducing the file size footprint of custom enemy groups I'd await the live release of the issue a happy man. Alternatively increase the current limit and introduce a "low speed internet users beware" warning for larger arcs.
Defiant 50�s: Generalissimo, Righteous Bob, Splortch, Brutus Cayuga
Union 50's: Chimera Obscura, Diet Anthracite, Grim Proctologist, Puny Little Minion, Raging Bitumen
In Soviet Russia, mission farm you!
You, sir, are a hero of unparalleled genius! This is fantastic.
�How do I like my MMOs? I like them the way Paragon Studios used to make them.� - a fitting tribute from kiasa.org
EU, Union mostly.
Wow!
EDIT: I mean - thank you! Suddenly I don't feel quuite as tired/hungry as I did just 1 minute ago.
Will it get unstickied when the 1st factual inaccuracy is spotted?
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
Hehe I know you're doing your best so no worries!
Support Centre for our English European players
Support Centre for our North American players
Plateforme d'assistance pour les francophones
Support-Center f�r deutschsprachige Spieler
[ QUOTE ]
Wow!
EDIT: I mean - thank you! Suddenly I don't feel quuite as tired/hungry as I did just 1 minute ago.
Will it get unstickied when the 1st factual inaccuracy is spotted?
[/ QUOTE ]
I got to say, this is an amazing piece. So well done.
And yes its a big help.
I hope they find a way to expand the allowed size (or decrease the "cost" of placing cusomisable "heroes and vilians") and soon because, for me at least, It's that feature that is by far the most awesome part of MA.
Don't get into a flap. It's only my opinion and I'm thick
Arc 56763 Lord Anarchys heaven
2 mission arc. Bring friends cause Lord Anarchy means business...
Fantastic Work
- The Italian Job: The Godfather Returns #1151
Beginner - Encounter a renewed age for the Mook and the Family when Emile Marcone escapes from the Zig!
- Along Came a... Bug!? #528482
Average - A new race of aliens arrives on Earth. And Vanguard has you investigate them!
- The Court of the Blood Countess: The Rise of the Blood Countess #3805
Advanced - Go back in time and witness the birth of a vampire. Follow her to key moments in her life in order to stop her! A story of intrigue, drama and horror! Blood & Violence... not recommend to solo!
[ QUOTE ]
C'est Finis!
...for now at least, though I may do some formatting - my table seems to have lost formatting (under code tags).
Please feel free to read and tear it apart or question my sanity!
[/ QUOTE ]
After teaming with you, I know you're insane... 
Well, this takes me back to the wonderful times of programming on my new shiny C64. Ah, thoose were the days!
In a way I like the idea of having a limited "toolbox" and trying to get the most out of it - much like on the c64.
The challenge is to make the most of what you got; even if that means you have to edit the files themselves.
( Is that allowed, btw? )
Great great great work!
Give the man a badge!!!
/notfarouto
[ QUOTE ]
Well, this takes me back to the wonderful times of programming on my new shiny C64. Ah, thoose were the days!
[/ QUOTE ]
Real men wrote AIs on their 1k of ZX80 memory - 64k was for the weak and wasteful!
I still write all my posts by manipulating my hard drive bit by bit (literally) with an electron microscope, just to maintain that feeling of geeky-techiness.
[ QUOTE ]
The challenge is to make the most of what you got; even if that means you have to edit the files themselves.
( Is that allowed, btw? )
[/ QUOTE ]
I believe so - the file is human-readable. I'm not disassembling, reverse engineering, modifying code or tar-and-feathering myself for kicks - so I think we can do this.
One of the real tests was this thread - I did consider that any objections to modifying the text files would probably get me modslapped/banned PDQ. But instead it gets stickied - so I don't think that it's incurring the wrath of NC mods & legal department.
BUT if you do edit costume/storyarc/critter/cvg files you could end up with a corrupt file that crashes your client - as the client isn't too fault-tolerant in some places, but then it doesn't normally need to be as it usually controls reading & writing of the file.
I've only hit this whilst trying to see if you can get rid of the apparently-unused costume blocks - and the client got upset a couple of times, but that's why I state that removal of these blocks probably needs to be done by the devs.
Save often and keep multiple copies... and don't go anywhere near files other than the costume/storyarc/critter/cvg files!
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
Someone has been playing with the files and if he turns out to be right will post up the results. If he is wrong, well he is going to just pretend he didn't post
This is very handy indeed. I havent managed to reach the file size cap yet, but i'm sure i will one day and the research you have done here will be at my fingertips to help me. Thanks Judgement_Dave.
Tidgy
---------------
[ QUOTE ]
Someone has been playing with the files and if he turns out to be right will post up the results. If he is wrong, well he is going to just pretend he didn't post
[/ QUOTE ]
Darn - I don't know if you're referring to me or if there's another person playing with the files.
I think the latter, but the voices tell me otherwise.
Just now there's too much to investigate/manipulate and test and not enough time...
it's about time I clone myself to multiply the work I can get done, although I'll have to remember to grab the new SG rank of 'Supreme Dave' before I start cloning...
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
[ QUOTE ]
[ QUOTE ]
Well, this takes me back to the wonderful times of programming on my new shiny C64. Ah, thoose were the days!
[/ QUOTE ]
Real men wrote AIs on their 1k of ZX80 memory - 64k was for the weak and wasteful!
[/ QUOTE ]
Then I must be a real REAL man then, as not only did I write my AI's in 1k of ZX80 memory, I also BUILT my ZX80 from that homebuild kit they did... At aged 11...
Great article btw, JD. Props to you.
@Devs, Use compression!
@FloatingFatMan
Do not go gentle into that good night.
Rage, rage against the dying of the light.
[ QUOTE ]
[ QUOTE ]
Someone has been playing with the files and if he turns out to be right will post up the results. If he is wrong, well he is going to just pretend he didn't post
[/ QUOTE ]
Darn - I don't know if you're referring to me or if there's another person playing with the files.
I think the latter, but the voices tell me otherwise.
Just now there's too much to investigate/manipulate and test and not enough time...
it's about time I clone myself to multiply the work I can get done, although I'll have to remember to grab the new SG rank of 'Supreme Dave' before I start cloning...
[/ QUOTE ]
Maybe I should have hinted heavier. The other person is ME! Text Files + format ability + My curiosity = Reinstall.. I mean lots of hacking at the files to see what happens.
Thank you very much - for everyone's kind comments on this thread. I hope it proves useful.
[ QUOTE ]
@Devs, Use compression!
[/ QUOTE ]
I've read pointers that the issue may be to do with the MA data memory footprint (probably at the server end).
So this would possibly still be the case if you did compress on disk or in transfer (you probably don't really want to be compressing/decompressing all the time on the server when it's serving the arc).
Losing unused blocks and moving to less verbose tag naming otoh could be done to benefit the server memory usage, though the client (during play) would probably see no benefit. But that's neither here nor there if it's not a limit imposed for the client whilst running the arc.
btw - hints about it being for server memory footprint could tie in with arcs being editible/republished whilst someone is playing them. Th eserver probably needs to take (and then maintain) a copy of the arc when anyone starts running it. That way the master-copy of the arc can be unpublished or edited/republished and the players mid-arc are not affected.
[ QUOTE ]
Then I must be a real REAL man then, as not only did I write my AI's in 1k of ZX80 memory, I also BUILT my ZX80 from that homebuild kit they did... At aged 11...
[/ QUOTE ]
Kudos on the homebuild.. but still not real enough.
I learnt formal logic/boolean algebra at 3.
I was programming (admittedly only BASIC) at 4.
I actually wrote a subroutine (in COBOL iirc) that went into commercial use at 6.
Aged 9, having been programming assembler on Z80 and 6502 chips (learning from teenagers books), I asked my parents not to buy me Easter Eggs but to get me the 'bible' of 6502 assembler programming by Rodney Zaks.
I'm so nerdy I can't figure out why I'm not rich and retired...
Oh yeah - that's it - I waste my time on a damn game.
EDIT: BTW - I did all the above because I wanted to learn, not through being pressured/pushed into anything.
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
[ QUOTE ]
Maybe I should have hinted heavier. The other person is ME! Text Files + format ability + My curiosity = Reinstall.. I mean lots of hacking at the files to see what happens.
[/ QUOTE ]
I figured you or possibly someone you know. But I wasn't going to call you on it just in case they decide it is edging on EULA breaking...
If they did that they can happily get angry at me, but I wouldn't want to drag anyone else into the flames of gehenna with me.
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)
[ QUOTE ]
[ QUOTE ]
Maybe I should have hinted heavier. The other person is ME! Text Files + format ability + My curiosity = Reinstall.. I mean lots of hacking at the files to see what happens.
[/ QUOTE ]
I figured you or possibly someone you know. But I wasn't going to call you on it just in case they decide it is edging on EULA breaking...
If they did that they can happily get angry at me, but I wouldn't want to drag anyone else into the flames of gehenna with me.
[/ QUOTE ]
Meh let them try to take me. Pigg diving is one of my hobbies. Some of the stuff in there is rather amusing. There is one image of Ms Liberty with [THIS TEXT HAS BEEN REMOVED FOR EULA VIOLATION] with a marrow
Did you know that putting mission fail dialog in your missions can actually make the size of your arc slightly smaller?
Or have you learnt to appreciate what a great colour black is? It not only makes you look slimmer, but makes your arc files smaller as well!
As people get into writing their own Mission Architect arcs they could easily find that the 100k (file/arc/story?) limit becomes a painful wall.
100k may seem a lot at first, and I suspect that if you use nothing but standard mobs it probably is a lot, but as soon as you start throwing in custom mobs your meter is going to start shooting up towards that 100% size... Then the big red error tells you 'The story is too large. Try removing details, shortening text, using less custom characters or custom groups'.
So I'm starting this thread/guide to try getting down:
* what is the size limit and what does it refer to;
* what sizes do various elements take up;
* some observations upon file size and why it bloats;
* tips on anything that we can do about it;
* suggestions to the devs to help us on this with better use of space.
I also present for your use the JDAG - JDs Arcsize Guesstimator - an easy to use quick arc-size calculator table, which can be used to guesstimate an arcsize even when you're nowhere near the CoX client.
btw - Comments welcome (even negative ones) - this is my 1st attempt at a forum thread/guide of this size. So I'm bound to make mistakes - starting with my verbosity!
Contents
Post 1
1.1 - Introduction
1.2 - Contents
1.3 - Why am I doing this?
Post 2
2.1 - What are the size limits?
2.2 - What are the MA files on our local drive?
Post 3
3.1 - What Size are the Story Elements?
Post 4
4.1 - JDAG - JDs Arcsize Guesstimator
Post 5
5.1 - Small Savings and Notes
Post 6
6.1 - The biggest savings may be out of our hands
Post 7
7.1 - Final thoughts?
Why am I doing this?
Many people seemed to be struggling with the story size. It doesn't help that we've been told we have a 100k limit, but haven't really been told how we measure that outside of the file size meter... And anecdotally the file size meter has often appeared wrong with 100% size stories publishing whilst smaller ones claim to be too big.
Because of this I'd already decided to see if I could spot any way to get more punch for your byte to help all MA authors.. but then the devs made it personal.
An arc that I published on Tuesday/Wednesday night tells me that it's too big on Wednesday morning...
The arc didn't take long to create (poor humour flows quickly from me) but I had built it with the reported file size in mind and had tried being very judgemental on what made the cut. And I'd done it. My arc published and I was darn proud of what I'd managed to do.
Then the devs say it's now too big. This brings looking at MA file/story sizes to the head of my to-do list... Especially as I figure that c.30% of my storyarc file is immediately superfluous as it's automatically included but does not appear to get used.
By my mohawk shall ye know me!
my toons
Funny: Ee-Ai-Ee-Ai-Oh! #3662 * The foul-mouthed Handyman! #1076 * City of Norms #132944
Serious: To Save A Single World (#83744) * Marketing Opportunity (#83747)