City of Heroes: Live Forum Archive

[IGOR] Got the 100k blues? This may solve them!

Aliana Blue

By Aliana Blue,
Posted on: 2009-04-24 20:09 in Mission Architect

 

Posted

This is a really nice tool. I hope you get the various limits figured out, no so much in the hope that we can bypass them, but just because it's nice to know about them.

If nothing else, this is going to let people have a decent chunk of custom critters plus very rich text, clues and dialog.

Blue
American Steele: 50 BS/Inv
Nightfall: 50 DDD
Sable Slayer: 50 DM/Rgn
Fortune's Shadow: 50 Dark/Psi
WinterStrike: 47 Ice/Dev
Quantum Well: 43 Inv/EM
Twilit Destiny: 43 MA/DA
Red
Shadowslip: 50 DDC
Final Rest: 50 MA/Rgn
Abyssal Frost: 50 Ice/Dark
Golden Ember: 50 SM/FA

 

Posted

[ QUOTE ]
I believe this is against the TOS / ELUA. You're modifying game files from my understanding.

[/ QUOTE ]

Kinda like the Offline arc creator by Leandro http://www.missionplanner.com.ar/
Or the VidiotMaps map pack, CoH Splasher, Vandens Icon pack...? none of these modify "Game Files" and have such been allowed for over 4 years.

Victory Wiki
Login Screen changer (OS X/Windows/*nix)
Web based Base designer
SO/DO Enhancement Standardization Pack
CoH Community Mods Installer

 

Posted

[ QUOTE ]
I believe this is against the TOS / ELUA. You're modifying game files from my understanding.

[/ QUOTE ]

Technically, you are modifying a file created by the game and used by the game. BUT, the modifcation doesn't harm the game's function, nor does it allow for altering game play. You can't use it to cheat, farm, exploit, etc.

In cases like this, such as the downloadable map pack and icons, it will probably not be contested by the powers that be. However, should it ever change the game in any way, you can be sure they'll "outlaw" it. As it should be.

You don't hit smiling monsters - Sister Flame

 

Posted

I finished an arc with 99.78% capacity. With no errors showing the Publish button remained greyed out.

Out of desperation I tried IGOR. After repicking boss names I was able to publish the IGOR file. File size remined the same though.

I used the mission planner and everything looked good text-wise.

I have 14 custom enemies, iirc. And text for everyone.

 

Posted

[ QUOTE ]
[ QUOTE ]

The latest patch even included something to improve validation of hand-modified MA files.

[/ QUOTE ]

"Improve" as in "It was letting through illegal hand-edits, now it shouldn't"

[/ QUOTE ]

I'm not sure what your point is. I said "improve validation". The word "validation" is important.

Blue
American Steele: 50 BS/Inv
Nightfall: 50 DDD
Sable Slayer: 50 DM/Rgn
Fortune's Shadow: 50 Dark/Psi
WinterStrike: 47 Ice/Dev
Quantum Well: 43 Inv/EM
Twilit Destiny: 43 MA/DA
Red
Shadowslip: 50 DDC
Final Rest: 50 MA/Rgn
Abyssal Frost: 50 Ice/Dark
Golden Ember: 50 SM/FA

 

Posted

Just a note on modifying files.

Prior to writing my guide about Subversion, I PMed Ex Libris about it, asking if it would be OK according to the rules. The guide included how to look into .storyarc files, compare, and modify them (by hand, or rather, with diff first then by hand).

I got an OK back. So the guide was written. And so this should be OK too.

Players' Choice Awards: Best Dual-Origin Level Range Arc!

It's a new era, the era of the Mission Architect. Can you save the Universe from...

The Invasion of the Bikini-clad Samurai Vampiresses from Outer Space? - Arc ID 61013

 

Posted

Maybe they want ppl's arc sizes to be bloated, using "100k is the max so file sizes aren't too huge" as a coverup for "we dont care if ur file sizes are bloated, we wont let u publish ur igor'ized versions of em either heh heh heh"

 

Posted

The publish button is unrelated to how things are stored on the server side - its very likely (though obviously not recommended) that forcing publish to be enabled would let you post stories of any size. The checks that we're dealing with are entirely client-size.

The issue is that the devs have stated there is a 100k limit, but this is really an abstract estimate - sometimes your filesize will be larger than 100k and it won't mind, sometimes it'll be much smaller and still mind. There are even things that the client-side filters trip on that isn't mentioned (filename length, for example) in the errors list... all that happens is publish and test grey out.

While NCSoft can simply declare things to be against the rules on merit of owning the system, the EULA thing is, from a legal standpoint, garbage. There is no such thing as a proprietary file format. There are such things as patented algorithms, and copyrighted materials. Neither of these apply to this. They don't, in fact, apply to the pigg files either beyond the fact that unless you change ALL the contents of a pigg file, you can't redistribute the results without violating their copyright on the things you DID NOT change. That's neither here nor there for this case, but I figured I'd point it out since that stickied thread on every forum grates on my nerves for giving illegitimate reasoning for what is just an arbitrary decision.

I will continue to investigate the *real* storyarc limits based on the info folks have generously provided in this thread. So far I'm hearing that 17 critters works in an IGOR'd file, so I'll see what shakes out of that.

Thanks again folks!

 

Posted

Actually, the 17 critter version of mine doesn't work, that was after i added some lol

 

Posted

Am I getting this right? What I'm seeing is that you CAN publish these IGOR files as long as you don't open them in the game? Wouldn't that mean you could never ever edit them at all?

Dec out.

 

Posted

Getting errors after an IGOR pass. It appears it is pulling out the group info for custom bosses that are in their own group. Any ideas?

<sigh> Viv says its no longer "all me".
http://wendy-mags.mybrute.com/

 

Posted

This likely falls into the same classification as modifying the map files which are made to be modified.

The fact that this thread has been left up means that

A) Moderator hasn't killed it.
B) No one has contacted those involved

I would be surprised if the Devs weren't interested in the results. Seems to me the BEST case would be in include these improvements in the regular game at some point (right now MA doesn't need any new issues lol).

----------------------------
You can't please everyone, so lets concentrate on me.

 

Posted

To go for broke on the reduction, it pulls out the referencefile lines (which do nothing but make a link between each of your custom critters and their original files... which we DONT want in this case). The bosses are not removed at all, but as was stated previously, you'll see a slew of errors if you open it in the editor after IGORing it. Most notably statements that custom enemy name or custom group names are missing. They're not missing until the editor loads the storyarc and finds the critter file links missing.

Yes, one of the things IGOR does is remove hard spaces which each use up 6 bytes. HTML, as you see from your posting, only does one actual space in a row... after that each space becomes &amp;nbsp; -- which is a giant waste of space if you're already over the filesize (?) limit.

This is only the beta. Once everything is settled I can but in configuration options that let you choose which reduction actions you want taken, or not taken, if you're finding you have an absolute need for hard spaces.

Its important that, when you report errors, you mention whether they're errors during publishing or errors when you open the resulting IGOR file in the editor. The latter is going to happen, the former is where we need to be concerned.

As to the question about "does this mean we can never edit the file?" -- yes, that's what it means currently, though IGOR does not alter your original storyarc! It creates a new copy of it that is reduced. So, if you need to edit your story arc... edit the original then re-IGOR it. Its possible to do a lot of reduction while still leaving the IGOR file in a format that will not give errors... but if you edit it and re-save it... it'll be back to full size, so... best to publish the IGOR versions, but work on your original versions.

 

Posted

[ QUOTE ]
The 100k thing is actually misleading in that even without reduction, some files over 100k still show as less than 100% while some that are under (as we've seen) show as over.

[/ QUOTE ]

It might be misleading, but its not intended to be nor does it claim to be the local file size. Local file size is independent from internal mission arc size. Many things affect mission size and local file size in a completely different way. My mythbusting thread unfortunately got eaten, but one of the things I discovered is that the recommendation to set unused costume parts to black is worthless: it doesn't decrease mission size (at least, it did not at the time I tested). It does dramatically reduce local file size, but that doesn't help the author.

Also, the 100k limit is actually 100,000 bytes, not 100k.

I'll take a look at IGOR to see what it does; some of the things it does (like stripping or reducing file references) might help, and then again it might not.

[Guide to Defense] [Scrapper Secondaries Comparison] [Archetype Popularity Analysis]

In one little corner of the universe, there's nothing more irritating than a misfile...
(Please support the best webcomic about a cosmic universal realignment by impaired angelic interference resulting in identity crisis angst. Or I release the pigmy water thieves.)

 

Posted

[ QUOTE ]
To go for broke on the reduction, it pulls out the referencefile lines (which do nothing but make a link between each of your custom critters and their original files... which we DONT want in this case). The bosses are not removed at all, but as was stated previously, you'll see a slew of errors if you open it in the editor after IGORing it. Most notably statements that custom enemy name or custom group names are missing. They're not missing until the editor loads the storyarc and finds the critter file links missing.

Yes, one of the things IGOR does is remove hard spaces which each use up 6 bytes. HTML, as you see from your posting, only does one actual space in a row... after that each space becomes &amp;nbsp; -- which is a giant waste of space if you're already over the filesize (?) limit.

This is only the beta. Once everything is settled I can but in configuration options that let you choose which reduction actions you want taken, or not taken, if you're finding you have an absolute need for hard spaces.

Its important that, when you report errors, you mention whether they're errors during publishing or errors when you open the resulting IGOR file in the editor. The latter is going to happen, the former is where we need to be concerned.

As to the question about "does this mean we can never edit the file?" -- yes, that's what it means currently, though IGOR does not alter your original storyarc! It creates a new copy of it that is reduced. So, if you need to edit your story arc... edit the original then re-IGOR it. Its possible to do a lot of reduction while still leaving the IGOR file in a format that will not give errors... but if you edit it and re-save it... it'll be back to full size, so... best to publish the IGOR versions, but work on your original versions.

[/ QUOTE ]

Upon sending the file through the program and reducing the file size the publish button still didn't light up to allow publish. Upon looking at the arc it said it had a correct file size, but it gave the errors about the custom critters missing.

Will try to post a more detailed explanation tonight when I can give it a pass through and detail for you what seems to be happening more accurately.

<sigh> Viv says its no longer "all me".
http://wendy-mags.mybrute.com/

 

Posted

That's a nonsensical statement, Arcanaville - what is the point of TELLING us that the maximum size is 100k if that doesn't correspond to anything we can measure ourselves? Given that approach they might as well say the maximum size is "100%" on the little meter, which it theoretically is but practically isn't... no more than the maximum filesize is 100,000 bytes. I have a non-reduced mission that is 92% in the editor, and 105k on disk.

The only rational explanation is that 100k was an estimate they gave based on some level of trial and error with the system, since it corresponds directly to nothing we can measure.

Shrinking the file does have a positive effect - it will take most storyarcs that go slightly over the maximum limit and allow them to be published successfully - this has been tested and requires no confirmation by you. What WOULD be helpful is thoughts on what changes DO affect the result and which do not.

Toril - what's the pre-reduced filesize on your arc, and how many custom critters? You can ignore the errors you see in the editor when you open an IGOR file in the in-game editor, though. If you're worried that its actually removing things, put one of the missions from your arc into its own 1-mission arc, make sure the publish button is lit for it, then IGOR it and test it to see that everything is still in place even though the editor will say things are missing. Something IS missing - bloat.

 

Posted

Tried tinkering with this a bit. It took my current project, 110k file size and 99.77% of the max size as read in game, down to 57k file size and 68.19% as read by the client.

While I don't yet notice any detrimental effects to my precious text, I did notice the changes it made to my custom NPCs. My Emp Defender donning a full white suit of Justice gear with the Vet reward angel wings, now has grey hair, no wings, and a black suit of Justice gear. This was observed during a Test run.

If this process really can be perfected though, then I can't wait to see what the finished product can do.

 

Posted

[ QUOTE ]
no more than the maximum filesize is 100,000 bytes. I have a non-reduced mission that is 92% in the editor, and 105k on disk.

[/ QUOTE ]

The meter measures memory footprint, not file size. That's direct from pohsyb in closed beta, and he wrote the thing.

 

Posted

Awesome, Ledain - I mean, not awesome in that it changed your critter, but awesome in that its something concrete to work with. Can I have a copy of the arc file to work with? Either that or just the .costume file for that critter works.

As for the meter being "memory footprint" - I'm willing to buy that, but likewise, that's the "memory footprint" as given by the editor, which is not the same as the routine used to decide if the arc can be published or not as evidenced by the fact that you can run IGOR on an arc and it'll publish, but if you were to load it in the editor it would give errors... and that some arcs which do not hit or go over 100% on the "memory meter" still have the publish button greyed out.

100k was the stated maximum size of arcs - unless the devs are total idiots (which we'll assume they are not) telling us that 100k is the maximum as measured by a system we do not have access to would be silly. Neither basic filesize nor the "memory meter" are foolproof ways to predict the acceptability of the size of a storyarc, so I'm going to continue to call both of them abstract estimates.

Maybe the Backwards Byshop can offer some insight - barring that, I'm going to keep plugging away things the old fashioned way.

 

Posted

[ QUOTE ]
That's a nonsensical statement, Arcanaville - what is the point of TELLING us that the maximum size is 100k if that doesn't correspond to anything we can measure ourselves?

[/ QUOTE ]

The meter is the thing that allows us to "measure for ourselves" and I did extensive tests during beta that confirm beyond any shadow of a doubt that at the time of testing the mission limit was 100,000 bytes, and those bytes were bytes of content as the MA measures content not the file size on disk. Adding 1000 asci characters to a text field increases mission size by exactly 1.00%, factoring out tag-expansion.

Beyond that, while some things that shrink the file will also shrink memory footprint, other things definitely will not - colors for example are "written out" in the mission file but clearly stored in the MA in packed format, because there is no size difference between 0,0,0 and 255,255,255 - conclusively proven by direct testing. So if your sole measure of your tool's optimization is the actual local file size without careful MA in-game testing, I'm afraid you'll pursue highly unprofitable lines of attack.

But its your time to spend. I'm only advising you of the reality of the system implementation.


[ QUOTE ]
Given that approach they might as well say the maximum size is "100%" on the little meter

[/ QUOTE ]

Uh...

[Guide to Defense] [Scrapper Secondaries Comparison] [Archetype Popularity Analysis]

In one little corner of the universe, there's nothing more irritating than a misfile...
(Please support the best webcomic about a cosmic universal realignment by impaired angelic interference resulting in identity crisis angst. Or I release the pigmy water thieves.)

 

Posted

[ QUOTE ]

Toril - what's the pre-reduced filesize on your arc, and how many custom critters? You can ignore the errors you see in the editor when you open an IGOR file in the in-game editor, though. If you're worried that its actually removing things, put one of the missions from your arc into its own 1-mission arc, make sure the publish button is lit for it, then IGOR it and test it to see that everything is still in place even though the editor will say things are missing. Something IS missing - bloat.

[/ QUOTE ]

Will try that. Before its 112k. After its 98k.

12 Custom critters.

Edit : Ok, after killing out all but the last mission it will allow publish. Still has that error of course, but it will publish. So now I am guessing its still got too much info in it somehow. I'll try pruning out a critter and seeing what happens.

<sigh> Viv says its no longer "all me".
http://wendy-mags.mybrute.com/

 

Posted

Arcanville - the point was they might as well ONLY tell us that... etc etc.. rather than mentioning the 100k size limit. As people have said, at times a lower than 100% value on the meter still won't publish, making it just as abstract a determination as 100k of filesize.

As for colors not changing the ultimate size - again, there appears to be a difference between the editors measuring and the measurement used to determine whether or not it will allow you to publish. This is evidenced in a few facts... first, that files run through IGOR will publish, but will give errors in the editor. Second, that a file run through IGOR that will publish can still read as &gt; 100% on the memory meter, even if the aspects that cause stated errors in the editor are removed.

And once again, can we remember that "100k" was not something users made up, and again, not something that would be worth telling players if it wasn't meant to correspond to something they can measure.

My guess is that it checks for which parameters are present in each object, and validates the ones that are present, and makes sure key parameters are present, but does not create default value parameters for all those that are missing - that happens later (when the story is being executed). It explains why removing some costume fields reduces the file in an effective way while changing values does not (while the ascii version of red and black are different sizes, the binary representation is not).

As for wasting my time... well, we know for a fact that some changes are effectively influencing whether or not a storyarc can be published in that some that go slightly over the limit do become arcs that can be published. I'm sorry if that runs contrary to a thread you posted and are obviously very proud of and all, but since all we have to go on as far as the validation routines is guesswork and trial and error, I'm going to stick with attempting things that may not work, rather than just writing about why I don't think anything will work.

We've seen that at least some things do work, afterall.