Sentinel+ character extractor: Here it is!
By
Ad Astra,
Posted on: 2012-09-06 24:01 in CoH and CoV General Discussions
Quote:
The checksum can be trivially recomputed (just going by the length, I bet it's CRC32). It's intended to detect accidental corruption, not deliberate modifications.|
Originally Posted by Alexis_NA
No real potential for hacking. The file is checksummed, so if there's even one bit modified it'll fail the checksum.
|
Edit: TonV had updated his original post since I first read it (or I did a bad job reading it the first time
), and commented on this later on in the thread. Looks like they aren't using it for error detection, but have actually taken steps to harden it.
Quote:
|
Originally Posted by ShadowNate
;_; ?!?! What the heck is wrong with you, my god, I have never been so confused in my life!
|
I just want to add my thanks for this. Even if all I can ever do is drop my characters into a demo to show off their costumes, it will be wonderful.
@Glass Goblin - Writer, brainstormer, storyteller, hero
Though nothing will drive them away
We can beat them, just for one day
We can be heroes, just for one day
Quote:
I had heard as much, but I just couldn't believe NCSoft would do such a thing. I managed to re-up for VIP status on Friday; I guess I slipped in under the wire.|
Originally Posted by Kitsune Knight
You can't resubscribe anyways. They shut that down last Friday.
|
That is just... I have no words.
And for a while things were cold,
They were scared down in their holes
The forest that once was green
Was colored black by those killing machines
Quote:
We're working on it.|
Originally Posted by SyphonStrike
Want.
For. Mac. |
Mind Forever Burning did some initial proof-of-concept work for us to test the feasibility of the idea, and it looks like it should work, so we're going be working closely with him to try to get a Mac version put together.
Fingers crossed.
Do you know if there is a MAC version?! I was so excited to see that I could save my hero data, but am using an iMac.
There have been theories earlier in the thread about how to make it work on a mac, but they require testing as none of us seem to have a mac. Myself and another have confirmed it works on linux, which is why the mac theories came to life.
Quote:
Well, TonyV said it was designed to tamper-proof the contents of the file. It'd be pretty lame if it was a straight CRC32. |
Originally Posted by Kitsune Knight
The checksum can be trivially recomputed (just going by the length, I bet it's CRC32). It's intended to detect accidental corruption, not deliberate modifications.
|
It's not, check yourself.
Calculated the CRC32 for the file as-is: not a match for the embedded hash
Removed the XML tag after </character> and computed the CRC32 without a match
I also tried a bunch of other tricks I won't post here because they're naughty.
Nope. Not trivial.
---------------------------------
Heartbroken
I lurked a lot but I'll miss you allAlpha Team sg, Pinnacle server
Black Citadel vg
If I had no problem with it at all do I still need to redo this? Worst nightmare is I have all these saved files and one day we summon this game back to life but my files are not compatible because I had an old version.
If I need to do this that is fine, but logging into characters over and over that I loved that I may never have time to play out and midmax is kinda depressing.
Quote:
No; there's nothing wrong with any files that were exported using version 1.0 of the utility. The only extra stuff in files exported by version 1.1 are inherent and temporary powers, so if you want those retained as well, you'll need to do a re-export. If you don't care about those, you can just keep your 1.0 files.
|
Originally Posted by Tanglefoe
If I had no problem with it at all do I still need to redo this? Worst nightmare is I have all these saved files and one day we summon this game back to life but my files are not compatible because I had an old version.
If I need to do this that is fine, but logging into characters over and over that I loved that I may never have time to play out and midmax is kinda depressing. |
We've been saving Paragon City for eight and a half years. It's time to do it one more time.
(If you love this game as much as I do, please read that post.)
Quote:
Glad this was stress tested. It'd be dirty for someone to hack it when these guys worked tirelessly, unpaid, just so we'd have a way to preserve our characters.
|
Originally Posted by Alexis_NA
Well, TonyV said it was designed to tamper-proof the contents of the file. It'd be pretty lame if it was a straight CRC32.
It's not, check yourself. Calculated the CRC32 for the file as-is: not a match for the embedded hash Removed the XML tag after </character> and computed the CRC32 without a match I also tried a bunch of other tricks I won't post here because they're naughty. Nope. Not trivial. |
Quote:
We did munge it up in various ways to try to make it incalculable. Like everything, though, if you get a dedicated hacker at it, I'm sure some people out there have the skills to reverse-engineer it. But if we did our job well enough (and we are at least pretty good...), you're not going to be able to guess what we did or brute force crack it.|
Originally Posted by Alexis_NA
Well, TonyV said it was designed to tamper-proof the contents of the file. It'd be pretty lame if it was a straight CRC32.
It's not, check yourself. Calculated the CRC32 for the file as-is: not a match for the embedded hash Removed the XML tag after </character> and computed the CRC32 without a match I also tried a bunch of other tricks I won't post here because they're naughty. Nope. Not trivial. |
If you are a low-level (as in close to the iron, not n00b) hacker and see this as some sort of challenge, we ask that you hold off on that for a couple of reasons. First, there's really not that much reward in doing it. Even for an experienced hacker, it would take a fair amount of time to deduce how we're calculating that checksum and all you'd get out of it is, at best, a bunch of decked-out 50s with sweet loot. At worst, you could be sabotaging our ability to allow everyone else to import their characters onto other servers at a later date or causing people who have spent hours exporting their characters to have to do it again because we'd have to change our checksum encryption algorithm and needlessly costing us time and effort working on other stuff when we need our skills focused on other reverse engineering efforts.
If you do manage to figure it out, please keep it to yourself. If it's just credit you're after, PM me or Guy Perfect and, if you're right and you have nailed it, I'll publicly acknowledge your greatness--IF you don't release what you found out.
I'm not going to sit here and say don't poke around with it because I know hackers, you're going to no matter what I say. But consider this: if you are one of those nitty-gritty close-to-the-iron type hackers with skills in reverse-engineering encryption protocols, how about instead of dinking around with this, you ping us at the Titan Network? If that's the kind of challenge you like, we've got much more interesting, hard, and worthwhile challenges for you.
We've been saving Paragon City for eight and a half years. It's time to do it one more time.
(If you love this game as much as I do, please read that post.)
Quote:
I'm not calling you out or stroking my ego or anything, but this boggles my mind. Given what the program can do, what could possibly possess someone to suggest that any part of it is amateurish? |-:
|
Originally Posted by Kitsune Knight
The checksum can be trivially recomputed (just going by the length, I bet it's CRC32). It's intended to detect accidental corruption, not deliberate modifications.
|
Quote:
For what its worth, the same thought crossed my mind reading Kitsune's post. (The thought in your reply here, not the one Kitsune expressed.)
|
Originally Posted by GuyPerfect
I'm not calling you out or stroking my ego or anything, but this boggles my mind. Given what the program can do, what could possibly possess someone to suggest that any part of it is amateurish? |-:
|
Blue
American Steele: 50 BS/Inv
Nightfall: 50 DDD
Sable Slayer: 50 DM/Rgn
Fortune's Shadow: 50 Dark/Psi
WinterStrike: 47 Ice/Dev
Quantum Well: 43 Inv/EM
Twilit Destiny: 43 MA/DA
Red
Shadowslip: 50 DDC
Final Rest: 50 MA/Rgn
Abyssal Frost: 50 Ice/Dark
Golden Ember: 50 SM/FA
Quote:
I hadn't read Tony's update (not sure if he had added the FAQ then) where he specified that the checksum was indeed to try to detect tampering, so had (incorrectly) presumed it was merely to help detect errors, not malicious modifications. After seeing Tony's follow up post (and then the updates to his OP) I edited that post.
|
Originally Posted by GuyPerfect
I'm not calling you out or stroking my ego or anything, but this boggles my mind. Given what the program can do, what could possibly possess someone to suggest that any part of it is amateurish? |-:
|
Quote:
|
Originally Posted by ShadowNate
;_; ?!?! What the heck is wrong with you, my god, I have never been so confused in my life!
|
Quote:
Because it is. I've researched CRC32's weaknesses as part of my job: I wanted to use checksums to verify every file making up a software installation, including the file containing the list of checksums, which requires being able to easily generate a file with a desired checksum. I'll send you the details in a PM, but in short, one of the weaknesses permits someone to undetectably modify the file even without knowing the details of how the checksum is computed. Cryptographic hashes were invented for a reason, you know.
|
Originally Posted by GuyPerfect
I'm not calling you out or stroking my ego or anything, but this boggles my mind. Given what the program can do, what could possibly possess someone to suggest that any part of it is amateurish? |-:
|
Methinks your definition of amateurish and mine differ.
Quote:
Yes, a simple CRC32 would be an armature way to "sign" the file. Where did the idea come from that this is all that was done? Edit: I'm guessing it's because the checksum looks like a 32-bit number.|
Originally Posted by Katie V
Because it is. I've researched CRC32's weaknesses as part of my job: I wanted to use checksums to verify every file making up a software installation, including the file containing the list of checksums, which requires being able to easily generate a file with a desired checksum. I'll send you the details in a PM, but in short, one of the weaknesses permits someone to undetectably modify the file even without knowing the details of how the checksum is computed. Cryptographic hashes were invented for a reason, you know.
|
I've actually defeated CRC32 checksumming in (old) games. I know how to do it, so I agree that it would be very poor as a defense mechanism.
Blue
American Steele: 50 BS/Inv
Nightfall: 50 DDD
Sable Slayer: 50 DM/Rgn
Fortune's Shadow: 50 Dark/Psi
WinterStrike: 47 Ice/Dev
Quantum Well: 43 Inv/EM
Twilit Destiny: 43 MA/DA
Red
Shadowslip: 50 DDC
Final Rest: 50 MA/Rgn
Abyssal Frost: 50 Ice/Dark
Golden Ember: 50 SM/FA
First of all, many, many thanks to Mind Forever Burning for his hard work on the Mac port.
Big news: We have a Mac version that is ready for testing. Once we've verified that it's working correctly, we'll update the page on cohtitan.com to reflect its availability.
http://cit.cohtitan.com/downloads/SentinelPlus.tgz
It should be as simple as dragging the app out of that archive into the folder that you want to save your characters into, then double-clicking the app while City of Heroes is running and you have the info window up. The OS should prompt for your credentials as it requires administrator access to run.
If you have the ability to test on both Mac and Windows, we would really appreciate it if you can try exporting the same character on both and verify that the files are the same. Failing that, if you want to test and don't mind putting your XML file somewhere accessible so we can take a look at it and make sure everything checks out, that would be helpful.
Glad to see you guys got a Mac port that is as simple as the windows port without having the mac user have to do any major modifications.
(As linux user, if we are running coh, we probably already have the modifications we need to run the windows exe)
Quote:
If it is a true checksum, its trivially vulnerable to both linear and differential attacks. Obviously, Sentinel is vulnerable to most forms of chosen plaintext attacks.|
Originally Posted by Katie V
Because it is. I've researched CRC32's weaknesses as part of my job: I wanted to use checksums to verify every file making up a software installation, including the file containing the list of checksums, which requires being able to easily generate a file with a desired checksum. I'll send you the details in a PM, but in short, one of the weaknesses permits someone to undetectably modify the file even without knowing the details of how the checksum is computed. Cryptographic hashes were invented for a reason, you know.
|
But if you knew how those worked you could also have trivially checked to see if the verification data was vulnerable to them instead of guessing.
Another thing about cryptographic hashes: they are very important for unstructured binary data, especially encrypted data, because the presumption is an attacker can pad cleartext freely. For very small structured datafiles like the Sentinel+ exports, that freedom is far more limited, making practical attacks against smaller size hashes far less likely. Especially something like a hash of hashes, which sounds tailor made for a rainbow table attack.
Guy's not an idiot, so I didn't even bother to check to see if the verification data was a true checksum. When I have more time, for fun I'll try to see if I can reverse-engineer it. But I suspect Guy hardened it against most trivial attacks, so I would probably go straight to the big guns and not waste a lot of time on the stupid stuff.
[Guide to Defense] [Scrapper Secondaries Comparison] [Archetype Popularity Analysis]
In one little corner of the universe, there's nothing more irritating than a misfile...
(Please support the best webcomic about a cosmic universal realignment by impaired angelic interference resulting in identity crisis angst. Or I release the pigmy water thieves.)
Guy and I had some discussions about the nature of the verification key while he was developing it. While I won't reveal what it is, I don't doubt it could be reverse engineered and spoofed by someone who knows that they're doing. It's not something you could google a premade answer to, and basic attacks won't work on it, but it is fast and relatively simple and probably wouldn't hold up to a detailed cryptographic analysis.
However, someone who is capable of doing that probably knows enough that even if it were a highly secure cryptographic hash, they could just fool the program into dumping some other data, or tap into it and modify the data before the key is computed, or any other manner of tricks.
The fact of the matter is, since the code runs on someone's PC and dumps that data locally, with the added restriction of not depending on network access, they are ultimately in control of it if they want to be. There's no way around that, and we all know it. So we're trusting the majority of people to be honest. Cheaters gonna cheat, no matter what you do.
If it were up to me it would be a 4096-bit RSA signed SHA512 hash with a secret salt that's assembled by hardened code with reverse engineering countermeasures, but that would be massive overkill and probably a waste of time to implement given that this needed to be available sooner rather than later.
Quote:
Overkill is underrated. I actually implemented a system back in 2000 that used 4096 RSA signed 512 hashes just because, rather than the specified minimum 1024 RSA 128 bit hash, because the system was only intended to be in use for four years. Its still in use today, and will probably pass security audit until sometime around 2100.|
Originally Posted by Codewalker
If it were up to me it would be a 4096-bit RSA signed SHA512 hash with a secret salt that's assembled by hardened code with reverse engineering countermeasures, but that would be massive overkill and probably a waste of time to implement given that this needed to be available sooner rather than later.
|
Why 4096 RSA and 512 hash? 512 was as high as my libraries went, and 8192 RSA took too long to compute.
[Guide to Defense] [Scrapper Secondaries Comparison] [Archetype Popularity Analysis]
In one little corner of the universe, there's nothing more irritating than a misfile...
(Please support the best webcomic about a cosmic universal realignment by impaired angelic interference resulting in identity crisis angst. Or I release the pigmy water thieves.)
Quote:
I realized earlier today (and confirmed a few minutes ago) that the wrapper I made for the binary fails if the path has spaces in it. Rookie mistake! Anyway, a fix is forthcoming.|
Originally Posted by Codewalker
Big news: We have a Mac version that is ready for testing. Once we've verified that it's working correctly, we'll update the page on cohtitan.com to reflect its availability.
http://cit.cohtitan.com/downloads/SentinelPlus.tgz It should be as simple as dragging the app out of that archive into the folder that you want to save your characters into, then double-clicking the app while City of Heroes is running and you have the info window up. The OS should prompt for your credentials as it requires administrator access to run. |
Also, due to the way I'm requesting the permissions required to run the program, the output files are written with root ownership. I might be able to find another way to do that, so if this becomes an issue for people, let me know.
And for a while things were cold,
They were scared down in their holes
The forest that once was green
Was colored black by those killing machines
Quote:
Back in 2000, I was tasked with implementing security for a micro feature within a larger software application. I put in a basic authentication handshaking protocol, and stubbed out the encryption part of it with a very dumb, very obviously reverse-engineerable algorithm. I won't even repeat it here, it was so ridiculous.|
Originally Posted by Arcanaville
Overkill is underrated. I actually implemented a system back in 2000 that used 4096 RSA signed 512 hashes just because, rather than the specified minimum 1024 RSA 128 bit hash, because the system was only intended to be in use for four years. Its still in use today, and will probably pass security audit until sometime around 2100.
Why 4096 RSA and 512 hash? 512 was as high as my libraries went, and 8192 RSA took too long to compute. |
Five years later, the stub had still never been replaced with real encryption the way I had intended. That year users found a way to trivially gain access to the micro feature... not by reverse engineering my ridiculous toy algorithm, but by simply swapping versioned components from other installations, defeating the higher level logic that decided whether the encryption handshaking needed to happen at all.
There's probably more than one lesson in there, somewhere.
And for a while things were cold,
They were scared down in their holes
The forest that once was green
Was colored black by those killing machines
I'm just keeping these in the hopes of a private server. If something legit came open but we all had to start from scratch, I'd still be happy.