CONFidence 2009 ESET crackme - solution

At last! A technical post!.. in which, I'll describe the ESET crackme from this years edition of the CONFidence conference. The CONFidence crackme (made especially for the conference - it was NOT their old crackme that is available on the ESET website for some time now) is available for download below, so one can try to break it (it's a "recover the password" type of crackme) himself:

confidence_eset_crackme.zip (the crackme, this is a post-conference compilation I received from Marcin, without some helpers that by mistake were present in the conference-compilation; btw, it works fine on XP, but has some issues on Vista)

Because the addresses of the new compilation are different then the addresses in the old one, at the bottom of this post one can download also the old binary, which breaking I will describe in this "tutorial" :)

Dear reader, if you plan to break the crackme yourself, STOP READING THIS PAGE NOW!!! :)
Below I present the solution to the ESET CONFidence 2009 Crackme.












First, a few words about the solution: as I have written in the previous post, I had three attempts to solve this crackme - the first two times I found out in the end that this is just not the right path. In this solution I'll omit the first two attempts, and focus on the correct solution.

As I've written earlier, the task was to recover/find the password, that, when fed to the crackme application, will display some message like "congratz!" (see the above screen). First, we'll start with recon - we'll use tools like PEiD (which of course I didn't have with me on the conference - I took an not-configured laptop with me, without most tools, and especially, without my source codes ;/), or Ent (this one I had with me). Below one can see the entropy chart from Ent (people who are not familiar with this kind of charts might be interested in reading this post):

As one can see, we do not have to deal with any non-trivial code encryption - it's good news - so, without hesitating any further, we can fed the executable into our favorite disassembler (that would be IDA Pro).

The Export list tells us that there are three starting points - 2x TLS callbacks and EP. This is where I made the mistake on Confi - I went straight to the EP, ignoring the TLS. Also this time I'll create some tension by heading straight to the EP :)

Let's start the analysis by finding the procedures responsible for acquiring the text string from the text field - GetWindowTextA, GetDlgItemTextA, etc - to do it, let's review the Import list. As one will find out, there is only one occurrence of GetDlgItemTextA:

.text:00405B1A GetDlgItemTextA proc near               ; CODE XREF: DialogFunc+42
.text:00405B1A                 jmp     ds:__imp_GetDlgItemTextA
.text:00405B1A GetDlgItemTextA endp

Additionally, we already see that GetDlfItemTextA is used only once - in DialogFunc+42:

.text:00401175                 mov     ebx, offset String
.text:0040117A                 push    20
.text:0040117C                 push    0
.text:0040117E                 push    ebx
.text:0040117F                 call    memset
.text:00401184                 push    20              ; cchMax
.text:00401186                 push    ebx             ; lpString
.text:00401187                 push    67h             ; nIDDlgItem
.text:00401189                 push    [ebp+hDlg]      ; hDlg
.text:0040118C                 call    GetDlgItemTextA
.text:00401191                 test    eax, eax
.text:00401193                 jz      short loc_4011AE
.text:00401195                 push    ebx
.text:00401196                 call    sub_403055
.text:0040119B                 test    eax, eax
.text:0040119D                 jz      short loc_4011AE
.text:0040119F ; "Congratulations, your password is corre"...
.text:0040119F                 push    offset aCongratulation
.text:004011A4                 push    [ebp+hDlg]      ; hDlg
.text:004011A7                 call    sub_401132
.text:004011AC                 jmp     short locret_4011C9
.text:004011AE ; ---------------------------------------------------------------------------
.text:004011AE loc_4011AE:                             ; CODE XREF: DialogFunc+49
.text:004011AE                                         ; DialogFunc+53
.text:004011AE ; "Sorry, your password is wrong"
.text:004011AE                 push    offset aSorryYourPassw
.text:004011B3                 push    [ebp+hDlg]      ; hDlg
.text:004011B6                 call    sub_401132
.text:004011BB                 jmp     short locret_4011C9

At the VA 00401196 a call sub_403055 takes place, and the acquired string is passed in the argument. Next, depending on the result of the function call, one of two messages is
displayed: "Congratulations..." or "Sorry...". So, the heart of all evil is the sub_403055 function.

Upon entering the function we are made aware that it's long. Very long. Veeeery very long. It has about 3000 lines of assembler code, mainly sub, xor, lea, ror, rol and add instructions. It looks like this:

.text:00403055 sub_403055      proc near               ; CODE XREF: DialogFunc+4C
.text:00403055 arg_0           = dword ptr  8
.text:00403055                 push    ebp
.text:00403056                 mov     ebp, esp
.text:00403058                 push    5
.text:0040305A                 pop     ecx
.text:0040305B                 mov     esi, [ebp+arg_0]
.text:0040305E                 mov     edi, offset go
.text:00403063                 pushf
.text:00403064                 xor     dword ptr ds:[esp], 100h
.text:0040306C                 popf

.text:0040306D                 nop
.text:0040306E loc_40306E:                             ; CODE XREF: sub_403055+2A94
.text:0040306E                 lodsd
.text:0040306F                 sub     eax, 8A14F2F5h
.text:00403074                 xor     eax, 7418FCC5h
.text:00403079                 lea     eax, [eax+3A61C552h]
.text:0040307F                 sub     eax, 0D101638Ch


.text:00405AE4                 ror     eax, 17h
.text:00405AE7                 stosd
.text:00405AE8                 dec     ecx
.text:00405AE9                 jnz     loc_40306E
.text:00405AEF                 nop
.text:00405AF0                 nop
.text:00405AF1                 nop
.text:00405AF2                 nop
.text:00405AF3                 mov     edi, offset go
.text:00405AF8                 mov     esi, offset dword_4070C2
.text:00405AFD                 push    5
.text:00405AFF                 pop     ecx
.text:00405B00                 repe cmpsd
.text:00405B02                 setz    al
.text:00405B05                 and     eax, 0FFh
.text:00405B0A                 leave
.text:00405B0B                 retn    4
.text:00405B0B sub_403055      endp

To make it short: the input string address is passed into the ESI reg, then the EDI is pointed to an empty buffer that will contain the encoded input string, and then, DWORD by DWORD (lodsd) the input string is encoded, up to 5 DWORDs max (20 bytes). At the end, the encoded input string is compared with the original encoded password, thats located at VA 4070C2:

The encoded original password:
0FBE0BB50h, 0D16C80CCh, 716786EDh, 3B77A739h, 493A8A5Ah

After the compare, depending on it's result, a certain value is returned. And the rest you already know :)

It looks very simple at the first sight, however, two things might make us think a little longer:

.text:00403063                 pushf
.text:00403064                 xor     dword ptr ds:[esp], 100h
.text:0040306C                 popf


.text:00405AEF                 nop
.text:00405AF0                 nop
.text:00405AF1                 nop
.text:00405AF2                 nop

The first code sets the Trap Flag (CPU Step Mode). I first thought of it as some anti-debug trick and ignored, but, as I'll reveal in the moment, I was extremely wrong.

The second thing are the four NOPs, lying there, looking silly - why the hell does someone put 4 NOPs in the middle of the function? Is there some code injected runtime? Or some other evil trick executed?

And now we return to the TLS callbacks, or, to be more precise, to the TlsCallback_0 at VA 40233A. The first thing that comes into sight is the "loading imports" string, and there are more such strings here and there, that quite well explains what happens in the code. The Callback function is short, and it just calls a few functions, from which the most interesting one is sub_40120E.

In the mentioned function we see, inter alia, a call to CreateProcess functions with DEBUG_PROCESS flag and a debugger loop that handles i.a. EXCEPTION_SINGLE_STEP event that is generated by the Trap Flag setting code that we analyzed a few lines ago! The pieces finally start making sense! Let's sum up what we know until now:

- when we execute the crackme it never opens any window, instead, it creates another process from its executable and acts as it's debugger!
- so we are talking 2 processes:
- a child process - that shows the window, checks the password, and sets the Trap Flag
- a parent process - that stays in the debugger loop and patiently awaits the TF activation

The bad thing about these things is that we cannot debug the child process using debugger API (sorry OllyDbg... but stealth debuggers like Obsidian or ring 0 debuggers will work of course, however, there is no need to use them anyway).

Let's look into the SINGLE_STEP event handler:

.text:00401302                 mov     eax, [CONTEXT.EIP]
.text:00401308                 mov     edx, [eax]
.text:0040130A                 cmp     edx, 90909090h
.text:00401310                 jz      short koniec
.text:00401312                 or      [CONTEXT.EFLAGS], 100h
.text:0040131C                 cmp     dl, 35h         ; XOR EAX, ...
.text:0040131F                 jz      short action_xor
.text:00401321                 cmp     dl, 2Dh         ; SUB EAX, ...
.text:00401324                 jz      short action_sub
.text:00401326                 cmp     dl, 5           ; ADD EAX, ...
.text:00401329                 jz      short action_add
.text:0040132B                 jmp     short koniec
.text:0040132D ; ---------------------------------------------------------------------------
.text:0040132D action_xor:                             ; CODE XREF: sub_40120E+111
.text:0040132D                 sub     dword ptr [CONTEXT.EAX], 2
.text:00401334                 jmp     short koniec
.text:00401336 ; ---------------------------------------------------------------------------
.text:00401336 action_sub:                             ; CODE XREF: sub_40120E+116
.text:00401336                 add     dword ptr [CONTEXT.EAX], 1
.text:0040133D                 jmp     short koniec
.text:0040133F ; ---------------------------------------------------------------------------
.text:0040133F action_add:                             ; CODE XREF: sub_40120E+11B
.text:0040133F                 xor     dword ptr [CONTEXT.EAX], 10101010h
.text:00401349 koniec:                                 ; CODE XREF: sub_40120E+102
.text:00401349                                         ; sub_40120E+11D ...

The mechanics of the above code works like this:
- from the child's EIP address four bytes are acquired
- if these 4 bytes are 90909090 (4x NOP - rings a bell ?) nothing is done (== the single step mode is turned off)
- else, the TF flag is reset (TF is turned off after each 'trigger')
- and then, the first instruction byte from EIP is checked:
-- if its 35 (XOR EAX, imm32), then an additional SUB EAX, 2 is emulated/made (BEFORE the XOR instruction)
-- if its 2D (SUB EAX, imm32), then an additional ADD EAX, 1 is made
-- if its 05 (ADD EAX, imm32), then an additional XOR EAX, 10101010h is made
- and the child process execution is resumed (for one instruction that is - since the TF flag was reset)

So, the cipher method has additional subtracts/additions/xors that are not visible in the listing (because they are performed by the external debugger!)... Cunnningg :)

OK! We have all the pieces to solve this mystery! There are a couple of actions we can now take, for example we could invert the listing, or make a brute force - I decided to do the later.

In such case, we'll have to start by copying the whole function containing the cipher, and then, using some regexps or sth, insert additional dec+dec, inc or xor instructions (it's best to do this in this order). After that we can remove the TF setting, and recompile (nasm it a good choice).

The final cipher procedure: esetcode.nasm

Now we create a simple C/C++ brute force that will:
- load the compiled cipher into memory
- fix some addresses here and there (to the original key and input/output buffers)
- and, using the fact that there are only 4G of combinations minus the unprintable cases, runs the cipher in a loop

Because the encrypted password DWORDs are independent, we can search in one loop for a solution to all (5) password DWORDs. Such a brute force looks like this (the code is ugly, and was written with haste during the compo, so don't expect miss-code 2009 ;p):

#include <windows.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

char buf[1024 * 1024];

DWORD a[5];
DWORD b[5];

 FILE *f;
 f = fopen("code", "rb");
 fread(buf, 1, sizeof(buf), f);

 int (__stdcall *func)(const char *a) = (typeof(func))&buf;

#define P1 0xd
#define P2 0x34B8
#define P3 0x34BD

 DWORD v12 = (DWORD)a;
 DWORD v3 = (DWORD)b;

 *(DWORD*)(P1 + buf) = v12;
 *(DWORD*)(P2 + buf) = v12;
 *(DWORD*)(P3 + buf) = v3;

 puts("done patching"); fflush(stdout);

 static unsigned char brute[40];

 DWORD myin;

 for(myin = 0; myin != 0xffffffff; myin++)
   *(DWORD*)brute = myin;

       if((myin % 0x01000000) == 0)

   if(brute[0] < ' ' || brute[0] > '~') continue;
   if(brute[1] < ' ' || brute[1] > '~') continue;
   if(brute[2] < ' ' || brute[2] > '~') continue;
   if(brute[3] < ' ' || brute[3] > '~') continue;

   func((const char*)brute);

   if(a[0] == 0x0FBE0BB50 || a[0] == 0x0D16C80CC || a[0] == 0x716786ED || a[0] == 0x3B77A739 || a[0] == 0x493A8A5A)
     char asdf[8];
     *(DWORD*)(asdf) = myin;
     asdf[4] = 0;
     printf("%.8x (%s) == %.8x\n", myin, asdf, a[0]);

 return 0;

Now let's compile (g++, I'm using typeof()), run, and... (the amount of dots may differ from reality ;p)

done patching
.................................20276e69 (in' ) == 716786ed
20756f59 (You ) == fbe0bb50
........................................6b6c6174 (talk) == d16c80cc
..6d206f74 (to m) == 3b77a739

As one can see it didn't find the last DWORD (since I excluded \0 from the "charset"), but that's not important. Now we have to arrange it in the right order, and guess the last 3 letters: You talkin' to m ->  You talkin' to me?, and it's done! :)

And thats it!

P.S. The original CONFidence compilation: confiesetcrackme.zip


2009-05-28 02:53:56 = Ron Bowes

My friend linked me to this post and, instead of reading the spoiler, I fired up my reverse engineering tools to see what I could come up with. After a couple false starts and confusing detours, I discovered what was going on. Unlike you, I chained together a few shell commands and totally inverted the listing. So, I put the numbers in and out pops the result.

It was a very cool challenge, and I learned some neat tricks from it (like what TLS means, and a bit about debugging).

If you're curious, all the code I used, as well as my .idb (for IDA 4.51) file and whatnot, is here:

A step by step writeup of what I ended up doing (including the hole I painted myself into and some 'fail!' and 'crash!' messages is here:

See ya!
2009-05-28 07:59:54 = Gynvael Coldwind
@Ron Bowes
Hi ;>

Great work! Thanks for sharing the path / source code you've used, it's very interesting (you seem to use some tools / tricks I thing I should try out sometimes) ;>

Take care ;>
2009-05-28 16:10:09 = Ron Bowes
Thanks! I have no real training in reverse engineering -- I've barely even read about techniques. Everything I do is pretty much derived from trying things out and seeing what works.

Same goes to you @ techniques, though! Every time I read about how somebody else did something, I learn a bit.

I like how you used the bruteforce technique, actually. I had originally considered doing that, and had planned it all out. Then I actually looked over the code and realized it'd be super easy to reverse the operations, and way cooler, so I did that instead. I'm glad to see that both ways worked!
2009-05-28 20:38:52 = Gynvael Coldwind
@Ron Bowes
Additional great work to you then ;>
I've considered making an inversion, but I was a little afraid I would miss some AND/SHL/SHR or similar instruction that is irreversible (I didn't have time to check all the instructions ;>), so I finally decided to do brute force ;>
But as you say, it's good to know there is more then one way to do it! ;>
2009-05-29 01:54:29 = Ron
Heh, well, I ran this command across it before I started anything:

$ cat assembly_code | cut -d -f1 | sort | uniq -c
252 add
540 lea
252 rol
252 ror
612 sub
360 xor

And it gave me the list of just 6 instructions, 5 of which can easily invert and one of which doesn't have to be (xor). Then, I used a shellscript to do the replaces automatically. Having a background in Linux and shellscripting helped me a lot, obviously -- I can process text files really fast when I need to.

That being said, when I actually ran it the first time, I was sure I would miss something, and that I'd have to start combing through line by line. I had the benefit of having unlimited time to work on it, though (I'd estimate I spent 4-5 hours total, across two days).
2009-05-29 09:07:01 = Gynvael Coldwind
Haha great command, I'll have to remember this one ;>

Take care ;>
2009-05-29 16:51:33 = Ron
By the way, can you recommend any other 'crackme'-type programs? I know there are a ton of them out there, but I'm sure some are better than others.

2009-05-31 20:55:46 = Gynvael Coldwind
I enjoyed cracking Shaker 2 (by bart) very much ;>
I'm not sure if bart still has it online somewhere, so I'll mirror it for now:
Have fun ;>
2009-06-01 16:46:18 = Ron
Oh wow, I don't even know where to start on that one. That's awesome! I'm looking forward to giving it more than a cursory look. :)
2009-06-01 20:02:36 = Gynvael Coldwind
Good luck! ;>
Let me know when you finish ;>
2009-06-13 07:37:39 = Ron
I haven't been able to spend a lot of time on this, what with work and life getting in the way, but I actually went head down yesterday and today. And not without encouragement -- I GOT IT! :D

A bit of proof that I have a working Keygen:
'R' = 18144
'Ro' = 3156144
'Ron' = 50433984
'This' = 808001648
'was' = 51388480
'way' = 51388576
'too' = 51413360
'much' = 821559376
'fun!!' = 59536723
':)' = 3198224

(I love logging in with the smiley!)

Here is the key function (no clue how well code will come out here, but I'll try!):
int calc_forward(char *str, unsigned int code)
unsigned int eax = strlen(str);
unsigned int edx = 0;
unsigned int ebx = 0;
unsigned int esi = code;
unsigned int i;

for(i = 0; i < strlen(str); i++)
edx = (edx & 0xFFFFFF00) | (str[i] & 0x000000FF);

edx = _rotl((((edx ^ eax) + 0x7b) ^ 0x3db) + 0x159, 4);
eax = eax ^ 0x34403;
ebx = ebx ^ edx;
esi = esi ^ ebx;

return esi;

To go the other way, I wrote a bruteforcer. I'm sure it's possible to run that function backwards, but I don't know how.

I took tons notes as I went along, since you're probably interested in how I managed to finish this. But, unfortunately, they're a disaster right now. It's also way past my bedtime. I'll post a link to my notes tomorrow, after a good rest. I couldn't resist replying ASAP, though!

2009-06-13 19:56:39 = Ron
All right, all rested up I cleaned my notes and threw everything onto my svn server. Check it out:

You'll find code and everything there. If you want to go straight to the good stuff, try:

That's my step by step notes that I took as I was working. You can see the wrong/right paths I took there. I wrote it as I went, so it's pretty complete. The source for my key (that I already posted, but that turned kinda ugly) is here:

Let me know what you think! And also, let me know how you solved it.. I'm really curious how close I was to any kind of a standard approach. My approach was to use a lot of debugging, but it could very easily have been defeated. I'm lucky that this was, in the end, a rather simple crackme -- if it was complex, PLUS the obfuscation, it would have been nightmarish. :)

Oh, and if you have any more cool crackme's, let me know! This was great fun in my spare time.
2009-06-14 18:30:27 = Gynvael Coldwind
Great work!!! ;>
Thanks for the solution, it's always good to learn some new points of view ;>
I've solved this crackme with GDB and a trace'ing script hehe, so basically a similar way to your, except the tool was different.
My friend j00ru has also used a similar way, only using OllyGDB ;>

Anyway, I'm afraid I don't have any more links to crackmes for you, since I don't did do many of those.
Maybe some other reader could provide some links for the crackme-hungry Ron over here ? ;>>>

Oh, you can take a look at the Hacker Challenge (hackerchallenge.org) crackme's, however I have no idea where are they published ;<

Take care! ;>

2009-06-14 21:10:26 = Ron
Cool, thanks!

Do you know the person who wrote that crackme? I'm curious how it was made in the first place.. like, was it written like that or was it obfuscated later?

In any case, that's cool about not knowing more crackme's.. when I'm bored one day I'll see if I can find a good one. If I do, should I send it your way? Or do you have other things going on?

2009-06-14 22:44:43 = Gynvael Coldwind
Yeah, I know the guy (hehe that crackme was the reason I met him btw ;>). If you could send me you e-mail to my e-mail (gynvael at coldwind _ pl) I'll send you his e-mail (huh, complicated ;D)

Anyway, if you find any good crackme, let me know ;> I'm always interested in good crackmes ;>

2009-06-18 02:07:09 = Ron
For what it's worth, me and my friend abstracted this a bit, and managed to remove the bruteforce aspect from the equation. Now you put in an arbitrary string, and it instantly (O(n)) returns the result.

Updated solution:

Add a comment:

URL (optional):
Math captcha: 1 ∗ 2 + 6 =