So its been quite a while since my last post. I stopped working on my NASM OpenGL implementations for a while after encountering too many issues with maths. I will return to that eventually but I have lost interest for now.
The latest project I have been working on is OpenRCT2. The project aims to rewrite Roller Coaster Tycoon 2 in C code by looking at the assembly instructions. After spending so much time playing in NASM I felt this was right up my alley.
This post is about how I went about decompiling the draw rain function. I haven’t quite decided the name of the function yet so it will be called sub_684027. Earlier this week I noticed that someone was working on the rain drawing functions. He decompiled all of the functions up until sub_684027. This is a snapshot of what was decompiled.
I had a quick look in ollyDbg at the 684027 address and realised it didn’t call anything else. This is always a good sign as it generally means that the function will not be too difficult to decompile. I made some initial notes on decompiling it and uploaded the changes. You will notice that the code is horribly formatted and does not compile. This was just a very rough draft of the function to try work out how it was working.
The next stage was to convert it into actual compilable code. Normally at this stage I would try iron out all of the bugs in the code. It just so happened that I didn’t have access to my dev machine that day so this code was never actually tested. When I did test it I noticed it was still a bit incorrect hence why I produced this version. Eagle eyed readers will notice I have also removed a bunch of global variable assignments that just created backups of the functions parameters. One part I always enjoy is searching for the address’s of every global variable to see if we have already labelled them. It was during this searching exercise that the drawpixelinfo struct appeared. This struct contains information about the screen and has a pointer to the actual pixels that are displayed. As soon as I spotted this it started to simplify the code greatly. I can immediately see which variables are for height, width, left, right, pointers.
At this stage I normally iron out every single possible bug I can find. I encountered an odd bug at this stage that caused rain drops to stay on the screen. It took me approx 2 hours to work out what caused this. After fixing all of the bugs now came the fun part of figuring out names for every variable and generally increasing readability. This is the code after the final cleanup sweep. There are still one or two variables that have not been renamed but I had to stop at this point as it was getting late. One not very obvious change is modifying the following lines:
eax = *((uint8*)ebp);
edi &= eax;
RCT2_GLOBAL(0xEE7870,uint32) = edi;
uint8 pattern_x_space = *pattern++;
uint8 pattern_start_x_offset = edi % pattern_x_space;
Its not immediatly obvious but edi&(eax-1) is the same as edi%eax. EAX in this situation will always be 2^n. I only know this from playing about with the code for a while. This shortcut has been cleverly created to save on doing a divide instruction that at the time of RCT2 was expensive. For us it just makes the code harder to read so there is no need for it.
The very final version of the code has now labelled all of the variables. I’ve replaced the while loop with a for loop and I’ve tried to group together anything that operates on the same variable. At this stage I can start labelling the input parameters and assigning an RCT2_ADDRESS name to each of the left over addresses.
Eventually when all of RCT2 has been decompiled we will go over all of the global variables and either remove them or convert them into normal C variables. I hope this brief guid on decompiling helps people get interested in OpenRCT2.