Category Archives: programming

Coders talk a lot about technical debt. here is a rough definition:

“Technical debt” (also known as design debt or code debt) is a concept in software development that reflects the implied cost of additional rework caused by choosing an easy solution now instead of using a better approach that would take longer.”

It’s often used in the context of justifying redoing some work. As in… “we have a lot of technical debt. The best thing to do is rewrite all this properly from scratch”. To some extent that can be justified. The engine for your games is effectively your house foundations. Don’t start building a house without ensuring you have decent foundations, and all that sort of thing. The problem with this mindset is that people (or rather…computer programmers) often get into the situation where they want to re-code everything from scratch again, and again, and again.

The thing is, the more experienced you get, the more you realize that the big, messy, patched, complicated looking smorgasbord of code that you are working with probably HAS to look like that. it probably looks like that for a reason. Its not pretty, its not clean, it doesn’t make for nice diagrams but it WORKS and thats important.

If you think thats not true I point you towards the apache web server (literally a pun on ‘a patchy web server’) and Microsoft windows, an absolute towering pile of code mess that is the very definition of technical debt. How can I possibly view these as good things…?

The simple answer is: Practicality and commerce. Apache is not the most beuatiful work of art in terms of highly structured, elegant, perfectly designed code, and windows is literally the state of the art in horrible hacks. (Like this one, to ensure sim city still runs). To quote:

They reported this to the Windows developers, who disassembled SimCity, stepped through it in a debugger, found the bug, and added special code that checked if SimCity was running, and if it did, ran the memory allocator in a special mode in which you could still use memory after freeing it.

And it you are someone who considers code to be beautiful, who likes to describe themselves as ‘a software architect’ instead of ‘hacker’ (or code monkey), then stories like this will fill you with rage but…

…Apache sure is installed on a lot of servers, and a huge number of PCs still run windows. Why? Because accepting that your patchy, confusing, held-together-with-string piece of code is actually MORE reliable than new stuff you could code today is actually a sign of coding maturity.

I do not open source my engine, and TBH nobody would likely use it if they did, but part of my reasoning is that it would be embarrassing. I couldn’t even decide on a naming convention. At one point I didn’t care, and I had classes called things like IniLoader. Then I thought it would be cool to have a ‘G’ prefix to indicate game engine, so I have GArc GFile, GHashTable. At some crazy point I had my classes in all caps, hence DEBUGENGINE.h. Some of the code separates functions with /************/ some of it with //////////////////////////////. Pretty much the only coding standard was a member variable naming convention with FirstLetterCapped.

By far the two biggest embarrassing pieces of dodgy code in the ‘more or less part of the engine’ GUI code are the following. Please try not to laugh.

Stupidity one: I have a GUI_ButtonBase class, and a GUI_WindowBase
class . They are DIFFERENT things, with no connection, despite a button obviously being a derived class of a window in any sensible system. Thus my GUI_WindowBase class contains separate lists of buttons, and child windows. *sigh*.

Stupidity two: All my windows have a virtual function call CheckClick(int x,int y), which they process, then call on any children. You would think that this took the x,y of the current mouse position and operated on it, but for some reason I gave up bothering and use helper functions like IsMouseInside() to check such stuff, which ignores the passed-in x and y. I still go to all the bother of passing x,y, down the hierarchy, but its ignored. I’m just paranoid about it.

Sure I could fix this, I could rewrite my button code, junk that dumb x,y, thing and rename all my classes sensibly while I’m about it. I would then need to spend several days changing all the code in Production Line, AND checking it, and I *ASSURE* you, I would miss something. There would be a new crash bug. I wouldn’t spot it, and maybe after a dozen players encountered it, someone would tell me about it. I would have achieved nothing, but frustration for my players, and a smug feeling that my code was less embarrassing.

I’m not giving into that. My current code WORKS, it has less bugs than it *ever* has. My current game (Production Line) is more stable than ever, and more stable than any game I have ever shipped. Going back and changing parts of my engine because they look messy, or because modern code fashions have changed or because there is a new API or code pattern would be MADNESS.

Don’t be too hard on your old code. Sometimes age brings wisdom.

So with all the current stuff in the news about Unity/Improbable and the counter offers from unreal, and the big epic vs steam thing going on… I think there are probably a few developers out there who think to themselves… ‘I wish I *did* have my own engine, then I wouldn’t have to worry about ANYBODY else’s code. The problem is… if you are used to unity or similar systems, you might have no idea where to start right?

The windows basics:

To have an app that runs under windows you basically need just two functions, WinMain and a Windows ‘procedure’ function that handles windows message. A stripped down winmain looks like this:

int APIENTRY WinMain(
                     HINSTANCE hInstance,
                     HINSTANCE hPrevInstance,
                     LPSTR pszCmdLine,
                     int nCmdShow)
MSG msg;

gInstance = hInstance;
GetGame()->InitApp();

//main program message pump loop//////////////////////////////////
while(1)
{
if( PeekMessage( &msg, NULL, 0, 0, PM_NOREMOVE ) )//any for us?
    {
    if( !GetMessage( &msg, NULL, 0, 0 ) )//if so get em
        {
          return msg.wParam;
        }
    else
        {
            TranslateMessage(&msg);
            DispatchMessage(&msg);
        }
    }
else
    {
        if(BActive)
        {
            GetGame()->GameProc();
        }
        else
        {
            Sleep(1);
        }
    }
}
return msg.wParam;
}

The only two exiting parts are InitApp() where I create all the Directx3D stuff, and load in any data I need, and GameProc() which is basically the games main loop. The window proc looks like this:

LRESULT CALLBACK WindowProc(HWND hWnd,
                                 UINT uMsgId,
                                 WPARAM wParam,
                                 LPARAM lParam
                                 )
 {
 switch (uMsgId)
     {
     //main switch statement for handling messages
     case WM_DESTROY:    //end the application
         PostQuitMessage(0);
         GetGame()->ReleaseResources();
         return 0;
         break;
default://default behaviour
        return DefWindowProc(hWnd,uMsgId,wParam,lParam);    
      }
}

Again the only exciting thing there is ReleaseResources() which basically closes down all that directx stuff and releases all the textures and any memory I allocated.

meanwhile inside that InitApp() thing I need to do this:

int width = GetSystemMetrics(SM_CXSCREEN);
int height = GetSystemMetrics(SM_CYSCREEN);

Which gets the desktop res for me to create a window, and show it to the world:

gWnd = CreateMainWindow("Production Line", gInstance, IDI_ICON1, IDI_ICON1, WindowProc, width, height, bwindowed, bborderless, 0, 0);
UpdateWindow(gWnd);
ShowWindow(gWnd,1); 

After that I initialise all the directx stuff, which I have written about a decade ago, its pretty easy to grab all that from the directxSDK. BTW that CreateMainWindow function is mine too, and looks like this:

HWND CreateMainWindow(char* appname,HINSTANCE hInstance,int IDI_SMALL_ICON,int IDI_TINY_ICON,
                       WNDPROC proc,int width,int height,bool bwindowed,bool borderless,int left,int top)
 {
     WNDCLASSEX wcex;
HICON icon = NULL;
HICON iconsmall = NULL;
if(IDI_SMALL_ICON == NULL)
{
    icon = (HICON)LoadImage(NULL,"data/icon.ico",IMAGE_ICON,0,0,LR_LOADFROMFILE);
}
else
{
    icon = (HICON)LoadImage(GetModuleHandle(NULL), MAKEINTRESOURCE(IDI_SMALL_ICON), IMAGE_ICON, 64, 64, 0);
    iconsmall = (HICON)LoadImage(GetModuleHandle(NULL), MAKEINTRESOURCE(IDI_TINY_ICON), IMAGE_ICON, 32, 32, 0);
}

wcex.cbSize           =    sizeof(WNDCLASSEX);
wcex.hInstance        =    hInstance;
wcex.lpszClassName    =    appname;
wcex.lpfnWndProc      =    proc;
wcex.style            =    CS_CLASSDC | CS_DBLCLKS;
wcex.hIcon            =    icon;
wcex.hIconSm          =    iconsmall;
wcex.hCursor          =    NULL;
wcex.lpszMenuName     =    NULL;
wcex.cbClsExtra       =    0 ;
wcex.cbWndExtra       =    0 ;
wcex.hbrBackground    =    (HBRUSH) GetStockObject (BLACK_BRUSH);

RegisterClassEx(&wcex); 

int flags = 0;
if(bwindowed)
{
    if(borderless)
    {
        flags  = WS_POPUP;
    }
    else
    {
        flags = WS_OVERLAPPED |  WS_CAPTION | WS_SYSMENU | WS_MINIMIZEBOX | WS_BORDER;
    }
}
else
{
    flags  = WS_POPUP;
}

HWND gWnd = CreateWindow(appname,appname,flags,
    left,top, width, height,
    NULL,//GetDesktopWindow(),
    NULL, hInstance, NULL);

return gWnd;
}

TBH the only fiddly bit is finding a program to actually create a decent windows ico .ico file in all of its myriad different sizes. Plenty of them exist, but its something you probably did not realise you need. I also have to use a batch file and a little command line thing to stuff the finished icon into the exe, so that it shows up in windows explorer as well as when the window is launched (weirdly these are different things…).

Thats pretty much all of the ‘initialisation’ code you need to create a windows app. Obviously you then need an actual graphics engine, but thats not *too bad* to write yourself if you are not coding something that is huge and requires amazing performance or tons of shaders.

In 2D, everything is pretty simple. You just need a sprite class (I have a Base Sprite, which is just 4 vertexes and a way to manipulate them by scaling, changing UV values, positioning them etc), maybe a textured sprite (the same but with a directx/opengl texture pointer), and if you want to optimise stuff you need a system that handles vertex buffers.

In general I just have one (big) vertex buffer that I stuff with sprites during the main game render loop, and then call draw() on it every time I need to change the texture or one of the render states.

Input stuff is pretty simple. You should just hook into the windows WM_LBUTTONDOWN messages and others like WM_CHAR, and do some processing on them. Way simpler than using middleware for all that. If you need to know the current state of a key you can use this:

bool CBaseInputManager::KeyDown(int key)
{
bool down = (bool)(GetAsyncKeyState(key)& 0xFF00);
return down;
}

I *do* use middleware for sound, but you can get pretty affordable, pretty simple sound middleware from lots of places these days. I have not updated my sound middleware for a LONG time. I don’t do any fancy sound processing so why would I? Playing an ogg file or streaming one…is not complex.

Now obviously if you have never coded outside unity, then there is a LOT of stuff you take for granted that you would have to now write some code for, but you only need to learn these things once. Loading in a file, or browsing a folder for files, is just a few lines of code. Even writing an absolutely bullet proof ini file loader that operates efficiently and correctly and without bugs is only 913 lines of code by my count, and thats a LOT of whitespace, helper functions and wrappers around it.

If you start writing an engine early, when you are still doing hobby games, preferably simple ones, you will find it quite easy to scale it up as you make more complex games. The code samples in this blog post have been stripped of their error checking, and some game-specific checks and extra processing to make them clear on first reading.

Basically every game I make involves me adding some new functionality to my engine. Production Line was the first game where I had to create an isometric object class, and isometric renderer (basically just a system to sort by Z and then render…), an animation compression system and some metric reporting and user-survey stuff, but that was probably just adding an extra 3-4% to the size of the total engine.

I wrote the multi-threading stuff ages ago (Democracy 3 if I recall), although I seriously improved it this time. The original windowed GUI system was coded for Starship Tycoon (OMG). Most of the directx stuff comes from Gratuitous Space Battles 2. The current text renderer dates back to Democracy 2 for its initial version. The vertex buffer code started in GSB 1.

Don’t get me wrong, there is a LOT of code in an engine, and it DOES take a fair bit of time, but its a big, long term investment that definitely pays dividends. I don’t have to worry if unity supports X or Y, or if it conflicts with Z, or if they are going to remove it without warning next week (or break it). All the code works in a style I like, and with absolutely zero bloat.

The source folder for my engine is 100 files and 620k total. Its not *that* big. Obviously it compiles pretty quickly and easily.

…and don’t forget you can still code using an IDE (Like Visual Studio) with built in code syntax highlighting and intellisense (I recommend visual assist!). Do not fall into the trap of thinking its unity OR just typing in a command line window without any help!

So my recent adventures in the land of code have taken me to port Production Line to 64 bit. The current 32bit build only allows me to access 2 GB of RAM and although even super large factories can fit in 800MB now, when you really pack things in and put the hours in, it *is* possible to hit 2 GB. With modding & any possible post-release expansion possibilities, there is arguable a need to remove that limit and so here we are.

I’ve basically done 2 days(ish) work on it, and have a release build and debug build 64 bit version of the game that seems to run just fine. It was relatively painless. The 3rd party stuff I use is mostly, Steam, some sound middle ware, Directx and an intel profiler, and all of this has 64 bit support, so the majority of the work has been going through the config for the game and changing include folders and paths to point to 64 bit DLLs and Lib files.

This has been complicated a bit by the mess that the Visual Studio (2013) software makes out of configurations. I can have a debug64 project config that then has the platform set to 32bit, and then god-knows where the exe gets put or which files get compiled and oh-my-god how messy. I think I have finally got close to getting it straight in my head, although I have ended up hard coding exe names and paths and may have to even rename my engines lib file to engine_64.lib to make ABSOLUTELY sure that it is not using the wrong lib file and thus mix-and matching.

I can totally see why people do not want to support both 32 bit and 64 bit versions of the same game, especially given the fact that, unbeknown to me, the mere *existence* of a 32 bit dll in the exe folder will seemingly stop my 64bit .exe launching. I guess you end up with separate folders? What a pain.

In terms of code, it was almost entirely painless. As I suspected, my one pain point was my GUI code for buttons. I have a base button type that takes a function pointer called BUTTONFUNC to execute when clicked. I cannot remember far enough back to work out why, but generally I end up passing an objects ‘this’ pointer as an (int) to the constructor for a button, if I want the button function to access it as data. So I end up writing code like this:

PCheckFree = new GUI_CheckButton(TRANSLATE(“CHOOSE_MISSION_FREEPLAY”), Freeplay, (int)this);

And then in the code that receives it I’d do this:

void GUI_Scenario::Freeplay(int data)
{
GUI_Scenario* pwin = (GUI_Scenario*)data;
pwin->SetType(SIM_Scenario::FREEPLAY);
}

Which is perfectly fine and lovely, assuming pointers are 32 bit and an int is 32bit. However, it turns out that porting to 64 bit is as simple as just replacing both users of (int) with (size_t) which varies based on platform, and voila! problem sorted. I expected this to be the first step in a whole world of nightmares,, but although I have not done serious testing yet, it appears to launch, run and allow me to load in massive save games, so I reckon I’m 95% of the way there. All I need to do know is investigate how all the various stores (Humble, Kartridge, Steam, Gog) handle multiple versions (64 bit versus 32 bit), to ensure I’m not leaving the tiny 32bit minority behind. I guess eventually that will not be an issue.

I’m definitely happy that this seems to have gone smoothly, as it amounts to days of coding and admin and investigation which are essential, but doesn’t make the game noticeably better for players, which is always a worry when the game is still in Early Access. Fun fun fun…

 

 

 

Yeah I know its not long anyway…but its 2018 and on a CPU this speed, why is it not virtually instant?

I just multithreaded the steam app_init so that stuff is effectively ‘free’ now, and its time to see where the rest of the startup time is going. My current measurements from AQTime:

Drilling into this it looks like the easy wins will be inside the Main Menu constructor, as the init3d stuff probably cannot be multithreaded, and involves a lot of disk-bound stuff while I load textures, which is really tough to speed up (I could turn on my pak file code, but I’ll probably only do that just before final release).

Sadly almost all of the main menu constructor code looks like its the loading in of a big png file for my menu background. This is a 2560×1440 png file that is 7.5MB on disk. This is big, but I load in WAY more graphics than that when I do the pre-load textures on all the cars, which are in dds format. I’ll experiment with shifting it to a DDS file. This *should* be way faster, as a png has to be converted whereas a DDS file effectively *is* in the memory format used internally by directx, so its just a straight dump into memory…

That *dopes* actually drop it down to 17.2% (from 21.73%). I suspect there is a further optimisation in that this is currently not a power of two texture, so changing it to be one (as a test) yields…

OMG. its now 1.54%. Resizing that texture seems to take insane amounts of time, and this change knocks 0.5 seconds off my startup time. The next candidate is my sound engine. Multithreading its startup code completely takes it out of the picture too.

MainMenu initialise goes from 116ms to 4ms just by converting a non pow-2 png to a pow2 dds file, and now things look like this:

Total startup time has shrunk from 2.5 seconds to 1.9 seconds. Nothing anybody will consciously notice, but it makes me happier :D. Plus I *do* think that subconsciously people do feel the difference. Snappy start-up times are great for games when you want to have a quick blast, and the perceived responsiveness is bound to make people feel happier about their experience with the game. Plus it means less CPU draw and less battery drain on laptops.

This is for my car factory simulation game Production Line, currently in early access.

 

 

 

There are no comments yet

Not of interest to many, but I made a little demonstration video (using Production Line) to show how I can check out the concurrency of threads in the game using Intels VTune software. Enjoy:

There are no comments yet