Background: I am coding a 2D space-based autobattler with ridiculous levels of effects called ‘Ridiculous Space Battles‘. I code my own engine, for fun, in directx9 using C++ and Visual Studio.
Because I love coding, yesterday I found myself wondering how well the multithreading in my game was holding up. So I fired up the trusty Concurrency Visualizer in Visual Studio. I love this tool and have used it a lot on previous games. One of the biggest demands on the CPU for my game is particle processing. There are a LOT of explosions, and other effects, and a crazy number of particle emitters (thousands of emitters, hundreds of thousands of particles). Obviously this would normally be a code bottleneck, so I have a task-based generic multithreading engine that handles it. The main thread builds up a bunch of tasks, the threads grab the next task on the list, and the main thread will wait until the task list is empty. If there are still tasks in the queue, the main thread will do one as well. So how did things look?
Disastrous!
So what is going wrong here? I have a generic ‘marker’ set up to show the span of the main thread’s drawing of the game (GUI_Game::Draw()). Inside that, a bunch of unlabeled stuff happens, but I added spans to show when the multithreaded task called UPDATE_PARTICLE_LIST is called. There are two massive things going wrong here. Firstly, there are only two other threads joining in to process the particles, and secondly one of those updates seems to take 20x as long as the other. Worse still, its the one the main thread chose… so its a huge bottleneck. Technically this is still a speedup, but its marginal. How have I fucked up?
Some background to my rendering algorithm is needed: The game has 2 blend modes for particles. A ‘Burn’ mode, that saturates color and is used for fire, lasers, sparks etc, and a ‘Normal’ mode for smoke and debris etc. The particle effects are batched as much as possible, but I cannot mix those blend modes in a draw call. Also, some particles are below the action (the ships) and some above, to give a semi-3D look and make it look like the explosions engulf the ships. So this means particle emitters fall into one of 4 lists: NormalBelow, BurnBelow, NormalAbove, BurnAbove. This is all fine and works ok. In action, the game looks like this:
Because you can freeze frame and scroll around, everything has to be properly simulated, including particle effects currently offscreen. Anyway it all works, and I have FOUR particle emitter lists. So naturally, to load-balance everything, I gave one list to each thread and considered the job done.
BUT NO.
It turns out that those 4 groups are not equal in size. They are laughably unequal. The ‘BurnAbove’ list contains all of the fire and spark emitters on all of the pieces of all of the hulks from destroyed ships, plus sparks from fiery plasma torpedoes, expended drone explosions, and missed or intercepted missile explosions. Thats MOST of the particles. When I checked, about 95% of particles are ‘BurnAbove’. I had 4 lists multithreaded, but they were not vaguely really load balanced.
Once I realized that the solution was theoretically easy, but fiddly to implement and debug. I decided I would add a new load-balanced list system on top. I created 8 different lists, and when an emitter was created it was added to the ‘next’ list (the next value circled round through all 8), and told what list it was in. When it was deleted, it was removed from the appropriate list. Note that ‘deleted’ is a vague term. I delete no emitters, they get put into a reusable pool of dead emitters, which complicates matters a lot…
So in theory I now have a nice load-balanced series of 8 lists that contains every particle emitter that is currently live. The original 4 lists are still valid and used for rendering and blend mode data, but this ‘parallel’ list system existed alongside it, purely to handle load-balancing. What this means is, that a load-balanced-list may contains particles from all 4 render groups, but this does not matter as I am running update code on them, not rendering!
It didn’t work.
Crashes and bugs and corrupt data ahoy. I worked on it for ages, then watched a movie to try and forget it. Then this morning, after some digging, it was all fixed. What actually was going wrong was related to smoke plumes. Because there are a lot of smoke plumes, and they always reuse the same particle config data, they exist in a separate system, updated separately. I had forgotten this! And what was happening was my new load-balanced lists stupidly included these emitters when they should have been kept out of it. The emitters would expire and be deleted in the multithreaded code, then later accessed by the plume code. CRASH.
I worked it out this morning before breakfast! I was very pleased. You might be thinking, what about the only 2 threads thing? LOL I had hard coded the game to use maximum of 4 threads, probably as a debug test. Idiot. I just changed it to be 10 and everything worked:
This is more like it. I wasted ages trying to get the dumb concurrency visualiser to show my custom thread names instead of ‘Worker Thread’ but apparently thats the category. Not much help. FFS show us the thread names! (They work in the debugger). But anyway, that image above is a snapshot inside a busy battle for the GUI_Game::Draw() showing how UpdateParticles tasks get spread over 8 threads. I’m still not sure why that sixth thread misses out on a task, which gets nabbed by the main thread…
Anyway, the point is it works now, and in theory updating particles is 8x faster than it would be with single threading. I do need to apply the multithreading to a lot more of the game code to get the best possible results. I am testing this on a fairly beefy GPU and CPU (Ryzen 9 5900X 12 Core @3.7GHZ and RTX 3080) in only 1920×1080 res. I want the game to look awesome at 5120 res or on a five year old cheap laptop, so plenty more to do.
If for some reason this tips you over the edge to wishlist the game, here is the link :D.