Game Design, Programming and running a one-man games business…

Re-thinking painters algorithm rendering optimisation

Before I even start this post, I should point out that currently the project I am coding is for fun. I actually really ENJOY complex programming design challenges, unlike many people, so the comment ‘this is done for you in X’ or ‘there is middleware that solves this without you doing anything in Y’ is irrelevant to me, and actually depressing. Whatever happened to the joy of working on hard problems?

Take an imaginary game that has a lot of entities in it. Say 500. Each entity is a sprite. That sprite has an effect drawn on top of it (say its a glowy light) and a different effect below it (say its another glowy light). The objects have to be rendered in a sensible way, so that if one object passes over another (for example) the glowy lights ‘stack’ properly, and you don’t see aberrations where the lights from something are seen through something else. Here is a visualisation:

In this case red is the ‘underglow’ grey is the sprite, yellow is the overglow. The bottom right duo shows the issue where things have to be rendered correctly. In a 3D game with complex meshes and no antialiasing, you just use a zbuffer to draw all this stuff, and forget about it, but if you are using sprites with nice fuzzy edges (not horrible jaggy pixels, you really need to use the classic ‘painters algorithm’ of drawing from back to front for each object. There are clear performance problems:

A naïve way to code this would be my current system which handles each object in 3 draw calls. So we set up the blend mode for the underglow, and draw that, then set a normal blend mode, draw the sprite, then a glowy blend mode to draw the top glow, then repeat for every other entity. The trouble is, with 500 entities, thats 1,500 draw calls simply to draw the most simple part of the game…

Unfortunately this only scratches the surface because there may also be other layers of things that need drawing on top of each entity, and between drawing each sprite. However there is a major optimisation to be had…. <drumroll>. Actually my game really works like this:

Where there is a grid, and <generally speaking> the entities stay in one grid box at a time. WITHIN that grid box, all sorts of stuff may happen, but items in the top left grid box will not suddenly appear in the bottom right grid box. In other words I can make some clever assumptions, if only I can find a way to generalize them into an algorithm.

For example I could draw the underglow in one object in each grid box all together in a single draw call like this:

So this is suddenly one draw call instead of 8. I then do the same for the sprites, then the overglow, and then do the other object in each grid square as a second pass (basically a second Z layer. Assuming 8 grid squares, 2 per square, 3 passes, thats 48 draw calls for naïve method and 6 for the new method. Pretty awesome! However, thats oversimplifying things. Firstly in some cases there are 16 entities in each grid, in others just 1. Secondly, in some cases items ‘spill’ over into the next grid square, so I need to ensure I do not get an anomaly where 2 objects would overlap just slightly and thus z-fight when drawn…

Like everything in game coding, the simple code design always accelerates towards spaghetti, but I would like to do my best to avoid that if possible…

In order to decide what is drawn at what time, I basically need 2 pieces of metadata about each object. Item 1 is what grid square it is in, and item 2 is the Z value of that object, where I can cope with say 16 different Z ‘bands’ for the whole world, meaning a group of 16 entities in a square will definitely be drawn with 16 different draw calls, and thus be painter-algorithm-ed ok.

I also want to do this without having to do insane amounts of sorting and processing every frame, for obvious efficiency reasons…

So my first thoughts on this are to handle everything via Z sorting, and make the list of entities the de-facto Z sort. In other words, when I build up the initial list of entities at the start of the game, I have pre-sorted them into z order. So instead of a single list of randomly jumbled Z orders. I get this situation, where the black -number is the Z position, and the blue references the grid square:

So my now pre-sorted object list goes like this:

A1 B1 C1 D1 E1 F1 G1 H1 A2 C2 F2 etc..

I can then do batches of draw class so my actual code looks like this

  • DrawUnderGlowFor1()
  • DrawSpritesFor1()
  • DrawOverglowFor1()

The nature of the game means that generally speaking, Z order can be fixed from the start and never change, which simplifies things hugely. The bit where it becomes a real pain is the overlap of objects to adjacent grid squares. I would also like a super-generic system that can handle not just these fixed entities, but anything else in the game. What I really need is a two layered graphics engine:

Layer 1 just send out a ton of draw calls, probably with some sort of metadata. This is like a draw call, plus relevant texture, plus blend state for the object. This is done in a naïve way with no thought to anything else.

Layer 2 processes layer 1, does some analysis and works out how to collapse draw calls. It can calculate bounding boxes for everything that is drawn, see what overlaps with what, and what can be batched, and then decides on a final ‘optimised’ series of calls to directx. This is done before any textures get set or draw calls are made. It then hands all this off to ideally a new thread, which just streams those draw calls to directx, while the other threads get on with composing the next frame.

I have attempted such a system once before in the past, but I was a less experienced coder then, and full-time working, and on a (self-imposed) deadline to ship the game, rather than obsess over noodling with rendering engine design. So hopefully this time, I will crack it, and then have something super cool that can handle whatever I throw at it :D. Also the beauty of this design is it would be easy to bypass stage 2 and just render everything as it comes in, so I could even hook up a hotkey to watch the frame rate difference and check I’m not wasting all ny time :D.

In praise of long-form content and deep work

While idly trying to find something of interest to read one evening I found myself at medium, looking for tech news articles. I didn’t end up reading anything, because I was suddenly struck by how awful it was that every article has a tag line underneath it telling us how many minutes it takes to read it. The examples I saw were ‘3 minute read’ ‘5 minute read’ ‘6 minute read’.

This is staggeringly depressing.

Imagine the situation where you want to read something, which means taking new information into your brain, and you are only able to commit to a 3 minute session of reading, not 5 minutes. Unless you are the US president, or Elon Musk, you probably don’t actually have to save the vital 2 minutes difference. So this metric is not really being given to be in any way helpful, its being given as a kind of reassurance, or in the 6 minute case, presumably as some sort of urgent trigger warning. Its no great revelation that social media is warping out minds, but I really don’t think we appreciate just how bad its getting. When you cannot make a safe assumption that someone’s attention will be held for even 6 minutes, we are in real serious trouble.

The world is scarily complex. Way more so than when I was a schoolkid. The sheer tonnage of stuff you should know about now, that wasn’t even a thing back then is scary. Primarily, its the way tech works, so you have to know how computers work (roughly), what apps are, what wireless internet is, how to charge a phone/laptop, how to use a trackpad or touchscreen, how to swap between apps, how to type (actually not considered important when I was 5 years old), some idea of what email and spam are, how to pick a password, what a username is, what a browser is. How to swipe between items on a phone, what an update is…

…and its not like the real world is simpler either. My parents were very unusual because they had traveled to Europe! but never beyond. Their parents had only left the country to invade/liberate others. As a result, knowing a lot more about an incredibly connected world is now much more important. I only knew the concept of ‘forename/surname’ reversal from the bajorans in star trek:deep space nine. I had no idea that was a thing. I only really understood the complexity of honorifics last year. As a child, the most exotic food I encountered was spaghetti.

In 2024, its just assumed that anyone in the UK is familiar with the culture and cuisine of pretty much the entire planet. Most stuff we buy comes from outside the UK (this was NOT true in the 1970s), most people travel outside their home country. This is awesome, and beneficial, and culturally enriching, but it makes like so much more COMPLEX.

Plus science and technology marches on, and culture becomes more complex. Watch a TV drama from the 1970s or 1980s. Something you may notice is the plots are relatively simple, the narrative is linear. The same is true of most literature. The ‘fractured narrative’ is a relatively recent trend. When ‘Lost’ debuted on TV, it was radically different, complex, inter-twined stories which required serious attention. Modern TV drama relish their super-complex plotlines, flashbacks, deception and feints. This was not always the case. At the same time, every academic field has got more complex. I studied Economics before Behavioral economics was even invented. It didn’t ‘replace’ what I learned, it augmented and expanded it. This diagram is apparently series 1 of ‘dark?

Life’s complex as fuck these days

But meanwhile we seem to be systematically destroying our capability to do what it is sometimes referred to as Deep Work. What is Deep Work?

Professional activity performed in a state of distraction-free concentration that push your cognitive capabilities to their limit. These efforts create new value, improve your skill, and are hard to replicate.”

In other words, pretty much anything thats really worthwhile is Deep Work. And to do deep work, you need the exact opposite of the social-media fueled doom-scroll focus-destroying mindset that every big tech company seems to be pushing us towards. Perhaps the worst examples are TikTok and Instagram. They seem hyper-focused on super-short, non-interactive consumption of tiny tiny nuggets of dopamine. By comparison even X is better, as it now supports long-form posts and at least seems to be more focused on interaction, and thus conversation/interrogation.

I find myself deeply worried about my own ability to concentrate on long form text, or to be engrossed in any long form activity. I can go play the guitar for 30 minutes, but after that I start to wonder if I have any email. As an 18yo kid, I would happily go 8 hours playing scales to a metronome. No distraction required or desired. I didn’t stop until desperately hungry.

Quite a few years ago, on a whim, I started to read Winston Churchill’s war diaries. All of them. Its a lot. When you are the leader of a major military state in world war 2, you have a LOT to write. And Churchill wasn’t famous for brevity. The volumes now sit proudly in a bookcase, a rare example of an entire set of tiny-text serious books that I made it through. Reading them was a bit of a slog, but I’m glad I did, especially considering how much time I now seem to just wase entirely checking twitter or scrolling through slashdot stories I will never read.

I do genuinely worry that we are already splitting into a society of basically 2 different groups. A tiny, tiny group of people who are working out how everything works, and doing serious research and work on making things better, and then a vastly larger group of us who just doom-scroll, or spend our days sending memes and emojis, and pretending to do work. Increasingly the idea of spending serious time to learn very complex new skills sounds ‘lame’ compared to trying to be witty on twitter in the desperate hope of ‘going viral’.

At this point you are wondering if there is any way I can steer this blog post back to either video games or solar panels, and believe it or not, I can, and much faster the Winston Churchill would have done, let me assure you. I’m going to talk about a specific videogame : Dwarf Fortress.

I’ve only tried to play Dwarf Fortress once. It was about 5 years ago. I didn’t get into it at all. Its not for me. I LOVE the idea of it, and love that it exists, but its really not for me. But thats not important. All you need to know about Dwarf Fortress is that it is ludicrously, hilariously, insanely detailed. Its a city builder for dwarfs, where the UI (originally) was just a bunch of text. It doesn’t matter, all you need to know is it was basically a hobby project, where the creators lived off donations from obsessive players. There are way more details here.

This is where I give you a dopamine hit by skipping narratives to talk about the plight of the average indie game developer trying to get press attention. I know a lot of people who try this, and their strategies are very different, but in many cases there is a reliance on social media, and a desire to use the correct hashtags, and construct posts just right so you can strike it lucky and go mega viral. This Never Works. And BTW, even when it does work, it doesn’t work. I once made a stupidly viral tweet with about 80,000 likes. Do you think I was able to steer that into more sales of my games? (spoiler: no).

Dwarf Fortress was the absolute opposite, its two creators basically just stuck their heads down, and kept working and improving and developing and updating the game for years and years and years. It was first released in 2006, so that’s 18 years so far, or much longer than World War II. To work on a single game for so long is impressive. It requires amazing attention to detail, incredible passion, and people who are perfectly happy to say to people literally decades later ‘yes I am still working on that thing’. And the very very best bit of the story? When they finally put their game up for sale for actual money on steam, they made bazillions of dollars overnight. It was a decades-long overnight success.

The thing is, if you say to people now “you need to work really hard, all the time, probably for decades, and you will then probably have the same result”, absolutely nobody will even consider this as an option. These days the idea of coding your own game engine is considered some freakish, weird, obsessive things that only super-autistics gluttons for punishment like me or jon blow would consider. It used to be the norm. At one point, everyone had to do it, and nobody died as a result.

I am currently coding a new game. The game may never be released. At the moment its still a secret, and mostly a hobby. Its VERY similar to another game I made, but because I dislike getting old code to recompile, and because frankly I LOVE to code, I’m basically recreating that old game, and its engine, entirely from scratch, instead of reusing the old code. This means thousands of small improvements, that I’d never make if I just ‘updated’ the old code. I enjoy doing it, and regret nothing. I am also quite glad I can still do it.

We desperately need to retain the ability to focus long and hard on complex stuff. Its an essential skill, and I suspect almost everyone reading this is struggling with it, in the face of social media, and tech companies desire to kill our attention spans. I urge you to fight back. Pick a seriously long book and read the whole thing. Or pick a new technology or skill and commit to serious long stretches to master it. It is incredibly fulfilling. I have spent a mad number of hours learning to play the guitar. At this point in my life it serves no real ‘purpose’ career-wise, but I still get a lot of joy from knowing I have that skill and can still do it.

Well done, you read the whole thing :D

Solar farm update. Fencing. Trenching. Lawyers…

Amazingly this is the first update of the year, and its May. This is absolutely ridiculous beyond words, but I might start getting really ranty and angry if I get drawn into it, but suffice to say that for absolutely zero justifiable reason, nothing dramatic has really happened on the site for months.

Its all due to paperwork. basically the energy company (the people connecting the farm to the grid, at vast expense, all billed 100% to me) decided that they needed a sub-lease from me to put their substation on the freeholder (farmer’s) land. This is fine. I have a lease agreed over a year ago now with the farmer. They want their substation on my leased plot. Fine. Go for it. They tell me that they will write the lease, and they have proposed a signing agreement of £1.

Yes £1.

But ok, whatever, I have basically no say in this. They write the lease, I read it, and agree and sign it within 24 hours. Then we move on. Meanwhile, unconnected to this, I get a nice diagram showing how the farm is wired:

And then…apparently the energy company are not convinced I have the right to sign a sub lease. they demand to see my lease with the freeholder. I say fine, I send it. They then claim there is no evidence the lease is valid because we didn’t register the lease with the land registry. That means I have to hire a lawyer (for the first time in my life, after 27 years of running a video games company that trades globally), to fill out some paperwork with the land registry. I do this, and it takes weeks, because the lawyers don’t believe who I am and demand identity documents ad-nauseum, despite it being a matter of legal public record that I am the director of the company.

Oh well…

Anyway, I do all of this, and they then say that they don’t believe the freeholder is who he says he is either. AFAIK this freeholder has owned the land for decades (probably generations), and lets remember at this point that there is NO DISPUTE about the terms of either lease between any party. They insist on me printing an A3 copy of a map, in color, with my signature on, and physically mailing it to them. I have to find a print shop to do it. I do this. They then want the freeholder to travel physically to a lawyer to prove his identity. There are then other complications which I won’t detail, but at every possible turn, I am delayed, ignored, then handed yet more impossible and ridiculous demands.

They charge me about £5k in “extra legal costs” associated with the pain of me not personally having recruited another lawyer. This is ridiculous, and you can only begin to imagine my rage. I make multiple angry phone calls and send a battery of very angry emails. Meanwhile, we are heading towards summer, but nobody cares, not the energy company, and certainly none of the many lawyers involved.

Unrelated, We put a fence up around the site:

I phone the energy company lawyer to complain, when I’m told some bureaucrats need to physically sign pieces of physical paper that needs to be sent by courier, which might take another week. I insist that if its not done immediately I will physically get in my car, and make a 4 hour drive to their office, pick up the paper myself and fucking drive it to the home of the person who needs to sign it, and sit on their doorstep until they do.

I think they are finally understanding what I think of this process. Of course meanwhile, we hurtled through the 1.5 degrees barrier which was the tipping point for global catastrophic climate change, and UK farmers start to warn people they cannot grow food now. Nobody seems to give a damn.

But woohoo, we have dug some trenches and put some ducting in, ready for the AC cabling that goes from the inverters to the site substation, which is MY substation, where all of the AC stuff gets wired together ready to be handed over to the DNO, who then use a transformer and step up the voltage. Here are the very rare pics of actual progress:

So yeah, there have been no updates because of all this. Words cannot express my anger, despair and frustration at this. Its absolutely insane. Do not ever, ever EVER complain to me that your energy bill is too high!

I am hoping to eventually get a DATE for when the overhead line is moved, for which I will travel to the site to witness, seeing as though it cost more than my first house. I will exhaustively take photos of this event that I have been trying to arrange for almost 3 years.

Democracy 4 comes to consoles!

Hello party members!

It is with great pleasure that we announce that Democracy 4 will be coming to Nintendo Switch, PlayStation, and Xbox consoles on 5th June 2024. With the help of our partners at Auroch Digital, Democracy 4: Console Edition has been built the from the ground-up with controller play in-mind to bring that feeling of Democracy 4 that we all know and love to home consoles.

Democracy 4: Console Edition is available to pre-order and wishlist via the following links:

Nintendo Switch – Coming Soon

PlayStation – https://store.playstation.com/en-us/product/UP5595-CUSA44561_00-0098238017880105

Xbox – https://www.xbox.com/en-US/games/store/democracy-4-console-edition/9N84BGD6VZ8K/0017

In Democracy 4: Console Edition, you will:

Choose to lead one of the following countries: USA, UK, Japan, France, Canada, Australia, Spain, Germany, Italy, or South Korea

Select cabinet ministers that align with your vision for the future of the country

Introduce new laws or tweak existing ones to carry out your plans for change

Deal with random events that can either increase or decrease your favourability with the public

Lay out your manifesto as election time approaches and aim to deliver a campaign that will take you to victory!

Auroch Digital are no strangers to porting PC titles over to home consoles, as they have done with titles such as The Colonists, Megaquarium, and Ogre, as well as developing their own titles such as Warhammer 40,000: Boltgun, Mars Horizon, and Brewmaster: Beer Brewing Simulator.

Having the chance to bring Democracy 4 to home consoles was something which intrigued the series’ creator, Cliff Harris. Cliff says: “As the developer behind Democracy 4, I’m genuinely thrilled about its upcoming release on consoles through the work of Auroch Digital. It’s been a long journey focusing on the game’s development for PC, and seeing it expand to a wider audience on consoles feels incredibly rewarding. The prospect of reaching more players and allowing them to experience the intricacies of political simulation on different platforms is both exciting and gratifying.”

Porting Democracy 4 over to home consoles is something which excites Auroch Digital’s Studio Director, Tom Rawlings. Tom says: “Bringing the game Democracy 4 to console has been a personal mission of mine – not only it is a great strategy game that deserves to have more opportunity to be played, but it is on a topic of central importance to people too. So, I was really pleased when

Positech Games were open to working together to make this happen. The team have worked hard to make it possible, so do check the game out, run a country, and try not to get assassinated!”

Democracy 4: Console Edition is available to pre-order and wishlist on PlayStation and Xbox consoles now, with the Nintendo Switch version available to pre-order and wishlist in the near future. The game will be released on 5th June 2024.

For more information visit: https://www.aurochdigital.com/democracy-4-console-edition

Sanding the woodwork

For eight years I worked in a boatyard.

Thats not the start of a novel, or a poem, I really did, which might seem weird if you know me as a software developer or game designer. But its true! It was mostly part-time, but multiple days a week, and it was crushingly hard work in ways I wont even begin to list here. This is a post about motivation, creativity and getting stuff done. Like a lot of my blog posts it will seem on a tangent until I attempt to bring it all together at the end. So stay tuned :D.

Almost all of the boats we worked on were clinker-built Edwardian Thames Skiffs. Basically 20-30 foot wooden rowing boats. The details don’t matter, but one of the most common jobs was to sand the woodwork for the whole boat, and give it a nice new coat of varnish. This was the easiest job we did, but also at the same time it was the most boring. Apart from anything, it was also very incompatible with the job I had the other 50% of the time which was part musician/session guitarist/guitar teacher. If you have played electric guitar you will know you get very helpful toughened callouses on every left hand finger. They are essential. They are also very easy to totally rip to shreds with sandpaper. I was constantly tying my fingers up with masking tape to prevent this…

Anyway…

One day a fellow worker at the boatyard decided to give me his top tip on how to approach the task of sanding a skiff. This job would take maybe 2 days. I listened with interest as he said “You start off by doing all the really awkward bastard bits that are annoying, and then when you finish them, you realize that the job is done.” He paused. “Because they are all fucking awful bastard bits”. This was funny, but funnier still because I actually thought at the start he was giving me practical advice. Working in a boatyard is grim. We milked any humor there was.

Me sculling(not rowing!) one of the boats we worked on

The point of this anecdote is to try and convey how boring, and hard and more than anything unsatisfying this job was. A wooden boat that has had every surface properly and smoothly sanded in preparation for a coat of varnish is not a massively satisfying thing. Its not like stripping dirt away to reveal the ceiling of the Sistine chapel, its not something that passers-by stop to gawp at and exclaim ‘well thats some decent thorough sanding work my good man!”. Nope nobody gives a fuck. Why did I keep doing this for maybe 16 hours? Because I needed the money to buy food.

£2.50 an hour

Yes, I’m old, and its probably better paid now. But anyway, its a very very long boring day, and there is NO satisfaction from the actual work whatsoever. It did not require any brainpower whatsoever. I did it for eight years. Thats a long time. Also, if you think that you would mostly be on twitter or having a coffee break: no. No mobile phones then, and 2 allotted tea breaks at 10.30 and 3.30PM. Apart from that, you better be fucking working or you’d be yelled at and maybe have sharp things thrown at you with great force.

Ok, I get it, this sounds like a four yorkshireman sketch, and a typical boomer ‘in my day’ whine about how work was harder in my day etc. But actually its totally not that. Its about understanding that some parts of the best jobs in the world are also like sanding down the woodwork.

When your WHOLE JOB is doing tedious unexciting unfulfilling drudge work, you kind of just accept it and get to fucking work. In a way, this is EASIER than many creative and fulfilling/rewarding jobs. In a creative job (like game development), there is a ton of really cool shit. When I made Gratuitous Space Battles, I spent a lot of actual work time stepping through space battle clips from Star Wars, deciding on the colors of laser beams, and designing futuristic space weaponry. It was awesome. But in-between all that cool shit, there was also stuff like coding a UI to save/load ship deployments. Coding a system to adjust the UI to fit at different screen resolutions. There was code to handle different fonts for each language. There was code to handle online challenges and error handling from PHP and SQL. This was the woodwork-sanding stuff.

When you can choose to do the cool shit, or sand the woodwork, almost everyone does the cool shit, and just leaves a bit pile of sanding to do ‘at some point’. When you run out of cool shit you are then faced with a huge ton of tedious crap you have to do in order to finish the project.

Don’t do this

Take my sarcastic boatbuilder buddy’s advice. Sand the fucking woodwork. When you start work, when you are motivated, when you are excited, do some of the grunt work. Code the error handling, optimize the rendering. Check the game runs on min-spec. Do the multi-language support. Do the steam API implementation. Do the options menu. Do the level editor. Do the modding support. Do the stuff you know you wont want to do later. This is the way. Then when you feel your motivation flagging, go decide what color the laser beams should be.