Category: programming

Before I even start this post, I should point out that currently the project I am coding is for fun. I actually really ENJOY complex programming design challenges, unlike many people, so the comment ‘this is done for you in X’ or ‘there is middleware that solves this without you doing anything in Y’ is irrelevant to me, and actually depressing. Whatever happened to the joy of working on hard problems?

Take an imaginary game that has a lot of entities in it. Say 500. Each entity is a sprite. That sprite has an effect drawn on top of it (say its a glowy light) and a different effect below it (say its another glowy light). The objects have to be rendered in a sensible way, so that if one object passes over another (for example) the glowy lights ‘stack’ properly, and you don’t see aberrations where the lights from something are seen through something else. Here is a visualisation:

In this case red is the ‘underglow’ grey is the sprite, yellow is the overglow. The bottom right duo shows the issue where things have to be rendered correctly. In a 3D game with complex meshes and no antialiasing, you just use a zbuffer to draw all this stuff, and forget about it, but if you are using sprites with nice fuzzy edges (not horrible jaggy pixels, you really need to use the classic ‘painters algorithm’ of drawing from back to front for each object. There are clear performance problems:

A naïve way to code this would be my current system which handles each object in 3 draw calls. So we set up the blend mode for the underglow, and draw that, then set a normal blend mode, draw the sprite, then a glowy blend mode to draw the top glow, then repeat for every other entity. The trouble is, with 500 entities, thats 1,500 draw calls simply to draw the most simple part of the game…

Unfortunately this only scratches the surface because there may also be other layers of things that need drawing on top of each entity, and between drawing each sprite. However there is a major optimisation to be had…. <drumroll>. Actually my game really works like this:

Where there is a grid, and <generally speaking> the entities stay in one grid box at a time. WITHIN that grid box, all sorts of stuff may happen, but items in the top left grid box will not suddenly appear in the bottom right grid box. In other words I can make some clever assumptions, if only I can find a way to generalize them into an algorithm.

For example I could draw the underglow in one object in each grid box all together in a single draw call like this:

So this is suddenly one draw call instead of 8. I then do the same for the sprites, then the overglow, and then do the other object in each grid square as a second pass (basically a second Z layer. Assuming 8 grid squares, 2 per square, 3 passes, thats 48 draw calls for naïve method and 6 for the new method. Pretty awesome! However, thats oversimplifying things. Firstly in some cases there are 16 entities in each grid, in others just 1. Secondly, in some cases items ‘spill’ over into the next grid square, so I need to ensure I do not get an anomaly where 2 objects would overlap just slightly and thus z-fight when drawn…

Like everything in game coding, the simple code design always accelerates towards spaghetti, but I would like to do my best to avoid that if possible…

In order to decide what is drawn at what time, I basically need 2 pieces of metadata about each object. Item 1 is what grid square it is in, and item 2 is the Z value of that object, where I can cope with say 16 different Z ‘bands’ for the whole world, meaning a group of 16 entities in a square will definitely be drawn with 16 different draw calls, and thus be painter-algorithm-ed ok.

I also want to do this without having to do insane amounts of sorting and processing every frame, for obvious efficiency reasons…

So my first thoughts on this are to handle everything via Z sorting, and make the list of entities the de-facto Z sort. In other words, when I build up the initial list of entities at the start of the game, I have pre-sorted them into z order. So instead of a single list of randomly jumbled Z orders. I get this situation, where the black -number is the Z position, and the blue references the grid square:

So my now pre-sorted object list goes like this:

A1 B1 C1 D1 E1 F1 G1 H1 A2 C2 F2 etc..

I can then do batches of draw class so my actual code looks like this

• DrawUnderGlowFor1()
• DrawSpritesFor1()
• DrawOverglowFor1()

The nature of the game means that generally speaking, Z order can be fixed from the start and never change, which simplifies things hugely. The bit where it becomes a real pain is the overlap of objects to adjacent grid squares. I would also like a super-generic system that can handle not just these fixed entities, but anything else in the game. What I really need is a two layered graphics engine:

Layer 1 just send out a ton of draw calls, probably with some sort of metadata. This is like a draw call, plus relevant texture, plus blend state for the object. This is done in a naïve way with no thought to anything else.

Layer 2 processes layer 1, does some analysis and works out how to collapse draw calls. It can calculate bounding boxes for everything that is drawn, see what overlaps with what, and what can be batched, and then decides on a final ‘optimised’ series of calls to directx. This is done before any textures get set or draw calls are made. It then hands all this off to ideally a new thread, which just streams those draw calls to directx, while the other threads get on with composing the next frame.

I have attempted such a system once before in the past, but I was a less experienced coder then, and full-time working, and on a (self-imposed) deadline to ship the game, rather than obsess over noodling with rendering engine design. So hopefully this time, I will crack it, and then have something super cool that can handle whatever I throw at it :D. Also the beauty of this design is it would be easy to bypass stage 2 and just render everything as it comes in, so I could even hook up a hotkey to watch the frame rate difference and check I’m not wasting all ny time :D.

I recently read about the news that garbage collection support, which was added to C++, is now actually being removed from it. Apparently most people didn’t use it, or even knew it was officially added, so it is no great loss. It always shocks me to read articles about C++ with a version number, because as far as I am concerned, C++ has no version number and never will, in the same way that a language such as English has no ‘version number’. I’m 54 and its pretty rare that I add a new word to my English vocabulary, and its even rarer for me to learn something new about C++ that I start to use in my code.

Back in the early days of modern computing. I worked in IT. My CV was basically: CNA, MCSE. That was it. That was all you needed to earn £54k a year 30 years ago in IT. There were basically 2 big computer systems, run by Microsoft and Novell, and your IT dude ideally knew them both. That was a long time ago now, and the amount of buzzwords and brand names the average IT admin has to put on their linkedin profile is probably quite ridiculous. However, I think its worse in the land of software engineering.

Again, go back a while and you were probably pretty employable if you could just mention C and C++. Then Java became a big deal, then a bunch of other stuff appeared. I have no idea whats cool now, but it feels like Python, Rust are much in demand. Then you have to add all of the recent methodologies. Do you know Agile and Scrum? How familiar are you with AWS? Whats your AI/ML skills like? PyTorch? Do you know the buzzword technologies that will get you hired this year? You better get a job quick, because the buzzword technologies will change every 2 years. Did I say 2? I meant ever year. No sorry, month.

I recently found myself thinking about poetry and code. My wife writes poetry, so I am exposed to this stuff. As a writer, she spends a lot of time… a LOT of time deciding what words to use in a sentence. Its a big deal. Sentence-by-sentence writing is an absolute skill that takes most people there entire life to perfect. Its worth noting that few poems are praised because they use the latest hip words. Good writing is not a matter of having a large vocabulary. Needlessly obscure word-choice is rightly seen as pretentious and alienating.

We really need to take some of that perspective and apply it to code

Take this sentence: “It is a truth universally acknowledged, that a single man, in possession of a good fortune must be in want of a wife.”

Thats considered literary genius, and it is. But its not using arcane language. Every word is commonplace. And idiot could have put that sentence together! but it took Jane Austen, and considerable experience, and huge skill to do it. We do not mock Jane Austen because she could only write in English. We do not mock her because she only wrote from a woman’s point of view. We do not mock her because all her novels were contemporary, in a similar setting, set in a single country, with a linear narrative. We accept all of those limitations and accept that she brings incredible skill to use a limited set of tools to create genius.

Imagine a modern programmer trying to get their first novel published. “English, yup I could write it in English, French, Italian, Chinese or South Korean if you like? I can do all the genres, yup, no problem. I can do first or third person if you like, and I’m familiar with fractured narrative or linear. If you want it funny I can do that, or harrowing, or in short story form too if thats what you are looking for.”

Madness

For some reason, people think that ‘proficiency’ in a programming language is something as superficial as being able to say ‘hello’ or order a beer in another language. This is insane. I am able to say ‘Hello’ ‘Thankyou’ and ‘Sorry’ In Korean, but you wont see me apply for a job writing Korean-language fiction.

If you have under ten years experience in using a programming language, let me be blunt and tell you that you don’t REALLY know that language. 20 years is better. 30+ years is ideal. Do you really think you speak French like a native after speaking if for a few hours a day for a few years? Of course not. Thats laughable. And here is the thing: A mistake in a language can cause confusion and maybe embarrassment, but unless you are a lawyer writing contracts, its not CRITICAL. Miss-using C++ can cause rockets to crash, reactors to overload, and god knows what else.

Why do we accept a superficial understanding of a language that is safety critical, but expect mastery of a language by anyone paid to use English?

I know C++. Thats it. A little bit of php, but a trivial amount. I use container classes and std::string from STL but thats it. A very few macros. My C++ vocabulary, even after 28 years using it, is tiny. The amount of std library stuff I know is very small. And yet… I can type C++ with as much confidence and speed as I type this blog post. In fact I can write C++ faster, with fewer mistakes than I can English. In many ways, I am MORE fluent in C++ than English. I code almost every day, and love it. I feel absolutely that I know what I’m doing, after 28 years, and a subset of C++.

The world is full of people claiming to have that fluency with 12 languages, and they are often literally half my age (I’m 54). This is utter bollocks. None of those people should be allowed ANYWHERE near mission critical code, or any code even tangentially involved with safety or security. I am sure that they ARE doing those jobs, every single day, because they all confidently think they are experts, and the people hiring them do not know any better. It a recipe for disaster, and its why year-after-year, software gets WORSE. Windows 11 runs dramatically worse than Windows 3.11 did, and it does it on hardware ludicrously faster. Skype is running at about 0.1% of its potential efficiency, has scrollbars that do not function as well as windows 3.11 did, and uses easily 100 times the RAM it needs.

Your computer is an absolute trainwreck of clusterfucks crashing into a dumpster-fire of wasted resources. All the people involved in arranging the trainwreck think they are multi-skilled geniuses, but hardly any of them have any real understanding of the code they write.

It doesn’t have to be that way.

We don’t appreciate Picasso based on how many colors he used, or how many styles he knew. We don’t berate any musician for only knowing one style. In Japan, people who make the SAME SUSHI DISH their entire lives, without variation, are considered legends, and experts. Its the norm in South Korea for restaurants to only serve one dish (but do it WELL).

I beg of you: If you are involved in recruiting software engineers, for the love of god only employ people who have real, genuine experience, measured in years but preferable decades, for stuff where you expect them to be able to code from day 1. No, they will not ‘pick it up quickly on the job’. Hiring interns or juniors is different, obviously.

I know I’m an old man yelling at a cloud, but sometimes old people know a lot about the cloud. I’ve been coding since I was 11, and its taken me this long to realize that programming languages should be treated like any other language. It might not be a popular view, but I want to put it out there. Experience really matters.

Maybe there is an easier way to code this, and I’m sure 99% of devs would just unity and copy paste some asset store effect, but I like to code these things myself becausae I like the intellectual challenge, and it gives you total freedom and zero dependencies so here we go:

I just finished coding a ‘shield impact effect’ for Gratuitous Space shooty game, and thought I’d explain how its done. The idea is that the ships are surrounded by energy fields, with various grid patterns, and when laser bullets hit them, they generate a sort of encapsulating ‘wash’ effect over that portion of the shield. To pull this off, I need two things: It needs to be clear where the impact hit the ship, and it needs to feel like it wraps around the target in 3D.

How do you do that in a 2D engine?

The first challenge is getting a graphic that looks like a ship shield in 3D. This is easy. You just get any pattern you want, such as a hex-link pattern:

Then you use the photoshop sphereize distortion filter to give you a nice alpha channel map of a cool space shield:

If I just blap that on top of a spaceship thats hit by lasers, you get a nice effect, but the problem is its non directional, and its pretty clear that its just one image placed on top of another. What I need to do, is to draw this image, but only bits of it at a time, and to wash over the image, revealing it over time.

The way I found was best to do this was to enlarge that texture a lot so it had a ton of empty space, and then use it at twice the size. The reason for this will become apparent, but I’m not working with this:

What I then do, for my shield effect is to use a sort of 2D mesh to wrap around that image over time. That wrap-around effect will originate from the point where the bullet hits the shield. To do this, I get the bullet position, then get the angle from shield center to bullet center, and travel the exact shield radius along that angle from the shield center. That gets me an EXACT location on the perimeter of the shield, even if the bullet is super fast and has already moved inside the perimeter. This is important!

So I now have the impact location, and I want to kind of ‘wash’ the image of the sphere into the players view over time like this:

The red dot is the impact point, the yellow rectangle gets thicker over time as it washes over the sphere. To do this, I ‘virtually’ place the sphere texture on the screen, but do not render it. Its there just as a placeholder for where the ‘full’ sphere would be if it was all revealed. I then calculate how to position that rectangle, and over time I stretch it so it completely covers the target sphere. In order to render the composite image, I render the yellow rectangle, but when I get the UV values of each vertex in the rectangle, I actually look up where they are in relation to my oversized virtual sphere and use that UV value, but the texture of the sphere.

This is why I had to enlarge the canvas the sphere is on, so that those UV values make sense and are still 0 to 1 in both directions. When I do this, I get a cool effect, but its basically a watermelon (shown here with early rubbish shield texture)

To fix that, I just need to set the color values of the ‘inner’ vertices at the top of the rectangle to be fully transparent, and I then get a nice fade effect over the sphere. To be honest, that looked fine, and I was happy with it, but even though NOBODY would notice, something annoyed me…

When you imagine something wrapping around a sphere like this, from a top-down view, you would notice that the speed at which the ‘frontier’ washes over the sphere is non-linear. In other words, I’m not taking any account of the curvature of the sphere itself. To do things right, the speed at which that rectangle washes overs the sphere should vary along the length of the top of the rectangle. To do THAT, I need the rectangle to actually be a tri-strip, so I can curve the top edge. And not only that, the speed at which the center point of that edge moves needs to be a nice curve defined as a sine wave…

In other words I need to achieve this:

That wireframe thing is my yellow triangle. Over time it will wash over the whole shield right to the back. here is two overlapping impacts:

You can see whats going on quite easily in the wireframe version there. The game is way too fast and gratuitous for anyone to notice, and TBH I am still tweaking it.

One of the really fiddly bits was getting that curve just right. I had to start at the left hand top of my yellow triangle, and go across the top edge, noting my progress along there. I then interpolated between the extent to which the ‘height’ of the rectangle came from a fixed point (I picked the top left at the ‘back’ of the shield’) or from my sine-eave inspired non-linear curve of the point that represented the center of the rectangle.

At the end of all that I then basically have a single tri-strip. I then run some separate code to derive UVs from the ‘virtual’ sprite, and just render it. For those who care about the details, here is the code for what I’ve described:

void GUI_ShieldRipple::CalculateVerts()
{
	//place relative to the shield
	float radius = PParent->GetGlowSprite()->Width / 2;

	//we have a tristrip here that is angled at RippleAngle and whose bottom center is at
	//InitialImpactOffset from the current shield center;
	float shieldcenx = PShip->GetWorldPosition().X;
	float shieldceny = PShip->GetWorldPosition().Y;

	//derive bottom center
	float radangle = D3DXToRadian(RippleAngle);
	float cosangle = COS(radangle);
	float sinangle = SIN(radangle);
	//note this is the inverse because the angle is from offset to cen and we want the reverse
//	WorldPos.X += (sinangle * ourspeed);
//	WorldPos.Y -= (cosangle * ourspeed);

	float botcenx = shieldcenx - (sinangle * radius);
	float botceny = shieldceny + (cosangle * radius);

	//now turn 90 degrees to the left to go to the start
	float angletostart = RippleAngle - 90;
	if (angletostart < 0) angletostart += 360;
	radangle = D3DXToRadian(angletostart);
	cosangle = COS(radangle);
	sinangle = SIN(radangle);

	float botleftx = botcenx + (sinangle * radius);
	float botlefty = botceny - (cosangle * radius);

	//invert for right
	float botrightx = botcenx - (sinangle * radius);
	float botrighty = botceny + (cosangle * radius);

	//how wide is each block of the tri-strip
	int prims = (MAXVERTS - 2);

	float chunkwidth = PParent->GetGlowSprite()->Width / prims;

	//now deduce chunkheight, which is basically our travel over the shield at the center point
	//we have non linear progress here, and we deduce that from a cosine wave curve imagined from 
	//Pi to 2xPi
	float cosinput = D3DX_PI + (D3DX_PI * Progress);
	float adjusted_progress = cos(cosinput);
	//now convert to 0 to 1 instead of -1 to 1
	adjusted_progress = (adjusted_progress + 1) / 2;

	//this gives us the center value, of the middle of the sphere
	float chunkheight = radius * 2 * adjusted_progress;

	//get top of the strip above the botleft
	radangle = D3DXToRadian(RippleAngle);
	cosangle = COS(radangle);
	sinangle = SIN(radangle);

	float innerx = botleftx + (sinangle * chunkheight);
	float innery = botlefty - (cosangle * chunkheight);

	//start botleft then up, then to 1 along and down...then up
	float currx = botleftx;
	float curry = botlefty;
	
	float offsetx = (botrightx - botleftx) / (prims/2);
	float offsety = (botrighty - botlefty) / (prims/2);

	unsigned long fullcolor = RGBA_MAKE(20, 234, 221, (int)(255.0f * Intensity));
	unsigned long nocolor = RGBA_MAKE(20, 234, 221, 0);
	for (int n = 0; n < MAXVERTS; n+=2)
	{
		Verts[n].dvSX = currx;
		Verts[n].dvSY = curry;
		Verts[n].color = fullcolor;

		//to get innerx we need to adjust between the full corner value and the middle adjusted
		//value based on our progress
		float progress = n / (float)prims;
		//convert that progress to a nice curve
		progress = sin(progress * D3DX_PI);

		//thats the extent to which we get the adjusted value rather than the base value of   chunkheight being radius
		float newheight = (radius * 2 * (1.0f - progress)) + (chunkheight * progress);
		float newinx = currx + (sinangle * newheight);
		float newiny = curry - (cosangle * newheight);

		Verts[n + 1].dvSX = newinx;
		Verts[n + 1].dvSY = newiny;
		Verts[n + 1].color = nocolor;

		currx += offsetx;
		curry += offsety;
	}
}

Its a lot of C++ to just generate that effect, but I love coding this stuff. I might change it so that the top left and right points of that rectangle are in fact the mid point, so the curve goes flat then inverts. I think that may look better. Also I’ll fiddle with render states and lightmaps to make it fizz more. In the meantime I uploaded a video to twitter showing it with, and without the wireframe. There is a lot going on, and there can be multiple overlapping effects at once. Thats another reason I needed to use a virtual shield texture for UVs, this way everything lines up perfectly, even with multiple impacts:

I've been fiddling with shield impact effects for #GratuitousSpaceShootyGame here is the effect with and without wireframe debug view:

And I blogged about how its done here:https://t.co/2pfl5SxNJq pic.twitter.com/x8Vx3sgGwg

— cliffski (@cliffski) October 23, 2023

Gratuitous Space Shooty Game will be on sale on itch soon