Category Archives: programming

Its really only in the last six months I’ve realized this, and I’ve been an indie for twenty+ years and coding for 39 years, so yeah…this took a while to sink in.

I am frustrated on a CONSTANT basis by the lack of productivity of almost everybody in the universe. I am especially irritated by the low productivity of most people in game development, and most indie devs. I almost never read about the development schedule of a game, (mostly through post-mortems, interviews or chatting to actual humans), without being shocked at how long it took to do stuff.

For most of the time, I have attributed this to an attitude. I work pretty much every day, and for most of the day, although my schedule these days is deliberately lighter than the early years. I’m prone to going out for lunch or to coffee shops, but then I’m prone to working all day Saturday and Sunday, so YMMV. I also often reply to forum posts, youtube posts, blog posts and emails in the evenings from my laptop. I’m often thinking about code when I’m not writing it.

Because of this, I find talking to people with a less work-centric attitude to be infuriating. It boggles my mind how long it takes most devs to add what seem like easy and simple features to games. I am constantly told that I am woefully inefficient because I don’t use unity, but still seem easily capable of working faster in terms of adding features & content than the very people who berate me for not using such productive tools.

So yup, I often think such people are just lazy. Or do not have the same attitude as me, or do not realize just HOW HARD it is to compete in this industry. In other words I think that their mindset is less focused, and its a personal weakness on their part, because yup…i’m a bit obsessed.

But now..I’m thinking there are two other things that explain the disparity better.

First thing: Lack of distractions. I have 3 cats, and live with my wife and these 3 relatively-low-maintenance pets, but no kids. I have a hobby of playing the guitar, which I make myself do a bit each day, but thats it. I am not having to take time out to walk the dog, pick kids up from school, drop kids at school, answer questions from kids, sort out other stuff for kids, walk the dog again, and so on. My wife is a writer, so has the same introverted ‘happy to be alone with a keyboard’ daytime work schedule as me.

Nobody ever phones me, unless its an elderly relative. I have a call screener device to prevent phone spam, and we live in the middle of nowhere. Nobody knocks on our door trying to sell us anything. There is very little noise. Its the perfect set up for zero distractions. If you possibly can do ANYTHING to reduce the distractions in your day, do it.

The second thing: experience.

This is the big one. I’ve been coding for 39 years. Thats an AGE. When I first started learning computer programming, this person was US president:

Image result for jimmy carter

Yup, exactly.

That means any silly mistake you can make when designing code…I’ve done in thirty times. 95% of my conversations with fellow devs when I’ve hit a bug go like this:

“Could be a memory-bounds issue…?”

Me: “Nope”

“Could it be that you deleted the object?”

Me: “Nope”

“…Maybe its a multi-threaded synch issue?”

me: “Nope”

…and so on.

Now that sounds super arrogant, like I think I’m the bees knees at C++. Actually I am not. I am not that good an all-round programmer *at all*. I am VERY good at learning in excruciating detail about the elements of C++ that I use, and nothing else. Because I work for myself I have no marketable need to be an all-rounder. I don’t need to learn ‘agile’ or ‘scrum’ or ‘.NET’ or RubyOnRails or whatever the hell jobs ask for this week/month/year. Its irrelevant to me, so I can be VERY good at VERY few things. This is hugely efficient.

Plus… again, trying to put my arrogance in context here… language proficiency is language proficiency, whether its English or C++. C++ is way less forgiving than English, but still…how good at English were you when you had been speaking it for just five years…versus thirty years? Hardly an exact comparison I know, but I think its a good mental exercise. I get better at C++ every year, but in a way that is not exactly how you would think:

I do NOT know more ‘clever tricks’ than a newcomer to C++. I do NOT have a better memory of the syntax of C++ than a newcomer. I do NOT type *that* much faster. I do not make use of a wider range of the standard C library than anybody else. I don’t do any of those things. What I *do* better, is that I have just learned from my mistakes.

A lot of mistakes.

I used to take the odd coding test in job interviews back in the day. These tests are good for one reason: to see if the candidate has any clue about syntax. Thats pretty much it. The amount of code required otherwise renders the test pretty much useless.

The trouble with C++ is that it attracts hotshot coders. These are people who think a super-complex algorithm, or the algorithm that uses the most clever combination of features will somehow get them more sex/money. This is predictable and sad, but not useful in terms of real productivity.

The best code, is the combination of three things:

Simplicity, Performance, Readability.

A lot of really, really good code looks fairly boring, because boring is often simple, fast and readable. The worst possible insult you can get from a senior/lead programmer with experience is this:

“That looks a bit over-engineered”

Its truly a damning insult, but you only really realize how insulting it is after about thirty years of writing crappy code. I wish I knew of an easy way to help people fast-forward those thirty years and develop the skills you have at the end of it, without those thirty years but I don’t think I can. The only advice I can offer is this:

  1. Write as much code as you can. Not over-engineered nonsense, but just code a lot. Put the hours in. At least the thousand obviously, but likely way, way more.
  2. Get a job with a really experienced coder and ask for criticism of your code. Only someone who works with you all the time will read enough of your code to really give you structural, high level advice on why your code sucks.
  3. Read code-complete at least twice, if you have not done so already.
  4. Get cats not dogs. Cats don’t need a walk.

Hope that helps someone :D

Website optimization in 2020

January 02, 2020 | Filed under: programming

Sooo… in a random moment of surfing a few months ago I encountered an article of the webp format and how it was faster, and how it was a Google thing, and they therefore wanted you to use it. I knew I had a server move coming up (long story) so delayed worrying about it until now…

Basically webp is like a super-amazing improved replacement for PNG that is MUCH more efficient. Full details here, but for example one of the files I converted to webp went from 942k to 189k which is not to be sneezed at. I still cannot tell ANY difference when I look at both images. Sadly wordpress is too useless to upload webp, but here is one embedded:

…and here is the png:

…exactly.

So with this in mind, I replaced some of the larger images on the Production Line webpage with webp equivalents to speed up the loading. This IS WORTH DOING, but its also worth remembering that some Luddites may be using stupidly old browsers that cannot cope with webp, and you need to also have the option of a png for these people. You can do this with some magic modern html like so:

<picture>
	<source type="image/webp" srcset="images/thumb.webp">
	<img src="images/thumb.png" width = "1000" height="563" >
</picture>

That basically says ‘show this webp image, unless you don’t have any idea WTF that is, in which case here is an old fashioned png. All of the attributes for your image still go in the src bit.

That got me a nice speed bump, but some test done both with googles site checker and also the popular web speed test showed I was mainly slowed down by third party stuff, specifically humble bundle widget and youtube embeds. (I embed 2 large youtube videos on that page). This is annoying, but after a lot of fiddling I found a reliable way to get around the slow youtube stuff.

What I did was have 2 identical sized elements on the page for each video. One a ‘panel’ and the other a ‘vid preview’, which was basically a big thumbnail made by me (webp obviously) with a fake play button to simulate youtube. The code in the actual page body looks like this:

<div id="panel">
<table width="100%" align="center" cellpadding="0"cellspacing="0">
	<tr>
	<td align="center" width="1000" height="563">
	<iframe id="trailer_youtube" width="1000" height="563" src="" frameborder="0" allowfullscreen></iframe>
	</td>
	<tr>
</table>	
</div>
				
<div id="vidprev">
<table width="100%" align="center" cellpadding="0" cellspacing="0" onclick="myFunction()">
	<tr>
	<td align="center" width="1000" height="563">				
		<picture>
		<source type="image/webp" srcset="images/thumb.webp">
		<img src="images/thumb.png" width = "1000" height="563" >
		</picture>						
	</td>
	<tr>
</table>	
</div>

In practice what this does is say ‘here is an embedded iframe called ‘trailer_youtube’ with NO source. And here in the same place is a big phat image. BTW if we get clicked call myFunction()’.

Then at the top of the page in the header we add some code:

<style>
#panel, .flip {
  font-size: 16px;
  text-align: center;
  color: white;
  margin: auto;
  z-index:1;
}
.vidprev
{
z-index:2;
}
#panel {
  display: none;
}
</style>

…which sets the z index (bottom to top stacking) of the two panels, and then we need some actual code for when the thumbnail is clicked on, also in the header:

<script>
function myFunction() {
  document.getElementById("trailer_youtube").src = "https://www.youtube.com/embed/IhGTKBAC94c";
  document.getElementById("panel").style.display = "block";
  document.getElementById("vidprev").style.display = "none";
}
</script>

…that code basically grabs the youtube panel, sets it visible, and assigns it a proper valid youtube link, handily deferring any connecting to youtube.com until we need to. it also hides the thumbnail. The result is a MUCH faster page load (roughly half the time).

In addition, I used some javascript called ‘lazy sizes’, to make the loading of some items lower down the page asynchronous, so they wont even get loaded until the visitor scrolls down. source:

<picture>
 <source type="image/webp" data-srcset="images/resources.webp" class="lazyload">
<img data-src="images/resources.png"  class="lazyload">
</picture>

and that requires an extra include:

<script src="./js/lazysizes.min.js" async></script>

The result is pretty good, and raises Google’s estimation of my site speed quite a chunk. That will be good for SEO with Google, and they are basically the only search engine that counts so…yay :D. here is the full waterfall chart:

So we now have an official coming soon page for the new Production Line expansion pack (Design variety pack). Here it is in all its amazing html glory:

https://store.steampowered.com/app/1174730/Production_Line__Design_Variety_Pack/

Obviously the most exciting part of the page is the ‘add to wish-list’ button, which i thoroughly encourage people to do, as gossip among indie devs is that having high wish-list numbers converts into valve sending you nicer chocolate at Christmas, or something like that (I forget the details). Actually the best thing for me to do is probably embed the steam widget thingy:

I have no idea why there is a scrollbar on that widget. I think its safe to blame the mess that is wordpress…

Anyway at the moment the store page is not translated into each language but I’m getting that done now. All the actual content is done, and tested in game, and the new cars look lovely. Its purely cosmetic, so don’t yell at me if you can’t afford it for ruining game balance or whatever. I read that epics cosmetic DLC earns them a bazillion dollars and I’d like to retire eventually (ha!.. will never happen), so somewhere in that paragraph is my reasoning for adding content to the game…

On less business-y levels… I’m tracking down some ultra-rare but annoying Production Line bugs right now. One is a thing where very, very rarely, sounds stop streaming, or the music stops (after a good number of hours). I am digging into this, but its super hard to pin down.

Another bug is related to an error message in logs (which is now harmless…but bugs me) relating to shaders, and some visual artifacting. I discovered that the two different systems I was using to set and unset shaders may potentially have come into conflict, so I fixed that abominable code architecture by ensuring the game only has one possible system for turning shaders on and off, and hopefully now there can never be a conflict or a shader ‘stuck’ on. This will all be in the next patch, just before the DLC release.

Oh…and expect more Democracy 4 update goodness soon

There are no comments yet

I thought I might try another of those ‘live blog’ entries where I go through my thought processes as I do some code profiling to speedup the route-finding slowdowns on super-huge Production Line factories. Here is the map I’m analyzing usage on.

I’m loading the map, waiting 20 seconds, then making a change to the resource route layout, then letting it run another 20 seconds, and taking a look at the function level profiling snapshot using aqtime. This save gamne has 8,994 slots in it (which is massive) and uses a custom map.

Here is the function breakdown for the GeneratePath() code which seems to be a major cause of any framerate issues when changing routes:

I probably need a refresher in my mind as to how this code works so… Lets look at when I change a route by (for example) placing down a new conveyor. This makes the following calls:

			SIM_GetResourceConveyors()->RefreshExits();
			SIM_GetResourcesRoutes()->PurgeAllImpossible();
			SIM_GetResourcesRoutes()->InvalidateRoutes();
			SIM_GetResourceObjects()->OnRouteAddition();

RefreshExits is trivial, taking on average 0.002ms, so not a problem, PurgeAllImpossible() doesnt show up in the profiler, but it basically sets a lot of flags to be false, and calls a sub function that also is presumably too quick to show up, so likely not a culprit.

InvalidateRoutes() is likely the culprit. It tells every slot, and every stockpile that it needs to begin verifying its current cached route table in case something has changed.

Finally OnRouteAddition goes through every intransit resource object and tells them they need to verify their route(over the next two seconds) in case a newer route is available, or the current one just got deleted. This is also too quick to show up.

So it looks like all that route invalidation is the slowdown. But why? The actual function takes 0.2ms, which is slowish…but not noticeable when a frame is 16.0 ms. Its the delay over the next few seconds that causes the problems… Basically every slot with a stockpile calls PrepareToVerifyBays()…

void SIM_PlaceableSlot::PrepareToVerifyBays()
{
	BayVerificationPending = true;
	int limit = BAY_VERIFICATION_INTERVAL;
	if (APP_GetPerformance()->DoRoutesLimitFramerate())
	{
		limit *= 2;
	}
	BayVerificationTimer = GRandom::RandomChoice(limit);
}

So in English, this code tells the slot that it needs to verify its nearest import bays, and to do so at some random time over the next 60 frames (1 second). A globally calculated value works out if we have a poor framerate, and if we are verifying bays, and thus notes that given this PC, this map etc…we need to allocate twice as much time as usual (4 seconds) to verify those bays…

Thus we have maybe 4 seconds max (4,000ms) over which to spread the effect of all this route re-calculation. That might mean up to 4 seconds of lag, which may seem a short time, but if you are watching a resource item getting delivered in the game, and it travels down the wrong route for more than four seconds…you would be perplexed, so its a reasonable target. How to make it faster?

Bays are verified every frame. a check happens every frame to see if a verification is pending, and if it is, a function gets called. That function is pretty huge and its called SIM_ProductionSlot::RecalculateNearestBay(). In my profiled sample it was called 2,200 times and on average takes 2.8ms. OH MY GOD. Almost all that time is spent inside SIM_ResourceConveyorManager::GetNearestBays(). This function itself does some fancy multithreading, spinning off taks to a lot of threads (8 on my PC) each of which processes a list of routes, eventually calling SIM_PathFinderManager::FindPath(). This gets complex so lets look at a multithreaded analysis of it in vtune:

This actually looks pretty efficient and packed, so i think that the real answer to speeding this up is not speeding up the actual route-verification, but culling the cases in which I actually even bother verifying a route. For example…Here is a simple map with production slot stockpiles at A and D, and import bays at B,C,E,F…

Both A and D keep a track of the shortest route along conveyor belts to the 2 nearest import bays. Note not every tile is a route, only each one marked with conveyors, so we have the following.

Now lets say I add a new conveyor belt tile to improve the routing from A to its two nearest bays…

Now as mere humans, its pretty clear to us that as soon as possible, we need to get A to recalculate those two routes it has to its nearest bays, because there is now a shorter route from A to B (but actually not C). Its also pretty clear that D’s routes (to E and F) are totally unaffected. Right now… my algorithm is dumb as fuck, because it tells A and D ‘OMG something changed! redo everything!’, which is a massive waste of resources. The problem is…How exactly does a better algorithm look?

This is one of those things that sounds simple but actually is not. If I learned computer science at school I’d likely be better at this, but I’m self taught. Here is my current thought process:

The route from D to E is 4 tiles. The distance from D to the new tile (pointed at by the arrow) is greater than 4. Thus there is NO way that my route from D to E can possibly be affected by this change. Thus I can leave D as a slot that keeps its old fashioned routes.

So an algorithm that made this optimization would be something like:

For Each Slot
bool bcheck = false
int max_dist_bay = GetMaxDistToBay()
For Each changed tile
If CrudeDistanceToTile <= max_dist_bay
bcheck = true

So its worth trying to see if that helps…

…And I tried it! I even wrote some debug output to verify that the number of slots that got verified each time was below 100%. Something I didn’t realize was that obviously for changes taking place right in the middle of a map, the majority of the map ends up being covered by this algorithm, as the distance for slots to the center of the map is often about the same as the distance to the nearest importer. However, when making changes nearer the edges and corners of the map, a way smaller percentage of the slots need verifying, sometimes just 2-4% of them.

Even if I just assume that we average 50% of slots being verified instead of 100%, this represents a 2x speedup in the game during these events, which should be super noticeable. Of course the only way to be sure this feeds through to user experience is to stick the new algorithm on a togglable switch and watch the frame rate counter and…

YES

It boost the framerate a LOT. Obviously its still spread out over the same amount of time, but the total amount of recalculating needed is way lower so… obviously we get a big framerate boost.

This is NOT a perfect optimised way to do it, but you can bet its already roughly twice as fast, which is great. I need a better way to work out what slots are needlessly recalculating, AND I need to ensure I can use similar techniques on situations that cause a recalc by other means, such as when stuff is deleted but…its definite progress.

I’ll probably work on the deletion case tomorrow, do some more extensive testing, then try it out in the 1.80 preview build (unstable beta) on steam.

In a recent conversation with fellow indies about how I can make production line look better, someone effectively said ‘why are you not using mip maps’, and at first I laughed because, LOL, I use mip maps, and then I remembered that the geniuses at Microsoft decided that D3DXSaveSurfaceToFile should not generate mipmaps so actually…the game didn’t use them for many of its props (the stuff in texture atlases, basically).

So obviously I immediately thought what a dork, generated mip maps and…

it kinda looked way worse. Or did I? I have stared at the pixels so much now I am starting to see things. Here is the game as it currently looks (mip maps enabled in engine, but most of the car graphics and prop graphics not generated with any). (click to enlarge)

And here is it with mip maps enabled. (click to enlarge)

From a distance does it look any BETTER to you? I’m not sure I can really tell much of a difference until I zoom in. Here is evidence of how blocky the current one looks when zoomed in…

versus the mip-mapped one.

Which obviously looks better, but its not *that* simple, because the mip-mapping also creates some artifacts. here is a montage of the current, and lots of mip-mapped styles, with different settings from mip map creation filters, sharpening and softening etc. I just can get those door lines to vanish…

Which possibly means that I need to adjust how those cars are being drawn, or means I have not yet found the perfect set of render options for generating dds mip maps. There is also the possibility that the way I render out my car-component atlases (with a black background) is bleeding onto the mip maps at lower levels, and that this is where the problem is.

Of course all of this is absolutely *a matter of opinion* and thus really annoying, as I am a data-driven guy and like hard facts,. so stuff like this is where I fall down a bit. I don’t like pixellated graphics ( I despise the look of minecraft) but on the other hand I also REALLY hate over-blurred images, which make me think my eyesight is failing or I need new glasses. Its also very tempting to give in to the mistake of zooming in to a static image and declaring the best one to be the one where a zoomed in screenshot looks best, which is WRONG because obviously when zoomed right in, the mip maps are irrelevant anyway. here is the current (non mip mapped) car zoomed in.

Which leaves me in a bit of a quandary. Is it even worth continuing to fuss over this stuff…does anybody really notice/care? Or should that time be better spent on adding new features to the game?