Game Design, Programming and running a one-man games business…

Anatomy of a rare, and weird heisenbug in Democracy 4

Heisenbugs are bugs in code which seem to go away, or change when you look at them in a debugger. They are the worst possible bugs, because they actively resist being found, or even observed by the coder responsible. Any kind of bug that can be reproduced on a developer’s machine, in their development environment can easily be analyzed, stepped through and reasoned with. But heisenbugs are a nightmare. I just (I think) fixed one, and certainly fixed A bug, even if not this one. I thought you might enjoy the process…

First some background. Democracy 4 is the fourth and latest in my series of political strategy games. They are text and diagram heavy, with a properly unique user interface, in 2D, and all of the code is custom, with a custom engine originally coded by me, with a unicode text implementation and vector rendering system coded by jeff from stargazy studios. This means all of the UI elements you take for granted in middleware you use, were custom coded by us. In this case, the bug is my fault, in code written by me. Here is a screenshot:

Then key thing that matters there is the text block under ‘food stamps. This is a policy screen in the game, and that text is the description. Depending on the length of the description (partly influenced by the language you play in), and the screen resolution, it might be that all the text fits in the available space, or it does not. That means sometimes there is a scrollbar to the right of that text, and sometimes its not there. As you would imagine, the scrollbar works just like any windows scrollbar. That means it responds to mousewheels, and also clicks, on the top and bottom buttons (very small) for example.

…anyway…

VERY rarely. And ONLY when I was playing the game outside the debugger, and only in very arcane sets of circumstances that seemed totally utterly random… screens like this (but not just this one) would occasionally not let me click on some elements in the interface. They would not respond at all… BUT… that block of text would scroll upwards, like it was being scrolled through. It was like completely unrelated events (clicking somewhere else) activated a scroll function on a user element I was nowhere near.

This was of course, absolutely infuriating, because it was REALLY rare. And the times I tried crazily to reproduce it in the debugbuild, or even in the release build, but launched from Visual C++, it would not happen. And there were NO reliable steps to reproduce. In fact, even quitting a screen when it DID happen, and then returning to that screen would make it go away.

What the hell?

After a LOT of time, I realized two interesting things. Firstly, when it happened, the game DID make the click sound associated with a mouse click (so the game IS realizing I clicked somewhere) and the scrolling only happened with text inside a specific UI element called a GUI_TextContainer. Although I did not realize it at the time, it was also the case that it ONLY happened where that text container did NOT have a visible scroll bar.

The clue to the bug lies in a closeup of the scrollbars used:

Like all such bars, there is a clickable ‘scrollup’ button, a draggable bar, with above/below page up/down regions, and a clickable down scroll button.

When I create a new GUI_TextContainer, I created a new GUI_ScrollingWindow, which is a container with all of those elements in side. Like any child element, I added it to the Text Container object with AddChild() to ensure it responds correctly and is processed blah blah. Thjn, later when the textwindow is initialized with its position, and the relevant translated text, I do a lot of calculating to work out if that text *is* going to fit in the available space. If not, I set the scrollbar in the correct position on the right, and make a note that the text does not fit (sets BFits to false). Then later, when drawing this UI element, I could skip out a lot of nonsense I don’t need if the text fits, and render it simply as a word-wrapped string. If It does NOT fit, I then drew the scrollbar, and the text in a different way, cropping the text render to the space given the scroll position and so on, blah blah.

All of this sounds reasonable. But it was a big mistake.

I was correctly NOT drawing the scrollbar if it was not needed (why bother! the text fits!) but I had pre-emptively added it as a child of the window anyway. Worse still, I had never initialized the position of the scrollbar and its sub-components AT ALL. This means that in debug / release-from-the-development-environment, all of the relevant variables in that window were getting set to zero, But in a live ‘customer’ environment, the initial positions of those subcomponents could be ANYTHING.

So what could happen, really rarely, is that the ‘scroll up’ or ‘page up’ parts of the scrollbar, might have coordinates like -32054,-5128,27140,41543. In this case, those clickable elements filled the whole screen… but were INVISIBLE. Thus, I have a huge UI mess-up, but I cannot see it, because my ‘quick and easy bath’ that checked BFits, doesn’t draw it.

So very rarely, I create a screen with an invisible scroll up button that fills the entire screen, and happily responds to every click with a scroll-up command. It even nicely plays the button click sound. Luckily, these buttons do not handle keys, so the escape key still quits that whole screen, and the next time it gets created, I’m probably lucky and the invisible bar has moved somewhere harmless.

I’ve changed the code now, to be efficient so that I do not even THINK about creating this scrollbar until I need it, and otherwise its NULL. I also make sure to safely delete it before creating it, in case somehow I go mad and initialize the same text container twice with different blocks of text. I could also have fixed it by initializing the position of a scrollbar safely to 0,0,0,0, but I also like the neatness of ensuring I don’t have a pointless orphan UI element at any point.

This was a classic dumb error. I added an element when I shouldn’t have, and its member variables were not safe. My own dumb slackness. I document it clearly to illustrate how the weird unrelatedness of a bug ‘clicking down here, scrolls text up there. sometimes. only in German, on a Wednesday etc…’ eventually makes total sense.

Hundreds of thousands of people have played probably over a million hours of the game, with this bug in it. No code is bug free, but if you do things right, the ones left in are pretty arcane.

Voting Systems DLC for Democracy 4

I don’t think I’ve even officially announced that I am doing this, so any games journalists reading this can consider this an announcement I guess :D. I am working on an expansion pack for my political strategy game Democracy 4, and that expansion pack will be called…

Democracy 4: Voting Systems

As you can clearly guess, its an expansion that is focused on the way people vote, and on how you campaign in the election. Its mostly a features, rather than just content expansion, so it includes new policies, but also a bunch of changes to expand on that part of the game associated with the election campaign and the voting. Here is a rundown of all of the new stuff I’m working on:

New Policies

There are a whole bunch of these, including the minimum voting age, a scary ‘maximum voting age’, rules on corporate donations to parties, and state funding of parties, bans or limits on donations, robo-calls and TV ads during the campaign. There are 11 new policies in total. Some of these will already be in place at the appropriate setting, like the minimum voting age, for all countries, and the compulsory vote policy for Australia

Campaign Focus

A year before the election, you will need to pick a single metric, from a pre-defined list, to be the main metric that will feature in your campaign literature and campaign advertising in general. This is the game equivalent of Clintons ‘Its the economy, stupid’ or Tony Blairs ‘Tough on crime, tough on the causes of crime!’. The idea here is that you pick a metric that you think makes you look good, and hope that nothing turns bad with regards to that metric before election day. Once chosen, a new effect will run from that metric, to all the voters. If you improve the metric…great but if for whatever reason it turns bad, there will be a big negative impact.

Once the campaign focus is chosen, its highlighted with a small red icon in the corner of that metric on the main UI, and also highlighted when viewing that metrics details. Once the election is over, assuming you won…the focus goes away.

Voting Systems

The majority of countries in the game use some sort of ‘proportional representation‘, but some use a constituency based or electoral-college system, that is called ‘first past the post‘. The two systems can give very different results, and you can now pass a law to change from one to the other. Proportional representation results in higher turnout, and can favor smaller parties in 3 party countries. You can expect to see more coalitions in countries with proportional representation.

Campaign Style

Immediately before the election, you have the option of tweaking how your campaign is run. You have a slider that lets you select anything from an optimistic, safe and respectable campaign, right through to a scandalously cheeky dirty-tricks style negative campaign that tries to smear your opponent. The extent to which such a campaign works (rather than backfires) is partly dependent on how trustworthy you are in the eyes of the electorate. Depending on the outcome, the effectiveness of your campaign spending can be increased or reduced.

Better Polling Data

A new option will appear on the polls screen. Currently, in the base game, this simply shows the voting intention of all voters, along with the current approval rating for all voters. This will get an extra option, via checkbox, that lets you change this to see the likely voters, rather than the whole electorate. Some people are more likely to vote than others, so viewing likely voters gives a better impression of the actual result. Toggling between the two will also allow you to see if you should take action to increase (or cynically maybe decrease) turnout in the election.

…All of this is coded, and implemented, and in the DLC that I am currently testing. Testing an expansion pack can be a pretty long process, because I need to ensure it all works, that its balanced and that it adds something cool to the game. I then also need to get all of the text for the new content translated into every language we currently support (which is a lot…and Arabic is coming too!).

This will likely take me a few weeks, and then I’ll be getting it into the hands of the Democracy 4 players! Hopefully all of these additions sound quite cool. Its certainly been interesting working on it. I know that a lot of people think I should add states/constituencies to the game, and support a whole bunch of different types of PR voting systems. I understand the appeal, but its likely overcomplex for a game like this. Most people are not *that* interested in the pros and cons of different types of PR, and certainly very few people know the political characteristics of all 50 US states AND all 650 UK constituencies and so on…for every country :D

Yet more refinement and complexity regarding party politics in Democracy 4

I am currently working on some voting systems DLC for Democracy 4. In doing so, I have carried out a lot of testing, and balancing, and evaluating, and have just encountered something that I think is lacking in the game. Or more precisely…wrong…or at least imperfectly modelled.

Take as exhibit A this chart (on the rhs) from the game showing the distribution of happiness (0 at bottom, 100% at top) of all of the simulated voters. Because this is in debug mode it draws 2 exciting lines, which are showing the voting thresholds in the game, drawn across the chart in red and blue

image

These (hidden from the player) thresholds are used to decide what party voters vote for, have sympathy for, and may even become members or activists for. In a very simple 2 party system, there is just 1 line at 50% (to start with..) in a 3 party system, they are at 40 and 60%. Below 40% = opposition party, 40-60% means centrist party, above 60% means the player party.

Canny maths experts have already spotted that this is not an even split. TRUE! but thats ok, because it represents the fickleness of a first-past-the-post electoral system. If you change it to PR, the game swaps to a 33/66% split, with each party representing one third of the available ‘approval space’ in the game. This is how we represent the fact that voters are more likely to find a party they can support at the ballot box if a more proportional system is used, with fewer wasted votes…

…also I need to point out that the system is not that simple! Over time, the position of the opposition parties moves towards that of the center-of-gravity of the electorate. So if, for example, all the voters are very unhappy, those thresholds will move down, as the opposition parties position themselves so they can win more votes. If you can imagine a new ‘gravitational center’ line in the middle of all the voters, both that red and blue line are getting sprung (to an extent) towards it.

So far so good. Thats how the game works right now.

BUT. Two things are amiss here, and I am now working on improvements. For one thing… in the real world, parties change their positions in order to WIN VOTES not to improve the ‘average approval’ of themselves. In previous blog posts I have labored the point that average approval and voting intention are different things entirely! I have edited the games code for 3 party PR-systems to take this into account. Now… in a 3 party PR system, the opposition parties move towards the gravitational center of voting intention (not approval) only for opposition voters. Let me explain with examples:

The UK has 3 parties of size: Conservative, Labour and Lib Dems. In the old system, the labour and lib dem voters would be essentially moving to a policy position that kept everyone in the UK as happy with them as possible, while still remaining vaguely rooted in their principles. However, in this new system, the Labour/Lib Dem parties will still gravitate towards this position…but the dividing line between those parties will now only be affected by the firm voting intentions of non conservative voters.

Under the old system, a policy change by the non-centrist opposition party (labour), that seriously upset conservatives, but made lib-dem voters happy… would maybe not be adopted, because the average approval of ALL voters (including conservatives) was being measured. Under the new system, labour policy totally ignores the hardcore conservative voters, and is only concerned with winning votes from the lib dems, and more importantly…winning VOTES…not just a change in opinion. So for example, a policy position that has no impact on right-wing lib-dems, but may well win over voters floating between lib-dem and labour…will get adopted.

(Of course the game abstracts this, and you don’t get to see oppositions literal policy positions, but its all represented in the way these thresholds move.)

Why does this matter? Because by making this change, a more sensible position is chosen for that red line. What that means is, under the new algorithm, with a PR system those red and blue parties are more likely to be equal sized, and we are more likely to get coalition outcomes. This is EXACTLY what you would expect under a system of proportional representation.

So thats one change… and something I am definitely still testing, and will be specific to PR, and thus the new DLC. However, when working on this I realised a fundamental misconception about party membership that is unbalancing the game.. <drumroll>

If you look at that chart again (in fact I’ll repeat it with some arrows..) you will see that the 3 parties also have a center point, which is used to determine how much someone believes in the party:

These locations determine how big the membership of your party gets. Someone close to the blue arrow is likely to join the bluie party, someone close to the green arrow is likely to…etc. See the problem?

Yup, the system is total bullshit. The centrist party has its membership center slap bang in the middle of its political sphere, but I placed the other 2 parties at the extremes. At the time, this made sense to me, but now… Ha. no. This makes no sense. Think about political parties on the edge of a mainstream political spectrum. Where are the members and activists? They are likely to be in the same position as the members in the centrist party: At the location CLOSEST to the center of policy position for that party. The voters shown at the red and green arrows are not diehard loyalists! they are the extreme libertarians or fascists or communists who consider the party they vote for to be a reluctant watered down compromise! They are actually the voters on the verge of ‘wasting’ their vote on a fringe party. No! The correct positions should be:

In other words, we should have some red party members here, and ok… maybe not a green party one yet, but certainly more likely than under the old system.

This is a fundamental coding screw up and design miscalculation. Its not exactly critical, but I strive for accuracy of my political modelling. This change will affect non-DLC vanilla games too. If you have been playing Democracy 4 and bemoaning the fact that only the middle party in 3 party countries ever gets any members, you are not wrong to do so. I’ll fix this for the next update.

Thoughts on potential Democracy 4 expansions / DLC

Democracy 4 has been released from early access over a month ago, and has had 2 big updates since then, and further updates to the game, especially regarding balance tweaks, are definitely coming. However, I am a big fan of the ‘expansion pack’ model of the business, because I think it has a number of positives:

  • It allows players who are only casually into the game to play an approachable and simple base version
  • It gives scope for players to choose which expansions to get, based on their level of interest
  • It means that I can learn from peoples’ feedback on the base game and produce the sort of extra content people might want

Of course, the big question is exactly what part of the game to expand upon or add to. There are a bunch of ideas floating around in my head, and I thought it only sensible to get those ideas out there in the wider world to canvas feedback from real players of the game, so in no order at all, here are some idea.

DLC idea #1 Voting Systems

We already did an electioneering expansion pack for Democracy 3, but all of that content is already included for free in the base game for Democracy 4. However, I think there is a lot of scope here. For example, right now the player has no influence in direct terms, on the voter turnout. We could add policies to encourage voter turnout, like information campaigns (which may or may not benefit you, depending on your electorate). Perhaps polling data could show what the result would be like when polling only likely voters! Also we could model voting reform, so whether or not to lower (or even raise) the voting age, whether party donations are capped, or can be anonymous, can come from corporations or foreign nationals…

There could also be some modeling of campaigning tactics. I like the idea of the player having to choose from a list of different metrics (GDP, Unemployment etc) on which to focus a year before the election, so the player has to be wary of what they drew attention to in their campaign literature. We could also allow attack ads, at a possible risk of backfiring.

Also, some countries like the UK ban political TV ads. We could have this policy, and also maybe a ban on robocalls. We could also have limits on who can stand in elections (US president must be US born etc), and if the state funds political parties at all.

Electoral reform: the fine print matters : Democratic Audit

DLC idea #2 Situation Room

This one would expand on the terrorist threat element already in the game. I’ve spent a fair bit of time on this idea. Basically it would introduce time-limited decisions to the game, where a serious threat or event takes place and the president (you) has to make a decision on what to do RIGHT NOW, in a ‘the clock is ticking’ style. This would mean no time to consult a lot of data, and a gut decision.

An example dilemma is shown below. I think it would be interesting if the advice and intelligence given in these sudden events would be dependent on how involved your intelligence/security network was, so it would be harder for liberal democracies, and easier for more authoritarian states, to add some tension between liberal principles and security:

dlc1mockup

I think of this idea as a sort of ‘west-wing situation room meets 24’ style political thriller expansion, where the player has to deal with lost of situations that with any luck, the general voting population never even hear about. We could have dirty bomb threats, suicide bombers, espionage from other countries, and the possibility of using illegal measures covertly to protect the security of the country.

In game-mechanic terms, I keep coming back to the idea of this involving what are in effect spruced-up dilemmas, but with a timer, and no opportunity to gather extra dat and come back to them.

DLC idea #3 Dirty Tricks

This would basically be a corruption, scandals and dirty tricks expansion, where the game effectively tempts you to be corrupt, with potentially bad consequences later. We can all sadly come up with tons of ways in which we could model this. I know one game (might be tropico) actually used to let you put money in your own private account, and we could do a similar thing here, literally taking money in exchange for policies :D

I think there is a lot of scope here. You could order the secret service to look for embarrassing or compromising information on opposition leaders or campaigners. You could literally bribe the press to give you better media coverage. There would be an option to just flat out lie about some government statistics, to prosecute whistleblowers, to threaten journalists and so on.

We have a very slight element of this in the game already whereby general media censorship means that political media spin events are more likely to be perceived as positive. However, I can imagine it being an option where an event clearly backfires but you can then choose to flat out threaten any journalists that reported on it.

Intimidation at polling stations by violent gangs ordered indirectly by you would also be included here.

Perhaps we could track both your own morals (on a good -> corrupt axis) along with your private bank account, as things that a player could choose to focus on for their own satisfaction :D. I can imagine an interesting situation where players who decide to play a game completely fairly and with principles would be tempted before a difficult election to bribe some journalists “just this once” etc.

Justin Trudeau anointed by Richard Nixon - at four months old | Canada |  The Guardian

DLC idea #4 Country Pack

Pretty self explanatory. I think there are a bunch of interesting countries that could be added to the game. Switzerland strikes me as interesting, as does Poland, maybe also Turkey and perhaps a country like Honduras. I think the goal would be to pick interesting countries, not just ones with the most players, and take the time to represent them as well as possible, so doing a lot of research into the economics and political environment for each one. That probably involves a bunch of new policies and situations and dilemmas for each country to represent their specific challenges.

West Bay Beach - Roatan Island, Honduras - West Bay Beach, West Bay  Traveller Reviews - Tripadvisor

Anyway… I have not decided yet, and am not currently working on DLC, but I was interested to get anybody’s first reaction to which direction to head in. I’ll check comments here, and post a twitter poll at https://twitter.com/cliffski/status/1504400682836758541

Another tweak to voter approval visualization

I blogged very recently about adding voter approval distribution data to the polling screen in Democracy 4. This is going into the next game update, but I’ve changed it since then and thought I would explain why. The aim has been to give the player some insight into the actyual distribution of voters over the whole range of approval values, rather than giving just two numbers at any point in time (which in a 2 party system are the ‘average’ of all voter approval values, and the percentage of voters for whom their approval exceeds 50%).

For reasons discussed before, these numbers, while interesting give no insight as to whether your current support is soft or reliable, so they are imperfect information.

Consider an example. The player is 2 turns before an election, their approval is 60% and their voting intention is 85%. The player has a fairly socialist-leaning electorate, but they are trying to move towards capitalism, and one of the steps along their desired path to this, is to privatise state industries such as rail. They are trying to decide if they should go ahead and privatise rail right now.

In one scenario, their support is pretty solid. The voters approval is distributed pretty evenly across the spectrum. Obviously an approval drop across the board will result in everyone slipping down slightly, but enough voters sit really high in the distribution that the voting intention will happily stay at 70-75% even with a drop in average approval.

In another scenario, their support is super-soft. Basically almost all of the voters are in a single big blob of approval hovering around the 60% mark. The unpopular privatisation might shift that whole bubble low enough to actually drop voting intention below 50%, losing the election

Right now, the player could go check the parties screen with the chart on the right giving them clues about the distribution, or the ‘everyone’ happiness distribution, which shows the same thing, but the data was never present on the polling screen, and more relevantly, it was only ever shown as an instant snapshot. Here (below) is my final implementation on the polling screen.

This is a new ‘optional’ view, thats turned off by default. (The checkbox to the right turns it on or off, and a tooltip over that area explains what it shows). This now shows the distribution (bubble sizes in the y axis) over time (the x axis, with the right being the present day).

I spent LOTS of time trying out different methods to show a bunch of stuff here. Initially I just had the background blobs, but they were all one color (feint green). I then realised that what was actually helpful here was to show what party the voters were going to vote for, so I added some feint red lines to show ‘voter thresholds’ for each party. This needs explaining:

In a 2 party system, the code is simple. Voters check their approval, and assuming no external factors (electioneering stuff, or party membership), they vote for the player if approval >50%, and for the opposition otherwise. In a 3 party system, there is a middle party, so you are looking for 60% approval to get their vote, with 40-60% approvals going to the middle party. However… its not quite that simple!

These thresholds MOVE over time. The game takes into account the fact that parties shift their position. This is the famous overton window, coded into party strategy. In short, if you are winning, the other parties move their (unstated and hidden for game purposes) positions in policy terms closer to your own. That means those 40% and 60% thresholds become more like 45% and 65%, or even closer. As you would expect, the reverse is also true. When you are doing badly, the opposition parties move further from you, so those thresholds can drop as well. This has been in the game for ages, but was not easily viewable by the player.

The chart above is actually fiendishly complex, because it displays a 2 axis chart with 2 lines showing different things over time (voting intention and average approval) and ALSO a backdrop that shows a four dimensional data set describing voter approval distribution and voting intention!

How come four dimensions? Firstly time (x axis) secondly approval (y axis), then the distribution of voters in each of 20 vertical groups to show the distribution of approval (size of each blob) and lastly the voting intentions within each blob (the colors of the blobs, or partial colors, indicating which party each voter would choose if the election happened that day.

No wonder its hard to get the GUI right :D

Anyway… it seems arcane, but it is actually all in the chart. You can see if you look closely that the point where voters switch from voting red, to voting blue, does not stay a fixed line over time, its actually changing in response to the popularity of my own party, which is based on the voters approval. (note this is a 3 party system, parties are red, blue and green, with green being the player).

If it sounds tricky to imagine as UI its even more annoying to code. I coded it as a sparse rendered system, instead of just generating, for example, 3 different colored circles for every x axis and y axis position. Even if you don’t draw them, that means 20x20x3 = 1,200 sprite objects being created, despite the fact that only a fraction will need ever drawing (most points on the chart have one single color circle, or are even unpopulated entirely.

Anyway, I think it adds some helpful extra data to the screen. I probably need to improve the tutorial/guides/in-game help at some point to explain it all though :(