Game Design, Programming and running a one-man games business…

Optimizing GSB2 rendering, the day fighters went in…

So today I added the first few new fighters to my GSB 2 engine. As a result, for the very first time I noticed the release mode build looking like it wasn’t running at a full 60 FPS at 1920×1200. My target is a healthy 60 FPS at 2560×1440, so this will mean some proper optimizing. I thought I’d keep a diary here of my investigations. First step is to do a ‘releasesymbol’ build (release build with debug symbols) and run aqtime pro, my optimizer of choice, to look for CPU slowdowns. This is an instrumenting profiler so it will be slooow…..

My initial invesdtigation shows that 50.68% of the time is drawing the battle,  31.62% is processing the frame. I decide to concentrate on the frame processing first, as this is easier to potentially multithread…

f1

So pretty clear I need to work on the debris processing. This is already being multithreaded though… digging deeper it seems that a sorting function takes up 99.95% of that time, and 99.73% of *that* time is spent in GUI_DebrisCloud::Clear() ouch. It’s immediately obvious that per-frame sorting is mega overkill anyway, but why is clear so slow? Aha, because it involves removing the object from another, less optimised list. Ouch… Some digging shows I only add it back to that list later in the frame anyway, so this is entirely redundant, so thats a very easy win! Next lets look at the GUI_3DManager::PreDraw().

f2

Yikes. Looks like STL is not being my friend at all here. A bit of investigation is called for… Again, this is an STL sort algorithm. I suspect I may have an unusually long list of items to sort there, and some digging suggests that this list has about 400 items in it. Not a lot, maybe the actual sort comparison is slow? No it’s a simple float comparison. what can be going on? is the STL list sort() really that inefficient for so small a list? apparently so. My options are to replace it with a vector (which makes removes/inserts a bit slower) or sort less often, or find a way to reduce the list size. The sorting is already being done why other threads are busy. I’ll experiment with a switch to vectors. While I’m at it, lets take a look at the slowest stuff inside the battle drawing…

f3Looks pretty clear that my post processing is slow, but as I recall, thats actually where most of the real drawing takes place… Some digging shows me that the main culprits are my lighting compositing and my lens flare streaks. This is the dreaded code where I read back from the rendertarget, which I knew would be hellishly slow. I *could* do it every other frame, and ‘lag’ very slightly. I *could* reduce the amount of the data I need to read-back, but I suspect this isn’t the problem. A tricky problem, whereas looking into the lightmap stuff I find a whole bunch of STL list iteration going on. I have mused before about using some fixed size (but big) arrays instead in this area… I also think I’m simply doing *too many* single sprite draw calls here, multiple ones for each fighter (don’t ask!). I’m pretty sure I can make some safe assumptions that compress those fighter layers into one, meaning an instant 50% less ship draw calls, so I’ll try that too… In fact some of them had THREE layers. ouch. Right thats three changes so lets go through the old sloooow profiling again. (results not 100% same as I’m not doing a scripted playback…)

Right then, first thing is that DrawFighting() is now taking 58ms vs 74ms, so already a nice phat boost. (is this microseconds? I think so, it doesn’t matter :D). ProcessFrame takes 10.61 vs 46.19. Oh yeah! The new frame processing looks like this:

f4Weirdly my reduced layers have made no difference. previously the ‘drawunlit’ function took 4.48,s, the new version takes 4.53.  I’m now making 360 draw calls per frame, previously it was 442, but thats had zero impact. Maybe the draw call count is harmless at this point? Interesting. (I make many other draw calls per frame, this is just a certain function). The 3DManager sort time went from 10.5ms to 1.628, so a massive win. So far I am 2 victories, 1 damp squib. I can see some asteroid related slowdowns, but they aren’t in all maps and I’m looking for broad wins here… I just spotted 484 calls per frame to ship::IsOnScreen(). This is possibly an inefficient function, as it cycles through layers and checks for each layer being onscreen, without any ‘quick win’ bounding box clauses… I’ll add a sanity check fast ‘if ship center offscreen by twice our hull size, then quit’. That *must* be faster… I can check this fast without a whole long profiling battle so…

Cool, that function is now 10x faster. yay! 3 out fo 4.

In my browsing I now spot this beast:

f5

What the hell? I was only flying along, there were no distortion waves, let alone roughly 1,000 new ones per frame. This sounds like a complete balls-up! Actually this looks like the profiler getting confused, possibly by some release build optimisation. It does draw my attention to a list I fill each frame… There are 3 slowdown in this,. the creating a new object for the list (tiny struct), the function call to add it, and the actual list push_back. This is all slow. it looks like I am already caching the objects, but clearly it’s not enough. Luckily AQTime lets me profile line-by line…

c1Hmm, so as suspected lists just suck for this purpose. I need to sort out my caching (I actually suspect the max cached objects is just too low…) but I’m going to have to switch to vectors or ideally just an array for this stuff. Less convenient, but it clearly will boost performance.

Anyway… I’ll keep plugging away. I love this stuff. I fully expect the game frame rate to double by tomorrow. This is early days easy win stuff.

Goodbye specular lighting stuff

For a while Gratuitous Space Battles 2 had a separate ‘lighting’ layer for ships that was used to pull off a few effects. This meant a ship hull might come with a color layer, a normal map, a specular layer, an illumination layer and a hulk layer. And those ship graphics are 4x the size of before, so they are 32bit color 1024square dds files. Those get pretty big, and with at least 40-50 new ships, suddenly the game is getting awkward to throw builds about with on my crappy rural broadband…

Luckily, after a bit of chin-stroking and going to-and-fro with the ship artist, we now don’t need that specular layer at all. It was fairly redundant, it was effectively being used as a separate ‘foreground lighting’ layer. It turns out I don’t really need that, I can just re-use the color layer. It’s a bit of a complex engine because it does a normal-mapped 3d thing, but also has lights that can be controlled separately to the foreground (ambient) lighting and also the normal-mapped directional lighting. This allows me to have all kinds of cool effects such as ‘dark’ battles where you can only see lasers and lights from ships, and also to have bright nebulas with everything glowing.

Of course, the minute I changed the code, everything else stopped working. First shadows stopped rendering entirely, then they worked, but I lost control of illumination brightness for ship lights. A whole lot of head scratching and debugging later and I am back where I started, but now the specular layer is irrelevant. That simplifies and speeds up the toolchain too, which is a very welcome bonus.

I’m only a few fighter-hulls away from being able to put together some battles with all-new in-game graphics and taking some screenshots for real, although the planets are still placeholder, and the explosions and particle effects all need re-doing. Still, progress is progress.

Unrelated, I’m off to see the nice people at Valve tomorrow for a UK meeting with developers.. Should be interesting

A whole new world of stats addiction

I’m a statistics addict. They say the first step is acknowledging that you have a problem. I used to spend a disproportionate time studying google analytics data. (I don’t do it so much now, but I do check it around game releases etc.) I also spend a disproportionate amount of time studying exchange rates and share prices. (how do i know this? well i have stats showing me…). I also keep banging on about my favorite profiler, aqtime, or pix. And of course if you have played my games, you may be aware of the fact that I’m the guy who turned just ‘being in your twenties’ into a game of statistics management.

Clearly I have that sort of sheldon-cooperesque geeky mind. I also have a scary memory for some things (not for all stuff, sadly).

So it is with fear, and excitement and mixed emotions that I discover I can get the latest nvidia nsight profiling stuff running for GSB2. Now I can have an entire new program full of charts and graphs and stats and data about what each atom in the video card is doing at each picosecond of every frame in my game…

IO really need to chill out about performance and just work on the game. I tend to work in debug build, which is very slow, and panic a lot, and then (as I just did) toggle to release, detonate an entire fleet of cruisers in 2560 res, and realize the frame rate is still just dandy, and then relax a bit.

Of course, I’ll still be blogging about performance and code a lot, as I love it. i just need to hit my rescuetime targets for the day first, as the stats say I need to get back to my code now…

Positech Games needs artists (as contractors)

Right, I need to get my act together and get some artwork commissioned. I need some good 2D artists, maybe 1, maybe two. Maybe the person I need is YOU or someone you know. I need some artists to work on Gratuitous Space Battles 2. If you haven’t heard of it before, it’s a space strategy game with really gratuitous explosions and effects, and lots and lots of exploding spaceships and lasers. Here is the website for the original game, and here is the placeholder for the new one.

I already have my spaceship and GUI artist chosen, but what I also need are some planets and nebulas.  Are you a decent planet/nebula artist? I’m talking about the kind of thing seen in games like Eve Online, or actually any really high quality decent space game. I want very high quality stuff, not 20 minutes fiddling with difference clouds in photoshop. I like billowing nebulas, really colorful stuff, stuff like those awesome hubble pictures…

hub

I need them as 4096 square jpgs. yeah that’s BIG, and yes jpg’s are lossy,. but I can deal with it. I probably need the original source as bitmaps or PSD files anyway… And I need about 10-12 of them, depending on cost. They all need to be totallyt original compositions, not something anyone has seen before. They need to have no stars in them (parallax starfields are added by the game). They also need to show some pretty cool variety. I may add bloom effects on top of them. I don’t care *how* they are made, they might be painted from scratch with a tablet, they might be composed algorithmically from fractal art programs, they might incldue some amazing blending of real life cloud sources, and will probably be some combination of all of the above. They just need to be fucking cool.

And also…

I need planets. I’ll probably paste these onto the nebulas in-game, using cunning parallax scrolling which is a new technique that has just been invented. I want some very cool planets, like city-planet of coruscant from Return of the Jedi (below), or some really amazing gas giant, or a planet with lots of wacky rings. Go nuts! These can be smaller, maybe I need some 1024 ones, and some 512 ones. And again, they have to be original, and awesome.

corus

I’m not looking for an employee, that involves tax and lawyers. I’m looking for a freelance contract artist. We never need to meet, of course you can work on these from home, to a deadline I’ll set, but I’m fairly flexible. I can pay you by wire transfer, and normally pay in dollars, but that’s negotiable too (especially if you are British). I pay reliably and on time, ask anyone. The only other restrictions are:

1) You must be at least18 years old (otherwise it gets tricky from a legal; POV).

2) You must not be working under a contract that could enforce ownership of your work. ie: if you are currently working for a games company, you need a cast-iron contract that states you can work for me in your spare time and that they don’t have any claim to your work. You *will* have to sign a contract to this effect.

Do you know the people I want? ARE you the person/people I want? Pls email me at cliff AT positech.co.uk. Pls feel free to send links to portfolios of similar work. I don’t want someone who has never drawn a planet or nebula before. i need to be able to see just how AWESOME you are.

And if you work in games, please tweet/share this to anyone who you think may be interested! Cheers!

Redesigning formation orders in GSB2

Formation orders in GSB sucked a bit. You had to select a ship, then a target and basically the order was ‘stay at this position relative to that ship’. This sucks in two different ways. Firstly, it means if the ‘target’ ship gets destroyed, the formation is instantly abandoned (yikes). Secondly, it is laborious to set up 32 ships into a formation.

The new system for GSB2 is simpler. You group select a bunch of ships and then add a formation order in one go. They then attempt to stay relative to each other, regardless who gets destroyed. Internally, the ships ‘elect’ a command ship, that has ‘free-will’ and the other ships will try to stay in relative position to that ship. That ship getting destroyed or tractored results in a new election. So far so good, and certainly better when it comes to ship destruction and setting-up GUI. However, it leads me to question a few things. Take this formation (coder art!)

form1

Just a simple line of 6 ships where randomly I’ve made the blue one the commander. If the commander heads to the right, then all is well. However, if he heads at an angle what do the ships do? Should they stay relative to the ship in absolute world terms, or in relative to the lead ships-angle terms? in other words, do we wheel?

form2

Obviously the two different behaviors are vastly different. It also brings up the topic of what to do if the command ship decides to retreat when damaged, does the formation follow? what if it’s just 1 damaged frigate… I’#m guessing they leave the formation at this point. In my mind, the reason behind formations is to keep ships trogether in the sense that they should be able to cover each other in terms of support, and shooting down incoming missiles etc. With that in mind, I reckon it would make sense to always elect the biggest ship as the commander, where viable.  Theoretically you could have a super-slow ship with the entire fleet locked into formation with it, effectively preventing anyone from moving.

What do you think? do I ignore angles and stick to world space, or pivot? and will the system of commander elections work ok? For reference, this is how fighter squadrons already work in GSB 1.