I haven’t been working on memory optimisations in GSB for a few days, in fact I’ve done nothing but work on features and bug fixes and other stuff for the campaign game. The game is getting better, and looks nicer now, but there is tons to do, a whole host of niggling little things, and some vital balancing and optimising still to do.

But as I work, I keep thinking back to my attempts to optimise the memory footprint of GSB. I wrote an overriden new and delete, to track all my memory (I never use malloc or free). I have spreadsheets showing all the allocations. However, it seems that the memory I allocate is just a trivial percentage of the RAM the game seems to use if you look at it with windows task manager or the windows performance monitor. Of course, there are about 36 different measurements of memory, such as ‘private bytes’ and ‘peak working set’, And nowhere is there a definitive answer on which one to use. However, even using the lowest one, there is still at least 50% of memory unaccounted for. At one point, I checked my stats were not reporting MB as KB, it was that bad.

Sadly, it’s hard to optimise when you can’t see the wastage. I suspect a big chunk of it is particles, but the maths don’t support that. It could be directx making system memory copies of video texture buffers and sound files, but would that be inside the GSB.exe memory footprint? Who knows!

In other news

1) The ecofan really works. In fact it is awesome.

2) Chopping wood from 3 year seasoned timber is trivial next to 10 month felled holly, which is like cutting neutronium with a spork.

3) The UK has a major apple glut. We have some apple trees in the garden and so many apples we literallty can’t walk in the garden for the fallen ones. I actually knocked on neighbours doors to hand them carrier bags full of apples today, just to get rid of a few hundred. It’s insane. We are eating as many as we can, and making apple cakes galore.

10 Responses to “Where does the memory go?”

  1. Rob Basler says:

    You might find this article on memory use in APB helpful:

    http://lukehalliwell.wordpress.com/2010/07/23/32-bit-windows-and-apb/

  2. Fargo says:

    You should find someone with one of those apple press thingies.

    I saw this post and it mentions some tools and techniques that may be of interest to you. http://www.gamedev.net/community/forums/topic.asp?topic_id=385264

  3. Kdansky says:

    http://www.nobugs.org/developer/win32/debug_crt_heap.html

    This might shed some light. If you are doing it perfectly wrong, then you can lose as much as 80 bytes per allocation, which, assuming you allocate 4 bytes, means you waste exactly 95% memory. Particle system sounds like a good guess for something like this.

    http://agner.org/optimize/optimizing_cpp.pdf

    This contains more interesting details on the workings of c++, mostly about performance, but that includes memory. I enjoyed reading it despite not actually needing most knowledge.

  4. You may be interested in a stack overflow thread I started a while ago asking about how to best track C++ memory allocations:
    http://stackoverflow.com/questions/910172/track-c-memory-allocations

    Also, from your description I’d guess that you are tracking allocated bytes without accounting for allocator overhead. This can be _very_ significant, especially for lots of tiny allocations (overhead is often larger than actual payload). Consider using something like boost::fast_pool_alloicator for such cases.

    cheers,

    Sören

  5. R3D says:

    you need to set up an apple cider press and stock Gratuitous Space Cider

  6. Archduke Astro says:

    R3D speaks wisdom! A good dry cider; kept pleasantly chilled by the lethal black void of space, fermented for reasons that are darn near gratuitous….. ;)

  7. Neikius says:

    Damn, I want to see java games, like full-scale stuff. Java is just so much friendlier to the programmer… though I guess there isn’t a lot of support regarding libs for game programming. C++ can be just so tricky :)

  8. Terry Byron says:

    You should check out MemTracer http://supzi.wikidot.com/memtracer-download . I started with this for tracking a similar issue in a game I was working on last year. We were running out of address space on 32-bit systems. This uses Detours to hook the windows API allocation routines(HeapAlloc,HeapFree,etc). This will catch all the system memory allocations for D3D copies of textures, vertex and index buffers, and give you a good idea of where your allocations may be going(our problem ended being like 700+MB of texture memory that I was able to reduce).

  9. cliffski says:

    That sounds excellent, I shall take a look at it immediately, Cheers :D