A Multithreaded Game Engine

Multithreading. The answer to a common problem with sizable games/engines: Being able to keep up with the huge amounts of processing that has to be performed in a modern game. Or is it really the answer?

Depending on the scale and size of the game you’re aiming to make, multithreading might be something you’re able to ignore completely. For my own game however, this is not the case. I set quite a lofty goal when it came to my own game: Being able to generate marching cube-based voxel objects procedurally, at run-time.

You may think “Minecraft already does this with cubes!” and you’d be correct. But I wanted to go as far as generating entire ships, stations and terrains to fight on, all at run-time. For the most part I’ve managed to pull it off, thanks to implementing multi-threading at an early stage during the development of the voxel part of the game engine.

However, the rest of the game engine suffered due to my laser focus on voxelization which as you would expect, eventually came back to bite me.

Generating voxel data takes no time at all, but uploading the mesh to the GPU and rendering it is where those big bitey teeth appeared. Rendering and updating being in the same thread was causing the renderer to be held back by everything else needing to update too. This badly dragged down the frame rate.

I finally realized what kind of mess I was in after reading an article about game engine architecture by Michael Kissner. He describes a system for sending messages between different sub-systems of a game engine. A kind of internal network only used by the engine. So this is where I spent the last two weeks of development, implementing a message system and separating the various different systems of the engine into separate thread/modules.

Surprisingly, it’s going quite well. I originally thought it would turn into an utter mess of old, partly refactored code mixed with multithreaded messaging code, but that isn’t the case (so far!). Here’s my attempt to demonstrate the structure I ended up with:

Engine layout

Workings of a multithreaded engine

Each sub-system has its own thread. Each of those request frame-time from a scheduler which acts like a policeman to determine whether there is enough frame-time left for a thread to run (or run several times). This avoids the situation for example, where one thread might be updating fifty times faster than the rest of the reads, which would allow it to spam messages to all the other systems, creating a backlog of messages that the other threads cannot handle fast enough. This would eventually cause the program to run out of memory as the message backlog grows in size.

So far it’s been working well in the tests. At the end of it I’m expecting a significant FPS improvement and much more efficient usage of CPU time. But I haven’t tried to render a full scene or anything complex yet. For now though, the refactoring must go on!

Posted in Development, Games Tagged with: , , ,

Custom Game Engine vs Unity

Back in November I decided to take a step back from making a game in Unity. One of the major causes of this was how unity dealt with external scripting (like those used for mods). Before Unity 5 was released, the only option to being able to extend Unity to make it do what I want in a nice way, was to pay the $1,500 Pro License fee for access to native libraries. Its an amazing engine with many great features that I could never make myself, but I felt restricted by it.

I had previously attempted to write my own DirectX 11 engine back in April last year, using SharpDX in C#, but for whatever reason I stopped. However, this is what I went back to after leaving Unity and haven’t stopped working on it since. Almost all of my free time goes into working on my game engine.

Here are a list of the features it has after around 3 months of “spare-time” work on it:

  • Full DX11 shader support (Vertex, pixel, geometry, hull, domain and compute)
  • Multi-threaded content management – capable of loading multiple files at once, in separate threads, without slowing the game down (much!)
  • Deferred Renderer – I’ve recently been showing a lot of progress for this on twitter
  • Client-server networking integration – For multiplayer! Lidgren framework has done most of the heavy lifting here
  • Full audio playback
  • Full keyboard and mouse support – including the extra buttons that a lot of games seem to forget about! *shakes fist*
  • Basic scene-graph system – this is still a WIP. It has no culling of any kind yet
  • Logitech G15/19 keyboard support – I would like more games to have this, so its only right that my own does too
  • UI system + editor
  • Model viewer/editor tool
  • Sprite sheet editor
  • Editor framework – I can plug new tools into the editor in a relatively short time

There are lots of smaller things I could go on listing but many of them are things you’d expect to come along with the ones listed above anyway.

Features like FMOD studio integration, mod management and physics will come later down the line, but what I really want to get back to soon is the voxel system I had working in Unity. It worked really nicely, and used as much multi-threading as Unity would allow, but it wasn’t enough. I want to be able to load up large amounts voxel objects very fast, sometimes generated on the fly. It can and will be done!

Is it worth it?
If you’ve made it this far, you’re probably wondering why in the seven kingdoms am I still making my own engine, when 2 perfectly capable and fully-featured engines are available, for free?

At the time I switched from Unity, I had no idea Unity 5 would provide every single feature free of charge. Making you own engine however, has some very tasty perks. You never have to worry about license fees, ever, except for any 3rd party tech you decide to license. You also have complete access to the engine source code, forever (you do backup your work right?). So the answer is, yes. For me it was worth it.

I believe that making your own engine is a massively invaluable learning experience. If you’re looking to get into the industry as an engine programmer, you will probably be expected to have attempted this at least once. But even if you’re not, you can learn a massive amount of knowledge about how a game works and all of the various technologies, terminologies and techniques that come with it.

Building your own engine doesn’t come without a cost though. It’s a huge time sink and you may have to forgo features like cross-platform support to keep it manageable. You also have to know what your game(s) will need before you’ve even started the engine. You wouldn’t build a V8-engine if you’re only going to make a small motorcycle would you?

To conclude this epically long post, if you just want to make games and you’ve never finished a game before (even if its pong!) then use an engine like Unity or Unreal. Do not attempt your own game engine. However, if you want gain an understanding on how games actually work, rather than just making them, then definitely go ahead and try making your own engine. But keep in mind, it involves a humongous amount of constant research, trial & error and most of all, screwing up. 🙂

Posted in C#, DX11 Engine Tagged with: , , , , ,

Unity and Lua

Since the last development report, progress has slowed down a little. This is going to be a long post, so I’ll start with some screenshots of what progress was actually made!

Randomly seeded stars

Random tests of the new star seeding system

Read more ›

Posted in Games

Voxels, Ships and Doorways

Its been too long since I last made a post!

Basically, I’ve been building on what I showed previously without any real planning or designing, except for a few random drawings of UI layouts.

I’ve decided to roll with a mixture of an RTS, sandbox, survival game with bits of FTL, supreme commander and many (many!) things I’ve always wanted to try but never had the chance to do so before.

The basic idea of the game is that you start with a single spaceship, which is randomly generated (like 90% of the content in the game), with its own small crew. Some things on the ship might be damaged, or maybe not, that is up to the game on what it randomly gives you.


Solar System Generation

Working solar system generation – Planets & moons orbit too!

You’re given a whole, randomly generated galaxy to explore, full of ships and stations that belong to randomly generated factions, each with their own crews and systems. This is where the RTS part comes in. You’ll also be able to build up your own fleet, faction or empire, with the ability to own multiple ships, stations and crews. This is something I’ll probably get down on paper soon because it needs to be done right. I want it to be easy for the players (hopefully you!) to switch between several ships/stations/crew, which might span across several battles in many different systems, so it isn’t something that can be hammered out in a day.



Doorway generation test aboard a ship.

For anyone who has already been following my progress on twitter, you’ll most likely already know everything will be damageable, partly thanks to the underlying marching cubes-based voxel system. But for those who are thinking “what the heck?”, it basically means you’ll be able to blast holes in ships and probably watch their crew run around trying not to suffocate while in the middle of a random space war.

Voxel damage system.

A previous voxel damage system test.

There is so much more to write about but until I’m more certain on specific gameplay mechanics, I’ll save that for another post. 🙂

Posted in Games

Unity Shenanigans

Ever since giving unity a try a few weeks ago, I’ve started trying out an idea I’ve always wanted to see happen in a game.

Its far too early for me to say anything remotely solid (or even liquid!) about the game in terms of gameplay, so I’ll let some screenshots do the talking. After all, pictures are supposedly worth a thousand words!

First attempt at generating a spherical, voxel world.

First attempt at generating a spherical, voxel world.

A random asteroid test.

A random asteroid test.

big voxel terrain test

A bigger asteroid test.

A bigger asteroid test.

First attempt at getting marching cubes working in Unity.

First attempt at getting marching cubes working in Unity.

Slightly improved marching cubes with a first attempt at biomes thrown in.

Slightly improved marching cubes with a first attempt at biomes thrown in.

water transparency test

On a side note, if you’re a Unity developer and have or would like to use Visual Studio to debug your Unity game instead of MonoDevelop, Microsoft released a new version of UnityVS over on MSDN a couple of days ago. Its now completely free of charge and its amazing, check it out!

Posted in Games

Stone Leaf Studios Joins BizSpark Programme!


You’re probably thinking: What the heck is this?

The answer is, in short, the most amazing thing that can happen to a startup indie game developer. BizSpark is sort of like winning the lottery. As a member you get access to every single piece of software Microsoft has ever made, for free, for 2 or more years!.

While that is totally awesome in its own sense, the best part for me was simply being accepted into the programme because it means I have built Stone Leaf Studios up to a certain quality milestone that was worthy.

Visual Studio 2013 pro/ultimate is my most needed piece of development software at this stage though, since I’ve been juggling with a Student edition of 2012 professional and the various different versions of Express for the last 2 years. They have served me well, and I thank Microsoft for providing such amazing tools for free, to everyone. They really do make a difference.

I’d also like to humbly thank everyone who has played my first and only two released games so far on Windows Phone (Physworks and Fizz Filler). Without you, I would never have kept going this long. Knowing there are people out there that like to play the games I make drives me to create even better games! 🙂

Posted in General

Lag Bugs and Windows Phone 7

This is a bit of a techy post so if you’re not into the geeky codey stuff, skip over it!. 🙂

For the last few days I’ve been working on the WP7 version of Fizz Filler. All was going fine, except every time the game saved, it would freeze for around 20 seconds.

I’ve spent since yesterday afternoon trying to find the reason behind this freeze, until finally throwing in a bunch of stopwatch timers to figure out which part of the saving code was slowing it down.

Low and behold, it turned out to be the most unexpected of them all.

For some unknown reason on WP7, it takes this single property a good 30 – 50ms to return a value, despite being as speedy as a usual on WP8.

The main part of my saving code uses a custom made class where data is written in a Key-value structure, which means each piece of data has a friendly word associated with it to make it more readable.

The part where the slowdown happened was where I had 3 calls to the above code for each Key-value that was saved as part of my error-checking code, which allowed a piece of data to be skipped if it was corrupt or incorrectly saved.

Heres the C# code that handled writing of key-value data in my KeyValueStorage class.

As you can see, 3 calls are made to writer.BaseStream.Position (4 if something goes wrong), where BaseStream is an IsolatedStorageStream instance. If each call takes around 40ms to execute, it soon adds up to around 120ms to write a single key-value. Fizz Filler stores a large amount of these values, around 170 of them. So that adds up to around 20700ms (20.7 seconds), which not even remotely acceptable/playable for a game!

For now, I’ve sort of fixed it the dirty way on WP7 by disabling all of the error-checking code, so that it doesn’t need to make any calls to the slow position property. But later, I’ll be solving the problem cleanly by first making it write to a memory stream, then grabbing the size of the data that was written from that, before finally writing it to your phone’s storage.

A memory stream works the same way as a storage or file stream, except instead of writing to your phone’s storage, it writes into your phone’s memory, which is much, much faster, but not permanent. So this is great if you want to prepare data (or get the size of it!) before finally storing it on the phone properly.

If all goes well, this will mean no more slow down, while I still get to keep the error-checking stuff on both WP7 and 8. 🙂


Posted in Games

Fizz Filler – Update 1.3

Its been a while since the last true update to Fizz Filler but here I am again with another update, this time which some juicy new content too!

The biggest addition in this update, is Hard Mode. Unlike the standard game-mode you’re used to, you start with just a single conveyor and must buy the other 2 conveyors to start producing drinks from them. Is that all, you ask? No. Not only do you have to buy the next 2 conveyors, you also have to upgrade them all separately too!

Hard ModeHard Mode Gameplay

Each conveyor is now numbered, making it easier to identify which conveyor you’re upgrading in the shop.

Hard Mode Shop

The shop has also had a small make-over for hard mode. You’ll now see 3 tabs at the bottom, each corresponding to a conveyor. The conveyors you haven’t bought yet won’t be enabled in the shop, but once you do, you’re free to upgrade them as you would in the original game-mode.

For those of you interested in the raw 1.3 change list, here it is:

  • Added Hard Mode
  • Added new artwork for fusion upgrade
  • A faster and more robust save format
  • Conveyors are now numbered
  • Event messages now pause the game correctly
  • Fixed positioning of tabs on help menu
  • Resetting now returns you to the main menu
  • Updated how to play instructions
  • Fixed incorrect display of time played on ticker

Between 1.1 and 1.3 there has been numerous patches and fixes to game to hopefully make it a smoother experience for all in general. If you’re interested in what was fixed or changed during this period, head over to the Fizz Filler changelog and check out the changes!

Last but not least, you’re probably wondering what comes next. Well, without spoiling too much of the surprise, its going to be one of the following:

  • Achievements
  • One-time improvements (sort of like upgrades, but one off, and a lot of them)
  • More upgrades
  • Score/Leader-boards

If you’ve not tried Fizz Filler yet, you can grab it from the Windows Phone 8 Store, completely free of charge! 🙂




Posted in Games

Fizz Filler Update 1.1

So it finally happened, the first true update to Fizz Filler. this brings a few, small new features to the table.

The first is optional endings. Currently these don’t really serve much of a purpose besides giving you a cue to aim higher or take a break (dumb, i know). In the near future however, these will be the way you submit a score. Think of them as checkpoints. The first one happens once you’ve accumulated 1 million coins, but there are a few more for the higher amounts of coin you might earn.

I won’t go into much detail about it right now. Anyway, changelog ahoy:

  • Fixed coins not being cleared after resetting the game
  • Added progress events to the game
  • Fixed shop scrolling not keeping up with drag gesture
  • Current coin count now displayed on main menu
  • Fixed game not saving progress when minimized / deactivated
  • Fixed reset count in statistics not saving correctly
  • Fixed help menu titles being cut off along the top
  • Enabled Brazilian market

You can also get the full changelog of past updates Here.

In my spare time I’ve also been tweaking and improving the website here and there, so if you spot anything different, that will be why. 🙂

As always, have fun!

Posted in Games

Minor launch issues

The release of Fizz Filler has gone quite well, all things considered. However, it hasn’t been bug-free!

Firstly, a small update (v1.0.1) has been pushed to the store to fix a couple of bugs:

  • All coins are lost if you close the game and start it again (ninja bug!)
  • Random lag that occurs when adverts change

This morning I also received notification that Fizz Filler is unavailable on the Russian Windows Phone store, this will be solved as soon as possible. However, since the above update is currently being certified it means I can’t make any changes to what markets are enabled until after it has gone through (sorry!).

Hopefully you’ll all be able to enjoy the game as it was meant to be enjoyed very soon!

Posted in Games