Crash Protection

Crash Protection from RuntimeCompiledC++ on Vimeo.

We felt that for RCC++ to be really practical, it had to have a form of crash protection. After all, it would be a frustrating experience if you could avoid quitting, recompiling, reloading only until your first simple mistake, when a null pointer forces you to close it all down and start over.

We looked at two main approaches to achieving this - using a separate process and structured exception handling.

Google Chrome uses the process approach for each of its tabs, allowing one to crash without affecting any of the others. This is a very robust approach, but unless your engine's architecture has this in mind, it may result in a very large number of interprocess function calls which would have severe performance impact. We wanted an approach that would be easy to drop into existing projects.

Structured exception handling (SEH) is a Win32 API feature which allows handling of runtime errors such as access violations. It behaves much like standard exceptions but is in fact quite separate - there are various reasons why standard exceptions are not used on games consoles, but these don't affect SEH. When a runtime error such as a null pointer dereference occurs, the OS checks a stack of possible handlers registered by the application to see how to proceed. The crash dialogs you see in Windows are in fact the default handler; when Visual Studio's debugger is attached, that adds another.

Using SEH it is quite easy to catch an error and carry straight on. In our case, the key place for this is around the update calls on our game objects. When an update fails, we disable it until the code has been runtime-recompiled, when we try again. As you can see in the video, the rest of the application - rendering, GUI, logging - all keep running in the meantime.

But actually, you don't want to handle a crash silently - you'd really like to find out what caused it first! You could add code to produce a stack trace. However, debuggers like that in Visual Studio already provide a really ideal interface when crashes occur, allowing you to inspect state easily. So really, we would like to crash, but then continue.

In fact, this is exactly what we do - we first allow the crash to proceed as normal so we can use Visual Studio to debug it, after which the user can hit the "continue" button in the IDE. Usually this is quite useless but in our case we catch the crash on the second attempt - and proceed with execution. We get the best of both.

Without crash protection, rapid changes to live C++ might just let you shoot yourself in the foot faster. But now this isn't a fatal injury, you might as well select full automatic.

Serious Changes

Serious Changes from RuntimeCompiledC++ on Vimeo.

In our second video, we show that Runtime-Compiled C++ can also handle much bigger changes - in this case adding new member variables and methods to a class to bring in entirely new features. You can add whole classes too when required.

Our first video's one-line change could actually have been achieved with "Edit and Continue" - a standard feature of Visual Studio. This essentially patches in changes to your original binaries while your app is running, and it's an enormous time-saver! However, E&C has major limitations. Most significant is that it will reject "changes to a data type that affect the layout of an object, such as data members of a class".
That essentially rules out everything but non-structural tweaking of your code. In practice however there seem to be many undocumented limitations to E&C - which mean that in a real codebase, whether it will work for a given file or a given change can feel like pot luck, and often when it fails there is no explanation as to why.

(You can read an old, but very detailed, discussion of E&C workflow here, and a list of its documented limitations in VS 2010 here)

In this video we show that RCC++ has no such limitations. This is because we compile changed classes from scratch, rather than using any binary patching technique. This also means the compilation is just as dependable as your regular compilation and when it does fail, you have an ordinary compiler error to explain why.

What's up next? Well, the ability to make quick changes is all well and good, until your changes crash the game. On our first null pointer, we have to start from scratch. Right?

Teaser

First Steps from RuntimeCompiledC++ on Vimeo.

Here's our first video: the briefest possible demo of what you can expect from the RCC++ testbed code.

The main thing you see here is our core feature: you can edit the source code for your game or application and see the changes applied, while it is running. 

How do we do that? The runtime-modifiable classes use a macro that registers the files themselves to be monitored. Whenever one of those files changes, we pick it up and recompile it into a DLL. By linking that DLL we can switch over to the new code, leaving the original EXE file untouched. This way we avoid any interpreter and use native compiled C++ throughout.

For projects with long compile or load times - like AAA games - this makes an enormous difference to iteration times while keeping all the advantages of pure C++ development.

You can also see that we haven't created a new editor or tool for working with the source you're changing - we continue to use Visual Studio, just as you would normally. The key difference is that you don't have to close and recompile your app to see the changes.



We'll explain more in our upcoming posts - or you can also check out the forums, the wiki, or download the source and have a play yourself.

Source Code Now Live!

Our Runtime-Compiled C++ demo source code is now available on Github under the permissive zlib license. If you're not familiar with Git, there is also a link to simply download as a zip file.

After our talk on Friday at the Paris Game/AI Conference, we wanted to make the source code available as soon as possible for people to try out themselves.

If you haven't seen the talk and are wondering what Runtime-Compiled C++ is about: in brief it's a layered framework for rapid development of applications, allowing you to change the C++ source and automatically apply the changes while it is running. It includes crash handling - most errors in runtime modifiable code that would usually crash your app, you will find you can continue from and the engine will keep running. Overall, we can achieve seamless iterative development in pure C++.

This is all best explained with a video! We hope to get one posted here soon.

This really is a prototype, developed in our free time, and not one we've been able to test on a huge variety of systems. There are a number of dependencies: at minimum you'll need Windows XP or above, the June 2010 DirectX SDK and Visual Studio C++ 2010, either a commercial version or the free Express one will do. You'll also need the redistributable packages for Visual Studio C++ 2008 (and possibly VS 2005), due to some dependencies we currently only have binaries libraries for but which we intend to move to full source. If you get a side by side configuration error message, you're probably missing one of these. It's fairly likely you'll already have these installed, so try and compile and run the code before searching down the dependencies. We also have an issue with spaces in the path, so place the code in a path with no spaces. We'll do our best to resolve these issues ASAP.

First things you'll want to know once it's compiled: set SimpleTest as the startup project, this is the "Pulse" demo; when it's running, click on the splash screen to start the action; if you click the icon in the top-left of the window, you'll get some buttons include "New game" and "Restart", which will let you switch to the full set of game objects and reset the simulation.

You will see "Aurora" mentioned in the project - this is what we call the open source engine we've put together for the demo. It's really a very simple one, but we do plan to build upon it, for instance completely replacing the graphics code and bringing in more open-source components wherever possible. Three of the projects in the solution are named RocketLib* - this is the GUI framework we use.

The code we've put together is a prototype that we plan to improve and build upon, and we'd be glad of help to do that - but it's also a testbed. There are alternative ways we can approach crash protection, serialisation, compilation - so we'd like to actively encourage people to fork the project, try their own ideas and let us know their experiences. We've focussed on how it can be applied to games development, partly because that is our background, but we also think it could be useful in other industries.

All of the above will be expanded on in posts to the forums.

We hope you find the source code useful and we'll be really interested to hear your feedback.

Coming Soon

Runtime-Compiled C++ is a way to reliably make major changes to your C++ code at runtime and see the results immediately. It's aimed at games development but could be useful in any industry where turnaround times are a bottleneck. We will be opening up the source code to our prototype and testbed after we present at the Paris Game/AI Conference in a couple of weeks time - we hope you'll come and check it out then.