The future of the engine

Let's take a minute to talk about the future of the engine. The fate of the engine is now forever tied to to the fate of the game, which is very slowly but surely going forward. The project is split into two parts: the game and a semi-general purpose 3D engine, but the split is not good enough yet to take the general purpose library and build a new game from it. A few key features like my new physics behavior is in the game. This might be remedied as time goes by. The general purpose library will be eventually opensourced, but I don't have time for that right now. I am psychologically unable to open-source code that is not near perfect.

So the engine of the game is the library plus the some parts of the game code and that is becoming relatively stable. Two main features that will need to be added or finished is the general unified lighting scheme and terrain LOD. Once these are done, this version of the engine will be considered done. The schedule for these features is closely tied to the schedule for the game. Except for shadows, I really can't think of any other important features to add to the engine. Maybe occlusion culling. So if this won't be the final version of the engine then there will probably be only one more.

I'm currently in a full throttle 22 week development cycle that focuses mostly on the game but also advances the engine.

As a conclusion, the game development is going well and the engine is nearing its current and potential design objectives.

But this is just the near future of the engine. What about the rest?

The biggest problems with the engine is that it is under XNA. XNA is going away and I'm getting tired of its limitations, so this will have to be addressed. The second biggest problem is that it is in C#. Now, I love C# and I would like to work more and more in C#, but for such a buffer heavy application C# is ridiculously slow. I spend more time trying to find hacks to compensate for the slowness of C# than working on the features. All those people who say that C# is not significantly slower than C++ need to try to implement something that by definition would be too slow under C++ in C# and we'll talk after. Unfortunately, this can't be addressed. The code base is far too big to port it to C++ in under 6 months, so I'm not going to do it. Even if I could port the code in a shorter period of time, BEPUphysics is in C# and if I were to replace it with a C++ physics engine, the feel of the game would change noticeably. Physics engines are not absolute simulations and are by far non-interchangeable. I have balanced my game around how physics behaves under BEPU and it would take me further months to re-balance it to a new physics engine.

So it's C# and BEPUphysics! All the way, baby!

But XNA has got to go. I knew this for quite some while and I took a measure of steps. My game and engine don't use any real XNA specific features. I'm not using the content pipeline to load or do anything with the assets of the game. I am not using the component framework of XNA to render my game or manage my components. I am not using the content importers from XNA. These and more have been eliminated. I still need to slowly phase out the reliance on the SpriteBatch and friends classes. After this is done, the only things I'd be left using form XNA are the vector, matrix and color types, the API to create the device and render targets and the API to set the state of the renderer (buffers, textures, shader parameter, etc.). These parts won't be easy to port, but they are a relatively small part of the code base. I basically need to port my Game, Renderer and Cache classes to whatever I'll be using and 90% of the porting is done.

I will be porting to SharpDX, which hopefully offers the full power of DirectX and I will go about the porting process as smart as possible. If I were to abandon today development and focus 100% on porting, I could have a working version in about 3 weeks of working round the clock. But I don't want to disturb the development process, so I'll be going much slower. First, I'll split the game and engine code into two parts: the XNA dependent one and the non-XNA dependent one. This will be done as a background task in the next month and is very easy, just make sure to minimize dependencies and move files into new folders. Then, I'll start using less and less XNA data structures and the math library and I'll control which version will be in use with compilation flags. From this point on, you will get two executables. One native XNA that runs at full speed and one that uses less XNA but will be slightly slower. This process will continue until summer 2014! So not 3 weeks but 9 months, making sure that I don't sacrifice any development speed for porting. I really don't want to spend more than a couple of hours a week porting. Maybe even one hour. Maybe on the clock. During this period, there will eventually be two versions of the executable, the main feature rich one and a secondary SharpDX version. Both will look identically, pixel perfect, but one will have less features and be less stable.

And I also have a plan for the direct porting to and learning of SharpDX. Before I started this development sprint of 22 weeks, you may have noticed a long period of inactivity on the blog and development wise. But I did not sit on my hands the entire period of time. I wanted to port the engine to C++, so I though I would revive my old DXUT aborted attempt at a tutorial series. I wanted to write 15-20 tutorials and then take the last one and turn it into the port of the game. I got to tutorial 7 before I realized that the physics engine was going to be a big problem, so I got my shit together and went forth on my development sprint.

So this is the plan. Develop the game as usual. Reduce dependencies on XNA. Make sure that new code does not use XNA. Publish my DXUT tutorial series, but don't go up to 15-20 as the original plan was and only publish the first 7. Create a new 8th tutorial, that ports the parts of DXUT that I'm suing to raw and very simple DirectX, while maintaining the same API. You will be able to run a diff on tutorial 7 and 8 and they will look the same. Then, take tutorial 8 that uses just DirectX but has a DXUT style API and port this to SlimDX for tutorial 9. Then take the SharpDX tutorial 9 and add the same camera, terrain and renderer class that I'm using in the game now with XNA.

So how does this sound?