63

I'm fond of the idea of unit testing but I'm having trouble applying it to game programming. Games are highly stateful and often the code doesn't break itself into distinct units. In my experience, most functions mutate state rather than returning values.

Consider a simple action like playerJump(height). I'd love to have a test suite that checks a large variety of cases to make sure that jumping always works as expected. However this function will likely return no value and have the side effect of, player.velocity.y = -height and checkCollisions(player). I can't think of a clear unit test to build around this.

Is unit testing just not viable in highly stateful applications like games? Are the advantages of unit testing so great that it would be worth programming games functionally?


Update:

Games from Within has a series of in depth articles about using Test Driven Development in games. I highly recommend them to anybody interested in this subject. Here is the first article:

http://gamesfromwithin.com/stepping-through-the-looking-glass-test-driven-game-development-part-1

1
  • For UnitTest++ go to github.com/unittest-cpp/unittest-cpp. Everything else is out of date.
    – Markus
    Sep 17, 2013 at 11:35

7 Answers 7

40

Often the code doesn't break itself into distinct units.

That's just poor design. The code doesn't break itself into anything. You, the designer, have to impose a structure on the code so that you can demonstrate that it actually works.

However, this function will likely return no value...

So?

...and have the side effect of, player.velocity.y = -height and checkCollisions(player).

Then test for that.

I can't think of a clear unit test to build around this.

Why not? You just gave an excellent specification for the results of the function.

You might need a few mock objects to replace the full-blown Player with a simplified MockPlayer that's easier to test with.

But your specification of behavior was perfect. Just test for the things you described.

5
  • 8
    Ah I see. I think I'm interpreting "unit testing" too literally. I'm imagining testing the inputs and outputs of each function. Rather, the test could validate constraints on the entire system after a function is called. Thanks, it's always nice to escape my programming funnel vision :)
    – Kai
    Mar 10, 2010 at 17:12
  • @Kai: Since the system's state change is the side effect of the function, you're still doing unit testing because the function is tested in isolation. It's very easy to test with Mock Objects. I'm not sure why you'd claim that unit testing is somehow limited to inputs and outputs of proper functions.
    – S.Lott
    Mar 10, 2010 at 19:00
  • 8
    An issue I have with this idea is that the test could end up being as complicated and bug-prone as the unit being tested. Take for example tick-based state changes like velocity where no settable values are changed yet gettable values change with each tick. One would have to create an emulation of the velocity behavior to check against, effectively re-implementing the original velocity code but if it fails to test, is it the unit that's faulty or the test?
    – user515655
    Feb 17, 2015 at 5:02
  • 3
    9 times out of 10 these sort of unit tests are simply not practical in the real world. Most people that disagree are talking from an abstract point of view. While there are parts of games that can be easily unit tested, most of it.. the complicated parts that are prone to break it's just not economical to create a battery of UTs
    – John Stock
    Nov 20, 2020 at 1:59
  • 1
    Apart from the OP, sounds like few of the responders have actually done game dev work. I am SDE by day, newbie game dev by night. I find unit testing complex game components very difficult, e.g. AI modules which require a lot of game state. So the tests end up being as complex as the original logic. Coupled with the fact that all this logic may need to scrapped/rewritten because it's not 'fun', makes the whole exercise questionable. Jul 2, 2022 at 1:06
30

Unit tests can be useful for a lot of the low level libraries you may use with game code but I seriously doubt it is of much use for the higher level stuff. Games are usually simulations and rely on massive amounts of shared state which can't meaningfully be mocked up and tested in isolation. Often functions in games do not return any sort of value that you can instantly check but instead set a process in motion which should complete at some point in the future. Testing such behaviour is worthwhile but requires a significantly different approach to the unit test idea of testing pieces of code in isolation.

2
  • 1
    This is a clearer description of the problem I was describing, thanks. It may be worth seeing if the process that was set in motion did indeed end as expected, but it seems more time consuming than testing simple function inputs and outputs.
    – Kai
    Mar 10, 2010 at 17:22
  • Exactly the difference. Matches my personal experience writing game code and application code. Complex game code shares state in ways application code almost never does. Jul 2, 2022 at 1:10
25

Programming is programming. Unit tests are useful for any kind of application because they help you detect and correct errors (and often more importantly, regressions inadvertently introduced as you refactor code) immediately and efficiently.

There are of course areas of high level behaviour that are difficult to unit test, but that's not really what unit tests are for - they are primarily for checking that individual methods or small parts of the codebase do what they are supposed to do.

For the higher level behaviours, you need to apply other testing approaches (regression testing, for example: feeding a fixed sequence of inputs into the game and then checking that you get the same results every time, or generating camera views at fixed locations throughout a level and checking that they always generate the same (or at least a reasonably similar) image)

Your example of PlayerJump is one such case. You can unit or regression test this by isolating it with constant inputs (programatically place the player character at a fixed location in a simple test scene and fire the jump event, then check that his collisions or final resting place is consistent. By building up a library of different objects the player can jump "at", you can cover a lot of test cases (e.g. that the jump succeeds over a gap of the prescribed maximum-jump distance).

Beyond that, though, games do require a lot of gameplay testing (where real users simply play it). This will find odd cases that you didn't cover with automated tests, but more importantly it will answer a question that automated testing will never answer: Does it "feel" right? Is it "fun"? These are tests only a human can conduct.

4
  • 4
    +1 Mind you unit testing a renderer is nigh-on impossible. Some people use the pixar method and snapshot the same control scene from the same angle with the same lighting and shaders and do an image compare
    – zebrabox
    Mar 10, 2010 at 19:45
  • Programming is programming, but when that's in the context of having something delivered, economics, teams and the real world it doesn't work like that.
    – John Stock
    Nov 20, 2020 at 2:00
  • @JohnStock: Indeed, the cost/benefit of any tool/technique must be considered. Unit testing everything is impractical. Unit testing nothing is foolhardy. The skill is finding the best unit testing compromise, targeting the places where it brings the best balance between effectiveness and cost. The effectiveness can also be impacted by the programmer's experience/skill - a junior programmer will usually make more mistakes so will be helped to a greater degree by unit testing. Nov 21, 2020 at 22:11
  • Programming is programming +1 People try to justify not unit-testing by saying "oh game devs don't do unit testing" its still programming, and computers are still faster at checking code than people are. So how is unit-testing not useful for game development? Jan 22, 2022 at 23:51
7

My experience with unit and automates testing during the development of Crysis 2 is available here: http://yetanothergameprogrammingblog.blogspot.com/2010/06/aaa-automated-testing.html Hope it helps.

Summary:

Automated testing improved deliverables stability, increasing productivity for both content creators' and engineers Automated testing is an effective tool to improve code quality and reduce the chances of having to work overtime The Game Industry as a whole is very reactionary in general, automated testing meets several irrational arguments against Don't call it testing, call it something else, almost anything else (Look at Behavior-Driven-Development) Be flexible, writing good tests is hard and require skills that are not widely available in the Game Industry

2
  • 1
    Hi, welcome to Stack Overflow! A link to a potential solution is always welcome, but please add context around the link so your fellow users will have some idea what it is and why it's there. Always quote the most relevant part of an important link. Imagine that page is moved to another server, or the direct link changes - future users will not be able to benefit from the answer. Please take a look at how to answer.
    – Jesse
    Apr 13, 2013 at 15:56
  • Link doesn't work and the question was about Unit Testing not a more general automated testing.
    – John Stock
    Nov 20, 2020 at 2:01
2

Many game programmers use the Entity-component-system architecture. They do it because it makes it easier to modify the behavior to game objects. But as it happens, it also makes it easier to unit test your code.

1

Take a look at PEX for automated unit test generation. It will generate unit tests for all possible input variations, which will help you test a lot of possible combinations.

2
  • 2
    Isn't PEX .NET only? This question isn't tagged as a .NET question. Mar 10, 2010 at 19:50
  • Considering Unity is a big player in the games development industry and it uses ,NET, this answer still has a lot of value.
    – John Stock
    Nov 20, 2020 at 2:03
-1

Unit testing just doesn't care at all how "stateful" your unit is. You have a more or less self-contained piece of code, and if its input and output vector are huge then testing is difficult. Whether or not these vectors are laid down as states before and after execution doesn't change a bit about the testing. If you want to tell us however that you can't think of a proper way to define test cases like in most simple-minded unit testing tutorials/papers, i.e. the test subject is something like "f(x) = y" , then yes, I agree, you will have a hard time proving that the few (x[100],y[99]) vectors that a human cane come up with yield sufficient coverage (numerics!). Try to formulate integrative properties and invariants and go for automated testing.

Not the answer you're looking for? Browse other questions tagged or ask your own question.