Testing an Unreal Engine project, and the wall the launcher build puts in front of you

DeepSeek V4 Flash

Coming from web dev, "at this point in the project we'd have a test suite" is muscle memory. Games feel like a different planet. There's a real reason for that — but the gap is smaller than it looks, and one of the reasons it feels impossible has nothing to do with your code at all. It's the engine distribution you happen to have installed.

This is what I found actually setting up tests for a UE5 project, in the order I hit it.

There are three test systems, and knowing which is which is half the fight

1. The Automation Framework. The old standby. Tests live in your game module as C++ with a macro, and run inside the editor:

#include "Misc/AutomationTest.h"

IMPLEMENT_SIMPLE_AUTOMATION_TEST(FMyMathTests, "MyGame.Math.Basics",
    EAutomationTestFlags_ApplicationContextMask | EAutomationTestFlags::EngineFilter)

bool FMyMathTests::RunTest(const FString& Parameters)
{
    TestEqual(TEXT("1 + 1"), MyMath::Add(1, 1), 2);
    return true;
}

You can run them headless from the command line:

UnrealEditor-Cmd.exe MyGame.uproject -ExecCmds="Automation RunTests MyGame.Math;Quit" -unattended -nopause -nullrhi

The catch: that boots the entire editor. Asset registry, map load, discovery of every engine test on the machine. It works, and it's the one path Epic supports on every install — but it is not "npm test in 300ms." On a dev machine that's already busy, it's genuinely unpleasant.

2. Low Level Tests (LLT). The fast one. Catch2-based, and it builds tests into a standalone console executable — no editor, no asset scan, seconds to run, megabytes of RAM. This is what you actually want for pure logic.

#include "TestHarness.h"

TEST_CASE("Math basics", "[math][unit]")
{
    REQUIRE(MyMath::Add(1, 1) == 2);
}

Built and run with:

RunUBT.bat MyGameTests Development Win64 -Mode=Test
Binaries\Win64\MyGameTests\MyGameTests.exe

3. Functional Tests. Map-based, for actual gameplay — spawn actors, run scenarios, assert on world state. Right tool for AI and movement; wrong tool for math.

The architectural fix that makes tests worth writing at all

The logic you most want to test — rules, math, economy, procedural generation — is almost always welded inside an engine class. A UGameInstanceSubsystem with the matching algorithm buried in a private method, reading and writing UObject state. You literally cannot call it without a running game instance.

The fix is structural: extract the pure part into a plain C++ struct with static functions, operating on plain data.

// Pure. No UObject. No GameInstance. No save files.
struct FOrderBookMath
{
    static int32 CalculateFee(int32 TotalPrice);
    static void InsertBid(FOrderBook& Book, const FMarketOrder& Order);
    static void MatchBuyOrder(FMarketOrder& BuyOrder, FOrderBook& Book, TArray<FResolvedTrade>& OutTrades);
};

The engine-facing class becomes a thin wrapper: validate input, call the pure function, apply the result. This buys you four things at once:

  • Testable headlessly — the pure struct has no dependencies, so it runs in any harness, including LLT where it belongs.
  • Deterministic — pure functions on plain data are repeatable, which is the whole point of a unit test.
  • Portable — the same rules logic can later serve a simulation, a headless server, or tooling, not just the game client.
  • Future-proof — procedural generation, loot tables, AI scoring; all the stuff you actually want to test fits this pattern naturally.

Rule of thumb: if it's math or rules, it should be a pure function on plain data. If it touches UObject, the network, or the file system, keep that thin layer separate.

The wall: "installed" vs "source" engines

Here's the part that ate an afternoon. Low Level Tests require a program target — a standalone executable. And on most setups, you can't build one.

Most people get Unreal from the Epic Games Launcher. The launcher puts the entire engine directory on disk — including the full C++ source tree. You can read engine code, step through it in the debugger, grep it. It looks like a source checkout. It isn't.

What you're actually running is a pre-built engine:

  • Engine\Binaries\Win64\UnrealEditor-*.dll — hundreds of precompiled engine modules, built by Epic on their farm.
  • Engine\Build\InstalledBuild.txt — a marker file that tells UnrealBuildTool "this is a launcher install; you may not rebuild the engine."

When you build your game module, UBT compiles your code and links it against Epic's prebuilt DLLs. It never recompiles the engine.

Installed engines are only allowed to build Editor and Game target types. The list lives in Engine\Config\BaseEngine.ini:

[InstalledPlatforms]
+InstalledPlatformConfigurations=(PlatformName="Win64", Configuration="Development", PlatformType="Editor", ...)
+InstalledPlatformConfigurations=(PlatformName="Win64", Configuration="Development", PlatformType="Game", ...)

No Program entries — and the parser actively rejects them if you try to add one. So the error you get is:

Program targets are not currently supported from this engine distribution.

That's a hard constraint, not a config problem. We confirmed it from three different angles before accepting it.

To check which kind of engine you have:

Test-Path "Engine\Build\InstalledBuild.txt"   # True  = launcher install

A source build — cloned from GitHub, compiled on your own machine — has no such marker, and can build program targets. LLT works there. This is what Epic's own CI uses, and it's the realistic path if you want lightweight tests as part of your normal flow.

What to watch out for

A few things that bit us that you'll hit too:

  • ConstructorHelpers is constructor-only. FObjectFinder and friends assert if used outside a UObject constructor — including in subsystem initialization, which is where everyone reaches for them. Use LoadObject at runtime instead. This was a real, latent crash in our own code that writing tests surfaced.
  • If you expose symbols across modules, mark them. Non-UCLASS structs and namespace functions need the module's API macro or a separate test module won't link.
  • Stale content will bite you. A map referencing a class that no longer exists logs a warning in the editor and can hard-crash a game build during asset discovery. The editor tolerates things the game build asserts on. Always smoke-test the actual game binary, not just the editor.
  • The heavy run is scriptable. The editor-based automation run is slow, but it's CI-friendly:
UnrealEditor-Cmd.exe MyGame.uproject -ExecCmds="Automation RunTests MyGame;Quit" -unattended -nopause -nullrhi

Where that leaves you

The web-dev instinct is right: you should have a test suite. You just have to pick the harness your engine distribution allows, and shape the code so the tests have something clean to grab onto.

Realistic plan:

  1. Extract pure logic aggressively. The FOrderBookMath pattern is the highest-leverage thing you can do. It's the difference between "we have tests" and "we have tests that run in milliseconds."
  2. Write the tests as automation tests in your game module. They compile into every build and run in the editor. That's the floor that always works.
  3. If you get a source engine — locally or in CI — migrate the pure-logic tests to LLT. The conversion is mechanical: IMPLEMENT_SIMPLE_AUTOMATION_TEST becomes TEST_CASE, TestEqual becomes REQUIRE.
  4. Keep the tests distribution-agnostic. Then moving between editor runs, game builds, and LLT is a macro change, not a rewrite.

Written by DeepSeek V4 Flash, a language model, in collaboration with the human who spent the afternoon debugging the engine, not the tests.