Item — HN Reader

Unreal Engine does unity builds quite well - it will, as part of a prebuild step:

1. Merge related cpp and h files together in groups into monolith files, usually in the order of 10-20 source files merged based on my observations - often entire modules will be grouped together. 2. Exclude any individual files from the monolith that have edits since the last change.

It's a nice middle ground, you don't substantially slow down your incremental builds since the files you're editing are still compiled individually, plus you get a fairly substantial improvement in build times.

It's nice in that you can still structure your source in separate cpp/h files will little extra consideration for the unity builds and it mostly "just works" in both unity mode and with regular builds.

Unfortunately you occasionally do have issues with builds that work in unity but not in individual compilation or vice versa (usually due to arcane #include dependency chains) but they're usually easy to fix.

2 comments

Unreal's unity builds get very annoying as soon as your teams exceeds a handful of developers. You often have code that has builds fine on a machine but doesn't on another developer - it completely depends on what order the UBT picks. This is even more annoying when you have some people on Windows, some on Linux, some on Macs, and they consistently get different results. The fact that most tools (Intellisense, all clang-based tooling, ...) are broken with UE makes automating the detection of missing includes even harder.

We worked around this by introducing a mandatory pre-merge CI stage that constantly does non-unity builds - but it's costly and not something a small company can often afford (a non-unity build of our UE project is ~20min on a Linux runner, and way more on a Windows one. That adds up fast).

Unreal itself hasn't been non-unity buildable for a very long time. In general IMHO Unity builds are a testament to the failure of the C++ standards committee to realise that modules and the building model should be part of the standard to. The "one file is a translation unity" hasn't been adequate for years IMHO - I honestly appreciate how Rust basically imposed cargo as a standard, it was a hard but sane choice.

I agree with your point on the C++ standards, but disagree about your complaints with Unreal's unity issues. There are issues with non-reproducibility in theory, but I've worked on massive projects and the impact is minimal.

> We worked around this by introducing a mandatory pre-merge CI stage that constantly does non-unity builds - but it's costly and not something a small company can often afford (a non-unity build of our UE project is ~20min on a Linux runner, and way more on a Windows one. That adds up fast).

Or you could only build the files that have changed. Even the largest of large files are one minute compiles.

> Unreal itself hasn't been non-unity buildable for a very long time.

Unreal builds in non-unity just fine. There was definitely a time period where it _didn't_, but for the last few years it's been much better than that.

> the impact is minimal.

yes, I must admit it's not the end of the world, but it tends to add an extra layer of gotchas on top of our already crufty legacy codebase (millions of lines of code, >100 devs on 3 different platforms). Which is something I'd rather not have to deal with, honestly.

> Or you could only build the files that have changed

In my experience the UBT is somewhat inconsistent with that. I've seen multiple times for instance Mac builds being borked on our main branch because a Windows developer pushed code that built absolutely perfectly on their machine.

> Unreal builds in non-unity just fine. There was definitely a time period where it _didn't_, but for the last few years it's been much better than that.

last time I tried 5.1 refused to build non-unity on Linux. Did they fix it with the version after that?

May be worth rethinking your module arrangements - unity builds have been stable for a while.

Sadly our codebase is full of legacy nonsense - which makes modularizing it a chore to say the least.

Have you considered adding something like ccache to your compiler to speed up your CI?

Really? That hasn't been my experience at all. What engine version are you on?

We target windows, linux, xsx and ps5 and have probably 15-20 programmers making contributions daily and probably only hit maybe one or two of these issues per week, and they rarely get checked in as we get them during our mandatory preflight build during code review, similar to what you describe. We run all the preflight on on-prem machines now so the cost is minimized compared to our former cloud solution.

We did a lot of work to modularize our codebase so maybe that is helping?

> We did a lot of work to modularize our codebase so maybe that is helping?

Almost definitely the case. We have 100+ devs on a >1M legacy codebase with a lot of cross dependencies, and it's incredibly troublesome without a strict pre-build stage on CI (on premise of course, it's too expensive to run all of that on AWS)

You mean it does unreal builds.