You probably don’t need Unreal Engine’s Cloud DDC yet
Someone posts in a UE Discord that their cook is taking four hours. Ten minutes later, someone tells them to set up Cloud DDC.
I’ve stopped rolling my eyes at this, because the advice isn’t bad. Cloud DDC is good software and Epic clearly put real thought into it. But I keep watching four-person tools teams spend three weeks on a Jupiter deployment — Kubernetes, object storage, an auth story, regions — to fix something a single file in a bucket fixes on a Tuesday afternoon. And now they run a distributed cache service. Nobody’s job is running it. So it becomes everyone’s job, which in practice means it’s the build engineer’s job, which means it’s yours, forever, on top of everything else you already own.
Work out which problem you have first.
Seeding is not sharing
Seeding: a CI job wakes up on a clean machine and would rather not recompile every shader in the project. It wants a blob of already-derived data sitting on disk before the build starts. One writer. Read-mostly. Tolerates being a little stale.
Sharing: a dozen engineers and a fleet of build machines all hitting the same cache, reading and writing constantly, everyone picking up everyone else’s work more or less as it lands. Many writers. Invalidation matters. Latency matters.
Cloud DDC is built for the second one and it’s genuinely good at it. But when I ask people what actually hurts, the answer is nearly always “CI is slow.” Not “my artists are recompiling shaders Bob already built yesterday.” Different sentences, different fixes, and the first one is much easier than the internet will lead you to believe.
The boring version
Unreal has shipped DDC pak files for years and I hardly ever see them come up. You run a build with -DDC=CreatePak and it writes out one .ddp with everything derived during that build. Drop it in <Project>/DerivedDataCache/ and the engine picks it up on its own. No ini edits. No service. Nothing for a developer to configure, which matters more than it sounds like it does.
The whole thing is:
- Job starts, pulls the current pak out of object storage, writes it into the project’s DDC folder.
- Build.
- On main only, create a new pak and upload it.
S3-compatible object storage and nothing else. No cluster. No eviction policy to tune. No failover doc that nobody reads. And when it breaks — which is rare, because there’s so little of it to break — the build gets slow again. It doesn’t stop. That distinction is worth a lot at 2am.
Get Unreal Cloud’s stories in your inbox
Join Medium for free to get updates from this writer.
Worth emphasizing: don’t let PR jobs update the DDC. Only the main job updates the cache, and only on a successful build.
Where it falls over
Better you hear this from me than find out in month three.
A pak is one blob, so there’s no partial invalidation. All of it or none of it. On a fast-moving project a good chunk of what you’re pulling down is already dead weight.
You pay that download every job, too. Ours are around 4 GB, call it 60 seconds on our VMs. Totally fine when it’s saving 60+ minutes of compiling. Not fine when the ratio flips. And it does move as the project grows, so keep an eye on it rather than setting it up once and forgetting.
Only main helps you, as well. Work done in a PR build disappears when the machine does. Work on someone’s workstation never enters the picture at all. If that becomes an issue for you, you can also have PR-scoped caches that disappear once the PR is merged or closed. This is similar to how GitHub Actions caches work as well.
When to actually upgrade
Any one of these is enough on its own.
Workstations need in. Once the complaint turns into “my artists are sitting there waiting,” you’ve crossed from seeding into sharing, and paks have nothing for you.
The download starts eating the savings. When pulling the blob costs a real fraction of what it saves you, per-key fetching wins.
You’ve got someone to run it. An actual named person. Not a volunteer, not “we’ll sort that out later.”
Some studios will hit one of those inside a year. That’s fine. The claim was never that paks are the permanent answer — it’s that they’re a small thing to walk away from later, precisely because there’s so little of them to walk away from. Many times, they will be all you ever need.
Anyway
The pattern I keep running into isn’t teams stuck on primitive tooling. It’s small teams carrying operational weight they picked up before they needed it, because a thread told them to, and that weight compounds quietly while the problem it was meant to solve never quite turns up.
Put the file in the bucket. Watch it for a month. And, only upgrade when your own numbers tell you to.
Disclosure: I build Unreal Cloud, managed build infrastructure for UE projects, which is why I’ve spent an unreasonable amount of time thinking about derived data. It does the pak thing described above.
