Running Umbraco in Containers – Share Your Experience

[nikolajlauridsen]

Are you running Umbraco in production using Docker, Kubernetes, or something similar? We'd love to hear how it's working for you! 🤩

• What challenges have you run into? 🤔
• Any workarounds, tweaks, or hoops you've had to jump through? 🛠️
• Any common pitfalls we and others should watch out for? 👀

We’re looking to improve how Umbraco runs in containerized environments, and your input can help shape future updates. If you have a Dockerfile or setup you’d be open to sharing, that would be super helpful too! 😄

Your feedback will help shape how we support containerized deployments in the future. Let’s discuss! 💭

[jrunestone]

Hi!

We're running a load balanced site in Azure with Docker containers via GitHub actions deployments. Our stage and feature environments are also running the same dockerfile. We're also using the same(ish) dockerfile to develop locally with docker-compose. We can simulate the load balanced environment locally quite easily that way.

Overall it's working great. It's quite easy to adopt an Umbraco site to run in a container, even an existing one. I'd say the most work lies in creating/maintaining the deployment flow and the hardest challenges lies in debugging container errors once deployed to the platform. But neither of those things are specific to Umbraco.

Challenges (most of them in development actually):

• Both: Need to pass along license files into the container (umbraco forms for example)
• Dev: Need to map out local volumes properly so as not to run into some issues (in docker-compose)
• Dev: dotnet watch is making the Umbraco models builder trigger a rebuild before the models are rebuilt (added to no watch) - might not be specific to container (don't watch those files)

Production Dockerfile: https://gist.github.com/jrunestone/ab845ec50d3a8636c6bf44355b6cc8fc
Development Dockerfile: https://gist.github.com/jrunestone/713b921d1da31ead4a7e65445efc14c9
Development docker-compose.yml: https://gist.github.com/jrunestone/9c71cb4254e5ba28ed155f89fe117870

[bielu]

@jrunestone i recommend switching to alpine image as default .net image is based on debian with quite a lot vulnerabilities which are not gonna be resolved.

[jrunestone]

Thanks for the tip

[nul800sebastiaan]

Calling @mjpraxis and @CarlSargunar for their container feedback!

[bielu]

I did Umbraco with docker in both Azure and GCP, and my biggest challanges are still 3 things (based on v13):

• Examine and Nucache(I think the nucache bit is redudant because of v15, but yeah didnt test and check what files), which basically force either to map container path for them to keep them persistent or accept really slow cold boots
• Umbraco storying licences through files not db, so we need either include licenses in repository and deploy inside of docker image or map them through docker volumes
• Umbraco logger requirying logs being stored in file, in a lot cases it causes Azure to restart app and move it arround because of size of files. I would love it to be able to store logs anywhere but files system.

The other bits which i usually do is disabling the keep alive job from umbraco, as it just make things harder as you need allow outgoing traffic to app and if you loadbalance with waf it doesnt make sense to do that from app and is better to use healtchecks to keep it alive instead.

[nikolajlauridsen]

I think a good solution for the logging is to swap out the Serilog sink, that way you don't have to store the logs on the filesystem, you can use whatever you want, for instance Azure Table Storage or Application Insights 😄

[bielu]

@nikolajlauridsen currently it is impossible to remove umbraco sink it is hardcoded, I think it would be worth to make Umbraco sink wrapper so it can use underhood any sink instead. so we can store logs in SQL/Table STorage etc. but that's bigger refactor for umbraco than just allow disabling sink and log viewer 🤷‍♂️

[nikolajlauridsen]

That's a very good point. We currently read the logs directly from disk when serving them in the back office.

I agree that we should get a better solution than just disabling the log viewer, my only concern here is that different sinks will have different ways they can be read, which means that we can't provide this OOB, however, there should be a reasonable extension point like you mention, for you to provide your own implementation for reading logs, would also make a good use case for a package I guess 😄

I'll create a task in the backlog to investigate this, we want to be able not to write logs to disk at all 😄

[bielu]

@nikolajlauridsen i looked into that:
https://github.com/serilog-contrib/serilog-ui
there are solution for this already ;)

[BradleyMcCallionCOT]

Can also confirm the logging issue is the biggest issue here since the Kubernetes native way is to log to STDOUT. Logging makes the ephemeral writable layer for the container fill up quickly and can cause it to frequently crash as a result. Alternatively you have to implement a StatefulSet with a PVC for what could otherwise be a simpler stateless application.

[jrunestone]

I have lots of feedback on the Azure hosting and not just container specific if that's of any interest ;)

[mjpraxis]

@nikolajlauridsen we are running several v15 Backoffice instances i Kubernetes(K8S).
I think it was pretty easy to setup and it runs with no errors.

There was some fine-tuning in the beginning making sure that the setting was correct:

The public URL to the Backoffice

• Umbraco__CMS__WebRouting__UmbracoApplicationUrl
• Umbraco__CMS__Security__BackOfficeHost

Making sure that the setting UseHttps in K8S is false, since the internal traffic is HTTP.
This choice do make the Umbraco Health check making warning especially if you also set the Umbraco__CMS__Runtime__Mode to Production. There is a hack that you can remove the UseHttpsValidator in the UmbracoBuilder at startup.

As we are only using the Backoffice, we had to make sure that K8S only had 0 (zero) or 1 pod running.
We are stille adjusting some K8S settings to make it more smooth for the users, when a pod is spinning up.
And we try to monitoring the cpu/ram usage. So we can set the right values for the container app.

[PeterKvayt]

@mjpraxis
Did I understand right, that you are using more than one instance of umbraco backoffice with one database (umbraco 15)?
Could you please share more details related to your solution? I have the similar situation, and umbraco docs suggests to scale only frontend (not backoffice) instances, but in my case, I use backoffice instance only and I want to have an ability to scale it.

[mjpraxis]

@PeterKvayt
We are only running ONE instance of the backoffice (Umbraco v15) per database.
When fully in production, we will have aprox 250 Umbraco databases. But the K8S will make sure that each of the 250 backoffices only will be running if there is incomming requests.

We are not using the frontend part of Umbraco. The frontend is in a different system. So we are pushing content from Umbraco to a Postgresql database.

[remyili89]

So, as someone who runs the k8s infra and other related cloud resources for our Umbraco team i can remark the following things:

• Passing secrets that Umbraco needs on env level is easily done and tested using ESO (External Secrets Operator), using the same keyvault set Umbraco uses to fetch the rest of the secrets;
• Using the dotnet image to build Umbraco on, it is important to set boundaries on resource usage. Dotnet can go a little bit wild on memory sometimes (cache?)
• Readiness and Liveness probes sometimes take a bit longer and need tweaking, depending on backend.
• I am not an Umbraco/Dotnet developer but i feel like passing a configMap with all the umbraco parameters in there, and read it from there, would be a lot easier and more controlling than passing env vars.

Ill edit this if more things come to mind.

[mjpraxis]

Regarding the configuration/appsettings for dotnet app.
There are several ways of feeding the settings to the app besides appsettings files and environment variables.

I remembered that I once saw a pretty good In-Depth video from .NET conf back in 2023.
Maybe you could get inspired of that: https://www.youtube.com/watch?v=aOXaBZFB0-0

In our solution we get for example the DB connectionstring by environment variabel. Before the UmbracoBuilder is build, we make some changes to it (just something we have to :-) )

[CarlSargunar]

My experience with containers is mostly through Github actions and Azure Container apps, but for my contribs:

• Local developnent is different for everyone, but I find a mix of working with .NET Aspire (as it's SUPER quick to get up and running from cloning a repo) and containers helps - but the same can be achieved with Docker Compose, and it depends on your setup.
• Env variables are used everywhere to configure each container (there are other options, but this worked for me)
• Use OpenTelemetry or the Serilog Application Insights sink to centralize logging is a must - OTel might be overkill, since the serilog app insights sink is enough most of the time, but you don't want to have to go digging for logs
• Licenses - not had to worry about these, but I experimented a while back with storing license in a volume which can then be accessed by any of the containers which needed them

[yannickjemmely]

Hi @nikolajlauridsen ,

We're running multiple websites (low, medium and big size) on our own k8s infrastructure since Umbraco 9. We are currently using the dotnet sdk and runtime official dotnet images. All our sites were recently migrated to Umbraco 13.

Here are key features of our environment:

• Website are deployed through a custom helm chart updated by Gitlab pipeline
• Secret are stored as k8s secrets and injected into the pods regarding the standard Helm mechanism
• A volume is used to store media and logs
• Website are deployed on three environments (development, integration and production) across multiple clusters
• Models generation is always disabled on production environment
• We're able to pull database from production locally to work on "real" data if necessary
• New document types are created in production and then database is pulled locally to generate the model. Finally we push them back in production in the next release
• We use single pod strategy because our CSI driver does not support volume access by multiple pods yet
• Website access is managed by a k8s nginx controller
• Dockerfile is quiet simple and close to the one generated by Visual Studio
• We defined memory and cpu limits and requests for each websites to keep control of resources usages
• Backend authentication is managed through Azure
• We develop and maintain our own nugget package which are stored in a private registry on Gitlab
• Logs are kept as json file but also redirected in console to be able to work with them with Kibana
• We use "User secrets" to deal with secrets locally

Our infrastructure is not perfect yet but allows each developer to work and deploy changes on environments using the Gitlab pipeline. The git flow and project structure is standardized for all website.

I'll be happy to give more information if needed

[nul800sebastiaan]

Just adding a link to this older setup from @liamlaverty (maybe he has new insights since then) https://24days.in/umbraco-cms/2023/automatons/umbraco-docker/

Also some great insights from Scott Hanselman on how to optimize your .NET build to be as tiny as possible (basically treeshaking) and keeping the docker images tiny as well: https://www.youtube.com/watch?v=KqhhaMgbGhU

[liamlaverty]

Since writing this, a pitfall I've come across a few times while supporting production containerised Umbraco apps is with SSL, and Umbraco's backoffice redirection.

In containerised environments, it's common to have the application inside the container listening on HTTP (non-ssl). Then you have something like Azure Front-door (or some other load balancer) deal with the ingress being converted to/from SSL. You do this so that the SSL cert doesn't need to be installed onto the container itself.

For most features, this is fine, but Umbraco has a config Umbraco.CMS.Global.UseHttps. In a non-containerised installation (where the app is listening on HTTPS only) this redirects any backoffice requests from http:// to https://, and any front-end links generated in the rich-text-editor are https://. Similarly, some Umbraco agencies have some hardcoded redirection middleware which forces https redirections.

When containerised, this can create a situation where your production site is fully SSL enabled, but gets stuck in a redirect-to-https loop (even for https links), because the application only ever listens on the http port, and thinks your https link is actually a http link.

One quick solution is to set Umbraco.CMS.Global.UseHttps to false, but then you end up with a different set of problems, where your backoffice healthcheck dashboard complains that the app is running insecurely, and some of your links on the front-end are to http instead of https addresses.

[NyXi4]

We are running Umbraco 13.7.2 to support two sites with medium to high traffic and are currently hosting on an AKS cluster.
For our setup we have two different services - the first (editorial) having only one pod instance is responsible for handling the CMS back office and does write operations to the database. The second service (public) which is load balanced handles external traffic and is read-only in terms of database transactions.

We previously were hosting with Azure app services and Umbraco 8 and since the migration have hit some issues with locking when content editing is happening.

This has been linked to us doing blue/green deployments during releases to avoid downtime. The approach works well for the public facing service but when we were doing it for the editorial one we hit locking issues which spanned long after the release had been complete and the older version pods decommissioned. The locking errors where observed when content creators would try to save/publish from the CMS.

After conversations with Umbraco support we have stopped doing blue/green for the editorial service and accepted downtime during releases and start up. Interestingly enough with Azure app services we did not have those problems and had a staging slot which was swapped with production as part of the release. Ideally, we would want to get back to doing this way but not certain how/if it is possible at this point in time.

[bielu]

@NyXi4 you were lucky to have it working with azure slots, as it is not support and I did experience same thing with locking on azure with slots, umbraco backoffice doesnt like be load balanced and slots are counting as load balancing!

[erikjanwestendorp]

We’ve been hosting almost all our clients on a Kubernetes cluster since the release of Umbraco 9. Overall, this has been smooth, with only a few minor issues—mostly related to some Views (especially from third-party packages) and case sensitivity due to the non-Windows file system.

We’ve built several CI/CD pipelines for hosting Umbraco in containers. Here’s an example of the setup we commonly use:
👉 Umbraco on Kubernetes (24 Days)

We also have experience running Umbraco on AWS:
👉 Umbraco on AWS (24 Days)

More recently, I did a proof of concept for running Umbraco on Google Cloud:
👉 Umbraco on Google Cloud (24 Days)

We haven’t encountered any major Umbraco-related issues when hosting in Docker containers or on a Kubernetes cluster. A recent challenge involved working with private packages, which we resolved using Microsoft’s artifacts-credprovider.

Our Dockerfile is relatively straightforward—here’s an example:
👉 Sample Dockerfile (gist)

Just don’t forget to pass the FEED_ACCESSTOKEN as a build argument.

[jamie-oconnell]

Am running around 20 sites currently under Azure app service but wanting to move them in to our k8s clusters in the near future. Primarily for the benefit of utilizing common tooling and the flexibility we have around compute scale. Glad to hear their are others out there successfully hosting this way.

From testing, the environment variables needed are

Umbraco__CMS__Global__UseHttps: 'false'
ASPNETCORE_URLS: 'http://+:8080/'
ASPNETCORE_FORWARDEDHEADERS_ENABLED: 'true'
Umbraco__CMS__KeepAlive__DisableKeepAliveTask: 'true'
Umbraco__CMS__Examine__LuceneDirectoryFactory: 'Default'

Here is a redacted Dockerfile. Using a chiseled image keeps the final output at around ~250mb.

# syntax=docker/dockerfile:1

# Client build stage
FROM node:18 AS client-build-env

WORKDIR /src/Client

COPY src/Client/. .

RUN npm install && \ 
    npm run build -- --output-path=/src/Client/build/client && \ 
    npm run build:server -- --output-path=/src/Client/build/server

# Publish Stage
FROM mcr.microsoft.com/dotnet/sdk:8.0-alpine AS publish

WORKDIR /src/build

COPY --from=client-build-env /src/Client/build/client ./CMSWeb/wwwroot/scripts/
COPY --from=client-build-env /src/Client/build/server ./CMSWeb/wwwroot/scripts/server
COPY src/ .

WORKDIR /src/build/CMSWeb

RUN dotnet restore "./CMSWeb.csproj" --runtime linux-x64 && \ 
    dotnet publish "./CMSWeb.csproj" -c Release -o /app/publish --no-restore --runtime linux-x64

RUN mkdir -p /app/publish/wwwroot/media

# Runtime stage
FROM mcr.microsoft.com/dotnet/aspnet:8.0-jammy-chiseled-extra AS final

USER $APP_UID
ENV ASPNETCORE_HTTP_PORTS=8080
ENV ASPNETCORE_HTTPS_PORTS=8443
EXPOSE $ASPNETCORE_HTTP_PORTS
EXPOSE $ASPNETCORE_HTTPS_PORTS

WORKDIR /app
COPY --from=publish --chown=$APP_UID:$APP_UID --chmod=760 /app/publish .

ENTRYPOINT [ "dotnet", "./CMSWeb.dll"]

Biggest challenge I face is not necessarily to do with containers or k8s but common across load balancing Umbraco in general. Is the examine indexes. For testing I created the load balanced front end containers as stateful sets with a mapped umbraco/Data and the LuceneDirectoryFactory set to default so the indexes get stored at umbraco/Data/TEMP. However when scaling up these take time to build and I haven't found a reliable Umbraco endpoint that I can use as reediness check that will prevent traffic being served to new containers until the indexes are ready.

Here is a redacted manifest

--- 
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: umbraco-frontend
spec:
  serviceName: "umbraco-frontend"
  replicas: 3
  selector:
    matchLabels:
      app: umbraco-frontend
  template:
    metadata:
      labels:
        app: umbraco-frontend
        azure.workload.identity/use: "true"
    spec:
      securityContext:
        runAsUser: 1654
        runAsGroup: 1654
        fsGroup: 1654
      serviceAccountName: umbraco
      containers:
      - name: umbraco-frontend
        imagePullPolicy: Always
        image: example.azurecr.io/umbraco:latest
        ports:
        - name: http
          containerPort: 8080
        envFrom:
        - configMapRef:
            name: umbraco
        env:
        - name: AppSettings__Environment
          value: 'Subscriber'
        volumeMounts:
          - name: umbraco-data
            mountPath: /app/umbraco/Data
  volumeClaimTemplates:
  - metadata:
      name: umbraco-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 1Gi

Would be great to connect with any community members that are hosting in a similar fashion to knowledge share.

[bielu]

@jamie-oconnell currently only reliable way for examine indexes is to use my elastic package or shannons Examinex, as currently each instance of umbraco will mantain own instance of search index, which is not great for load balancing :) about endpoint, there is no option to check readiness curently of indexes 🤷

[nikolajlauridsen]

Thank you, everyone, for all your contributions. It has been amazing to follow along with all the great knowledge sharing 🤩 It's greatly appreciated, H5YR.

We initially started this discussion to see if there were any technical problems we needed to solve in the CMS. Thankfully, this is not the case 😄.

I think a good next step is to create some documentation with information about what you need to be aware of when hosting Umbraco with Docker. Some things I've noted down for this are

• Storing media - ideally using something like Azure Blob Storage or similar, but volumes are also a possibility.
• Volumes and files, in particular license files.
• Alternative sinks for Serilog.
• HTTPS - How containers might want to use HTTP, and how to disable health checks and runtime mode checks when doing this.
• General workflow process, I.E, don't create/edit files from the backoffice.

This is not intended to be a strict guide, since the specifics are very project-dependent, but rather general information about what you need to be aware of when hosting Umbraco in Docker.

I'll leave this discussion open for a week so you have an opportunity to come up with input on this, and then I'll close it.

[nikolajlauridsen]

I've now created a document in the docs for running Umbraco in Docker. I've made it as more of a general overview of what to consider when hosting in Docker. Now I'm not a hosting guru, so if you have any improvements, they're greatly appreciated as contributions over on the documentation repo.

I'll close this discussion now, thank you all again for your participation 🤩 .