In this post, we’ll detail our solution for publishing, and consuming, Kotlin Multiplatform Swift Packages. Our solution leverages a custom Gradle plugin publishing XCFrameworks to Google Cloud Storage and a Google Cloud Run service to download Swift Package binaries requested by XCode. With this solution in place, we’ve been able to more efficiently serve multiple Kotlin Multiplatform libraries to our iOS application.

This is a part of an ongoing series on our usage of Kotlin Multiplatform at Premise:

• Part 1: Kotlin Multiplatform at Premise
• Part 2: Kotlin Multiplatform Project Structure for Integrating with Brownfield Applications
• Part 3: Building a CI Pipeline for Kotlin Multiplatform Mobile Using GitHub Actions
• Part 4: Publishing Kotlin Multiplatform Swift Packages Using Google Cloud Storage and Cloud Run — This Post
• Part 5: Generating BuildConfig Files for a Kotlin Multiplatform Library — Coming Soon
• Part 6: Optimizing Local Build Times for Kotlin Multiplatform Mobile Projects — Coming Soon

Premise and Kotlin Multiplatform Swift Packages

We’ve been using Kotlin Multiplatform in production since early 2021 in the form of our mobile-shared project.

During that time, we’ve consumed our shared code as a Swift Package within our iOS application. The integration of that Swift Package has gone through several iterations.

• v1: Use the multiplatform-swiftpackage plugin to build the Swift Package and store the XCFramework binary in GitHub
• v2: Use our own custom Gradle plugin to build the Swift Package and store the XCFramework binary in GitHub

These two solutions were very similar. Build the XCFramework. Generate the Package.swiftfile. Check both into git with the desired version tag.

These approaches worked fine for a while, but eventually we started to pay the price for our simple initial solution.

An XCFramework binary can be pretty large. Ours were in the ballpark of 100MB. So checking 2–3 of these into each commit (1 for each iOS architecture) started to really increase the overall download size of our git repo.

Why is this a problem?

Well, Swift Package Manager does a full clone of the git repository when it has to sync package dependencies or reset package caches. So, when our developers had to switch branches with difference library versions, or had to reset their Xcode caches, they had to download GBs worth of XCFramework binaries; thereby dramatically slowing down their Xcode sync.

Once we determined what was causing this slow down, we knew we needed a new approach. We decided to move to a remote url distribution model for our Kotlin Multiplatform Swift Packages. Rather than check the binary into source control, we would upload the binary to some remote location, acquire the url to that resource, and add that url, along with the checksum, to our Package.swift file.

This is conceptually pretty simple. However, there wasn’t an ideal out-of-the-box solution at the time. So, we built a solution ourselves which has been serving us well in managing the publication and integration of Swift Packages for multiple Kotlin Multiplatform projects.

The solution is broken down into three parts:

• Uploading XCFramework binaries to Google Cloud Storage
• Publishing the Package.swift file to GitHub
• Consuming the Swift Package from Xcode using Google Cloud Run

Uploading XCFramework Binaries to Google Cloud Storage

As part of our Kotlin Multiplatform project, we have a custom Gradle plugin named SwiftPackagePlugin that adds several key tasks to our build.

class SwiftPackagePlugin : Plugin<Project> {
    override fun apply(target: Project) {
        target.registerMoveXCFrameworkTask()
        target.registerZipFileTask()
        target.registerSwiftPackageTask()
        target.registerLocalSwiftPackageTask()
    }
}

registerMoveXCFrameworkTask moves a build XCFramework from the build directory to our desired output directory location

/**
 * Creates a moveXCFramework Gradle task to move files from the
 * /build/swiftpackage directory to a /swiftpackage output directory
 */
private fun Project.registerMoveXCFrameworkTask() {
    tasks.register("moveXCFramework", Copy::class.java) {
        group = TASK_GROUP_NAME
        description = "Moves XCFramework into swiftpackage output directory"

        dependsOn("assemble${rootProject.name.capitalized()}XCFramework")

        from("${this@registerMoveXCFrameworkTask.buildDir}/$OUTPUT_FILE_DIR_NAME")
        into(File(OUTPUT_FILE_DIR_NAME))
    }
}

registerZipFileTask creates a zip containing the build XCFrameworks

/**
 * Creates a zipXCFramework Gradle task
 */
private fun Project.registerZipFileTask() {
    tasks.register("zipXCFramework", Zip::class.java) {
        group = TASK_GROUP_NAME
        description = "Creates a ZIP file for the XCFramework"

        // dependsOn assembleMobilesharedXCFramework task
        dependsOn("assemble${rootProject.name.capitalized()}XCFramework")

        archiveFileName.set(project.getZipFileName())
        destinationDirectory.set(File("${this@registerZipFileTask.buildDir}/$OUTPUT_FILE_DIR_NAME"))
        from("${this@registerZipFileTask.buildDir}/XCFrameworks/release")
    }
}

registerLocalSwiftPackageTask constructs a Package.swift file useful for locally testing changes in our iOS project

/**
 * Creates a createLocalSwiftPackage Gradle task that generates a
 * Package.swift file needed for the generated XCFramework so it
 * can be consumed as a local Swift Package
 */
private fun Project.registerLocalSwiftPackageTask() {
    tasks.register("createLocalSwiftPackage") {
        group = TASK_GROUP_NAME
        description = "Creates a local Swift package to distribute the XCFramework"

        dependsOn("zipXCFramework")
        finalizedBy("moveXCFramework")

        doLast {
            val packageDotSwiftFile = createPackageDotSwiftFile(
                outputDirName = OUTPUT_FILE_DIR_NAME,
                filename = OUTPUT_PACKAGE_CONFIG_FILE_NAME
            )

            val packageConfiguration = SwiftPackageConfiguration.local(
                packageName = rootProject.name,
                zipFileName = project.getZipFileName()
            )

            packageDotSwiftFile.writePackageDotSwiftFile(packageConfiguration)
        }
    }
}

registerSwiftPackageTask uploads a zipped set of XCFrameworks to Google Cloud Storage, builds a SwiftPackageConfigurationand then constructs a Package.swift file pointing to this remote file.

/**
 * Generates the Package.swift file needed for the generated
 * XCFramework so it can be consumed as a Swift Package
 */
private fun Project.registerSwiftPackageTask() {
    tasks.register("createSwiftPackage") {
        group = TASK_GROUP_NAME
        description = "Creates the Swift package to distribute the XCFramework"

        dependsOn("zipXCFramework")

        doLast {
            val packageDotSwiftFile = createPackageDotSwiftFile(
                outputDirName = OUTPUT_FILE_DIR_NAME,
                filename = OUTPUT_PACKAGE_CONFIG_FILE_NAME
            )

            uploadeToCloudStorage(
                projectId = project.swiftPackageGCPProjectId,
                bucketName = project.swiftPackageBucketName,
                packageName = project.getZipFileName(),
                filePath = "${project.buildDir}/$OUTPUT_FILE_DIR_NAME/${project.getZipFileName()}"
            )

            val packageConfiguration = SwiftPackageConfiguration.remote(
                packageName = rootProject.name,
                zipFileName = project.getZipFileName(),
                zipChecksum = project.zipFileChecksum().trim(),
                distributionUrl = project.swiftPackageDistributionURL
            )

            packageDotSwiftFile.writePackageDotSwiftFile(packageConfiguration)
        }
    }
}

Uploading to Google Cloud Storage

The implementation of uploadToCloudStorage looks like the following. This relies on the com.google.cloud:google-cloud-storage:<version> dependency for interacting with Cloud Storage. By extension this also means the task must be run from an environment authenticated with Google Cloud. Locally, this generally means authentication via application-default credentials. In CI, we use the google-github-actions/auth GitHub Action to authenticate via a service account that has permissions to upload to our desired Cloud Storage bucket.

/**
 * Uploads a specified XCFramework zip file to a specified Cloud Storage bucket
 *
 * @param projectId The ID of your GCP project
 * @param bucketName The Cloud Storage bucket, within the specific project, to upload zip files to
 * @param packageName The name of the XCFramework zip file
 * @param filePath Path to the file to upload
 */
private fun uploadeToCloudStorage(projectId: String, bucketName: String, packageName: String, filePath: String) {
    val storage = StorageOptions.newBuilder().setProjectId(projectId).build().service
    val blobId: BlobId = BlobId.of(bucketName, packageName)
    val blobInfo: BlobInfo = BlobInfo.newBuilder(blobId).build()

    // Optional: set a generation-match precondition to avoid potential race
    // conditions and data corruptions. The request returns a 412 error if the
    // preconditions are not met.
    val precondition = if (storage[bucketName, packageName] == null) {
        // For a target object that does not yet exist, set the DoesNotExist precondition.
        // This will cause the request to fail if the object is created before the request runs.
        Storage.BlobWriteOption.doesNotExist()
    } else {
        // If the destination already exists in your bucket, instead set a generation-match
        // precondition. This will cause the request to fail if the existing object's generation
        // changes before the request runs.
        Storage.BlobWriteOption.generationMatch(
            storage[bucketName, packageName].generation
        )
    }

    storage.createFrom(blobInfo, Paths.get(filePath), precondition)
}

Building a Package.swift file

To build the desired Package.swift file we construct an instance of a SwiftPackageConfiguration which will hold the values we want to write out to disk

private data class SwiftPackageConfiguration private constructor(
    private val packageName: String,
    private val swiftToolsVersion: String,
    private val platforms: String,
    private val zipFileName: String,
    private val zipChecksum: String,
    private val isLocal: Boolean,
    private val distributionUrl: String,
) {

    val templateProperties = mapOf(
        "toolsVersion" to swiftToolsVersion,
        "name" to packageName,
        "zipName" to zipFileName,
        "platforms" to platforms,
        "isLocal" to isLocal,
        "checksum" to zipChecksum,
        "url" to "$distributionUrl/$zipFileName"
    )

    companion object {
        internal val templateFile =
            SwiftPackagePlugin::class.java.getResource("/templates/$OUTPUT_PACKAGE_CONFIG_FILE_NAME.template")

        fun local(
            packageName: String,
            swiftToolsVersion: String = DEFAULT_SWIFT_TOOLS_VERSION,
            platforms: String = DEFAULT_IOS_PLATFORM_VERSION,
            zipFileName: String
        ) = SwiftPackageConfiguration(packageName, swiftToolsVersion, platforms, zipFileName, "", true, "")

        fun remote(
            packageName: String,
            swiftToolsVersion: String = DEFAULT_SWIFT_TOOLS_VERSION,
            platforms: String = DEFAULT_IOS_PLATFORM_VERSION,
            zipFileName: String,
            zipChecksum: String,
            distributionUrl: String,
        ) = SwiftPackageConfiguration(packageName, swiftToolsVersion, platforms, zipFileName, zipChecksum, false, distributionUrl)
    }
}

With this class, we then configure our instance as follows

val packageConfiguration = SwiftPackageConfiguration.remote(
                packageName = rootProject.name,
                zipFileName = project.getZipFileName(),
                zipChecksum = project.zipFileChecksum().trim(),
                distributionUrl = project.swiftPackageDistributionURL
            )

With an instance of this SwiftPackageConfiguration we then build our Package.swift file by using a template and subsituting in our desired configuration values.

// Writes the Package.swift file by
// first loading from a template file and
// second subtituting in the config values
private fun File.writePackageDotSwiftFile(packageConfiguration: SwiftPackageConfiguration) {
    SimpleTemplateEngine()
        .createTemplate(SwiftPackageConfiguration.templateFile)
        .make(packageConfiguration.templateProperties)
        .writeTo(writer())
}

The base template file lives in /src/main/resources/templates/Package.swift.template

// swift-tools-version:$toolsVersion
import PackageDescription

let package = Package(
    name: "$name",
    platforms: [
        $platforms
    ],
    products: [
        .library(
            name: "$name",
            targets: ["$name"]
        ),
    ],
    targets: [
<% if (isLocal) print """        .binaryTarget(
            name: "$name",
            path: "./${zipName}"
        ),""" else print """        .binaryTarget(
            name: "$name",
            url: "$url",
            checksum: "$checksum"
        ),""" %>
    ]
)

When the configuration is applied to the template, it resulst in an output Package.swift file that looks like the following:

// swift-tools-version:5.8.0
import PackageDescription

let package = Package(
    name: "mobileshared",
    platforms: [
        .iOS(.v14)
    ],
    products: [
        .library(
            name: "mobileshared",
            targets: ["mobileshared"]
        ),
    ],
    targets: [
        .binaryTarget(
            name: "mobileshared",
            url: "https://<cloud run url>/swiftpackage/mobileshared-<version>.zip",
            checksum: "<XCFramework zip checksum>"
        ),
    ]
)

Note the url value. This is a url pointing to a Google Cloud Run service which will be described below. The purpose of this service is to give us a known location to build up our artifact urls so we can add them to this Package.swift file.

Additional plugin code

To round out the plugin implementation, we have a handful of of other configuration values and helper functions/classes.

private const val DEFAULT_SWIFT_TOOLS_VERSION = "5.8.0"
private const val DEFAULT_IOS_PLATFORM_VERSION = ".iOS(.v14)"

private val Project.swiftPackageGCPProjectId
    get() = System.getenv("SWIFT_PACKAGE_BUCKET_GCP_PROJECT_ID") ?: ""

private val Project.swiftPackageBucketName
    get() = System.getenv("SWIFT_PACKAGE_BUCKET_NAME") ?: ""

private val Project.swiftPackageDistributionURL
    get() = System.getenv("SWIFT_PACKAGE_DISTRIBUTION_URL") ?: ""

private fun Project.createPackageDotSwiftFile(outputDirName: String, filename: String): File {
    return File("$buildDir/$outputDirName", filename).apply {
        parentFile.mkdirs()
        createNewFile()
    }
}


private const val TASK_GROUP_NAME = "Premise Swift Package"
private const val OUTPUT_PACKAGE_CONFIG_FILE_NAME =  "Package.swift"
private const val OUTPUT_FILE_DIR_NAME = "swiftpackage"
private fun Project.getZipFileName() = "${rootProject.name}-${rootProject.version}.zip"

internal fun Project.zipFileChecksum(): String {
    val outputPath = "${buildDir}/$OUTPUT_FILE_DIR_NAME"
    logger.info("checksum for path: $outputPath")
    logger.info("checksum for file: ${getZipFileName()}")
    return File(outputPath, getZipFileName())
        .takeIf { it.exists() }
        ?.let { zipFile ->
            ByteArrayOutputStream().use { os ->
                project.exec {
                    workingDir = File(outputPath)
                    executable = "swift"
                    args = listOf("package", "compute-checksum", zipFile.name)
                    standardOutput = os
                }
                os.toString()
            }
        } ?: ""
}

Publishing the Package.swift File to GitHub

With our SwiftPackagePlugin applied to our project, we can run the createSwiftPacakge task from CI to build the binaries, upload them to Google Cloud Storage, and generate a Package.swift referencing that version.

However, we will need to publish that Package.swift file to GitHub so Xcode can reference it, and use it to eventually download the binaries.

To do this, we are using GitHub actions. The following, is a snippet from our release build that includes the neccessary job steps to build and publish the Swift Package. (this does not include the neccessary gcloud env setup)

# runs our Gradle tasks for building and uploading
- name: Create Swift Package
  uses: gradle/gradle-build-action@v2.4.2
  with:
    arguments: createSwiftPackage

# copies Package.swift to desired location in repo so Xcode sees it
- name: Prepare iOS Artifacts
  run: |
    cp -r shared/build/swiftpackage/Package.swift .

# commits the updated Package.swift and tags it so Xcode can find it
- name: Tag Release
  run: |
    git add --all
    git commit -m "[skip ci] Update Package.swift to version ${{ env.ARTIFACT_VERSION }}"
    git tag -a ${{ env.ARTIFACT_VERSION }} -m "mobile-shared v${{ env.ARTIFACT_VERSION }}"
    git push
    git push origin ${{ env.ARTIFACT_VERSION }}

Consuming the Swift Package from Xcode Using Google Cloud Run

From Xcode, we can now point our project at GitHub and ask it for a specific tagged version. Xcode and Swift Package Manager will then

• download the repo
• examine the Package.swift file for the tagged commit
• extract the binaryTarget url
• attempt to download the binary

When building the Package.swift file, we construct the url using the name of the shared artifact, the version of that artifact, and add those values to a known service url.

https://<cloud run service url>/swiftpackage/<artifact name>-<version>.zip

That service is our swift-package-distribution-service .

It’s a Google Cloud Run service that is permissioned to access our Cloud Storage bucket and stream the requested files back for download.

Implementing swift-package-distribution-service

How you would go about implementing your own version of this type of service is for you to decide. We built our service with Python using Goblet. Goblet is a framework designed to streamline the creation og Google Cloud Services using Python.

With Goblet, our entire service looks like the following:

import io
import os
from goblet import Goblet, goblet_entrypoint, Response
from flask import stream_with_context
from google.cloud import storage
import base64

CHUNK_SIZE = 8192
BUCKET_NAME = os.environ.get("CLOUD_STORAGE_BUCKET")
SWIFT_DISTRIBUTION_USERNAME = os.environ.get("SWIFT_DISTRIBUTION_USERNAME")
SWIFT_DISTRIBUTION_TOKEN = os.environ.get("SWIFT_DISTRIBUTION_TOKEN")

app = Goblet(function_name="swift-package-distribution-service", backend="cloudrun", routes_type="cloudrun")
goblet_entrypoint(app)


@app.route('/swiftpackage/{name}')
def home(name: str):
    auth_header = get_auth_token()

    if not is_token_valid(auth_header):
        return Response("Invalid token", None, 401)

    swiftpackage_filename = f'{name}'
    storage_client = storage.Client()

    bucket = storage_client.get_bucket(BUCKET_NAME)
    if bucket is None or not bucket.exists():
        missing_bucket_msg = f'Bucket {bucket} does not exist'
        app.log.info(missing_bucket_msg)
        return Response(missing_bucket_msg, None, 404)
    else:
        app.log.info(f'Located bucket {bucket}')

    swiftpackage_blob = bucket.get_blob(swiftpackage_filename)
    if swiftpackage_blob is None or not swiftpackage_blob.exists():
        missing_blob_msg = f'Requested Swift Package {swiftpackage_filename} does not exist'
        app.log.info(missing_blob_msg)
        return Response(missing_blob_msg, None, 404)
    else:
        app.log.info(f'Located file {swiftpackage_filename} and started download')

    swiftpackage_content = swiftpackage_blob.download_as_bytes()

    app.log.info("Streaming response")
    return stream_response(swiftpackage_content, swiftpackage_filename)


def stream_response(content: bytes, filename: str):
    return stream_with_context(
        read_bytes_chunks(content)), \
        200, \
        {
            "Content-Type": "application/octet-stream",
            "Content-Disposition": f'attachment; filename={filename}'
        }


def read_bytes_chunks(content: bytes):
    app.log.info("Reading chunks")
    content_stream = io.BytesIO(content)
    with content_stream:
        while 1:
            buffer = content_stream.read(CHUNK_SIZE)
            if buffer:
                app.log.info("chunk")
                yield buffer
            else:
                app.log.info("no chunks left")
                break


def get_auth_token() -> str:
    # We'll skip this specific implementation
    # We include a token taken from .netrc on developers' and CI machines
    # If the token is not present, or incorrect, we will reject the request
    # This is used due to Swift Package Manager not allowing us to auth via gcloud


def is_token_valid(auth_header: str) -> bool:
    # We'll skip this specific implementation


def decode_auth_header(auth_header: str) -> str:
    # We'll skip this specific implementation

With this service deployed, we can use its url when building our Package.swift file.

https://<cloud run service url>/swiftpackage/<artifact name>-<version>.zip

And now, when Xcode tries to download the file referenced in Package.swift it makes a request against this service, validates the auth token, and downloads the desired XCFramework binaries.

From there, our iOS build is off and running.

How Has This Solution Worked?

We’ve been using this solution for 6+ months now without issue.

By enabling us to move away from checking binaries into GitHub, it dramatically improved package sync time for our iOS devs that were regularly switching between branches requiring different versions of our mobile-shared project.

Because the plugin, and service, are general purpose, we were able to apply the same plugin and deployment model for our second Kotlin Multiplatform project in the org.

Because the plugin and service were both already in place, applying this approach to the second project was trivial; not requiring any real new development work beyond configuring the plugin in the new project.

What Are The Drawbacks To This Solution?

Perhaps the biggest drawback to this solution is that it requires cloud resources for storage and request handling. In an organization already using cloud resources this is likely not an issue. However, for open source projects or hobbyists, this could be a significant barrier to entry; both in terms of cost required to run these resources and the administration/security of cloud resources/organizations/etc.

On a similar note, another drawback is that it’s not an off-the-shelf solution. It required dev work leveraging knowledge of Swift Package Manager, Gradle plugins, and Google Cloud. This might not be something every time has access to and could be an additional barrier to entry. For a more, off-the-shelf option, I recommend looking into KMMBridge from TouchLab.

Finally, another, albiet smaller, limitation of the current implementation is the ordering of binary upload and Package.swift file creation. If uploading the XCFramework binary to Cloud Storage succeeds, but creation of the Package.swift file fails, it would result in a stranded artifact in our Cloud Storage bucket. In practice, this wouldn’t result in any significant cost unless it was happening repeatedly; and in 6+ months of using the current implementation I don’t believe we’ve ever encountered this issue.

Premise is constantly looking to hire top-tier engineering talent to help us improve our front-end, mobile offerings, data and cloud infrastructure. Come find out why we’re consistently named among the top places to work in Silicon Valley by visiting our Careers page.

Publishing Kotlin Multiplatform Swift Packages Using Google Cloud Storage and Cloud Run was originally published in Engineering at Premise on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this post, we’ll walk through how to deploy a Ktor service Docker container from Artifact Registry to Cloud Run Using Gradle. This post builds on our two previous posts:

• How to Publish a Ktor Docker Image to Artifact Registry Using the Ktor Plugin
• How to Publish a Ktor Docker Image to Container Registry Using the Ktor Plugin

Using the approaches outlined in the previous series posts, we can leverage Gradle to automate build tasks for our service deployment. Primarily, we can use Gradle to compile our code, build our Docker image and deploy it to some container registry such as Artifact Registry.

To complete our deployment pipeline, we’ll once again use Gradle to automate the deployment of a Docker image from Artifact Registry to Cloud Run. By doing so, we can leverage Gradle’s task avoidance characteristics to avoid building or deploying if our source code hasn’t changed. Additionally, we keep deployment logic contained in a common language and build system as the rest of our Ktor service code; making it easier for developers to maintain both the application and deployment code.

Prerequisites

Since we’ll be deploying our Ktor service container to Artifact Registry, that comes with a new Google Cloud Platform (GCP) prerequisites.

We need to have access to a GCP organization

1. How to setup a GCP organization?
2. You’ll eventually need to set up a billing account for your organization. To try out GCP for learning purposes, there are often ways to find free, or discounted, billing credits
   - Trial period with $300 free credits for new users
   - GCP credits for faculty and students
   - Google Cloud Innovators Plugs program

We need to have a project set up within that organization

• How to create and manage GCP projects?

Artifact Registry must be enabled within that project

• How to enable Artifact Registry for a GCP project?

Cloud Run must be enabled within that project

• How to enable Cloud Run for a GCP project?

Reviewing Artifact Registry Deployment

Before we start on deploying to Cloud Run, let’s review our Gradle buildscript configuration for publishing our Ktor service to Artifact Registry.

We have a single Gradle module named :service to start. We have a simple class ArtifactRegistry that implements the DockerImageRegistry interface coming from the Ktor plugin. This allows us to configure the needed information to publish our built Docker image to Artifact Registry.

// :service/build.gradle.kts

private class ArtifactRegistry(
    projectName: Provider<String>,
    regionName: Provider<String>,
    repositoryName: Provider<String>,
    imageName: Provider<String>,
) : DockerImageRegistry {
    override val username: Provider<String> = provider { "" }
    override val password: Provider<String> = provider { "" }
    override val toImage: Provider<String> = provider {
        val region = regionName.get()
        val project = projectName.get()
        val repository = repositoryName.get()
        val image = imageName.get()

        "$region-docker.pkg.dev/$project/$repository/$image"
    }
}

Next, we can configure an instance of our ArtifactRegistry class; pulling the configuration data from environment variables and project properties.

// :service/build.gradle.kts

// deployment config
private val deploymentProjectName = System.getenv("GCP_CONTAINER_PROJECT_NAME")
private val deploymentRegion = System.getenv("GCP_REGION")
private val artifactRegistryContainerName = System.getenv("GCP_CONTAINER_NAME")
private val deploymentEnv = System.getenv("ENV")
private val deployedImageTag: String = "${rootProject.version}"
private val deployedImageName = rootProject.name

private val artifactRegistryConfig = ArtifactRegistry(
    projectName = provider { deploymentProjectName },
    regionName = provider { deploymentRegion },
    repositoryName = provider { artifactRegistryContainerName },
    imageName = provider { deployedImageName },
)

Lastly, we put it all together by configuring the Ktor plugin to build the Docker image and set our ArtifactRegistry config property as the externalRegistry for publication.

// :service/build.gradle.kts

ktor {
    docker {
        jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)
        localImageName.set(deployedImageName)
        imageTag.set(deployedImageTag)

        portMappings.set(
            listOf(
                io.ktor.plugin.features.DockerPortMapping(
                    outsideDocker = 8080,
                    insideDocker = 8080,
                    io.ktor.plugin.features.DockerPortMappingProtocol.TCP,
                ),
            ),
        )
        externalRegistry.set(artifactRegistryConfig)
    }
}

With our build.gradle.kts file configured in this way, we can run the publishImage Gradle task to build our Ktor project as a Docker image and publish it to Artifact Registry.

Building a Custom Gradle Plugin to Deploy from Artifact Registry to Cloud Run

What are we trying to automate?

To deploy an image from Artifact Registry to Cloud Run we have several options. Perhaps the most straightforward is using the gcloud cli. With gcloud, we could build up a deployment command like the following:

gcloud run deploy <service name> --image <docker image path> --region <deployed gcp region>

This command would take the specified Artifact Registry image, and deploy it to Cloud Run as the specified service name in the specified region.

There are plenty of ways we could automate this command for deployment. For our purposes, we will use Gradle so our build and deployment commands are consolidated into a single language/tool pairing.

What should the final buildscript configuration look like?

Our end goal is to be able to configure our Cloud Run deployment from our build.gradle.kts file in a similar way to how we configure our Artifact Registry deployment. The final result should look something like this:

cloudRunDeploy {
    deployedName.set(deployedImageName)
    deployedRegion.set(deploymentRegion)
    image.set(artifactRegistryConfig.toImage)
    imageTag.set(deployedImageTag)
    env.set(deploymentEnv)
}

Creating the Gradle plugin

To achieve this, we’ll start by creating a new Gradle plugin project named :cloudrun-deploy-plugin and adding it as an included build to our main Gradle project. Our :service module will eventually apply the plugin provided by :cloudun-deploy-plugin to enable our Cloud Run deployment.

Before that, we need to update our root settings.gradle.kts in add this new Gradle project as an included build:

// settings/gradle.kts

includeBuild("cloudrun-deploy-plugin")

Creating a configuration extension interface

With the plugin project created, we can create an interface defining properties for configuring the Cloud Run deployment.

// :cloudrun-deploy-plugin/build.gradle.kts

interface CloudRunDeployPluginExtension {
    val deployedName: Property<String>
    val deployedRegion: Property<String>
    val image: Property<String>
    val imageTag: Property<String>
    val env: Property<String>
}

Notice that these properties match the properties we configured in our :service/build.gradle.kts file.

Now that we have this interface, we can stub out how we will configure Cloud Run deployment.

We will add an extension method on the Gradle Project type that takes an instance of CloudRunDeployPluginExtension as a receiver. This will provide us the configuration syntax we’re looking for and allow us to retrieve these configuration values when needed during task execution.

// :cloudrun-deploy-plugin/build.gradle.kts

fun Project.cloudRunDeploy(configure: Action<CloudRunDeployPluginExtension>) {
    (this as org.gradle.api.plugins.ExtensionAware).extensions.configure(EXTENSION_NAME, configure)
}

Defining a deployment Gradle task

Now, we just need two more things to pull this all together. We need a custom Gradle task to do the deployment. And, we need to define the Gradle plugin that will apply that task to our Ktor project.

We’ll start with the task.

We’ll create a new class CloudRunDeployTask that extends Gradle’s DefaultTask type.

// :cloudrun-deploy-plugin/build.gradle.kts

private abstract class CloudRunDeployTask : DefaultTask() {

    @get:Inject
    abstract val exec: ExecOperations
    @get:Input
    abstract val deployedName: Property<String>
    @get:Input
    abstract val image: Property<String>
    @get:Input
    abstract val imageTag: Property<String>
    @get:Input
    abstract val deployedRegion: Property<String>
    @get:Input
    abstract val env: Property<String>

    init {
        group = TASK_GROUP
    }

    @TaskAction
    fun deploy() {
        val deployCommand = "gcloud run deploy ${deployedName.get()} --image ${image.get()}:${imageTag.get()} --region ${deployedRegion.get()} --set-env-vars KTOR_ENV=${env.get()}"
        logger.warn("Executing: $deployCommand")
        exec.exec { commandLine(deployCommand.split(" ")) }
    }
}

There are a few important items to note in the implementation of this task class

1. We define a property of type ExecOperations so we can execute terminal commands such as gcloud
2. We define properties of type Property<String> so we can, at runtime, access the configuration values needed to deploy our service
3. We define a method deploy() and annotate it with @TaskAction to define what should run when this task is executed. Within this implementation we
4. Build up the desired gcloud deployment command
5. Use the ExecOperations property to run the gcloud command

Defining a custom Plugin class

With this task type created, we can now define our custom plugin.

1. We’ll create a new class CloudRunDeployPlugin that implements Plugin<Project>
2. We must override the apply() method to register our custom deployment task
3. We’ll create an instance of our CloudRunDeployPluginExtension interface so it can be accessed and configured from the buildscript
4. We’ll register our CloudRunDeployTask type with a name of deployToCloudRun
5. Within the register configuration, we can then set the task properties using the values set on the created CloudRunDeployPluginExtension

// :cloudrun-deploy-plugin/build.gradle.kts

class CloudRunDeployPlugin : Plugin<Project> {
    override fun apply(project: Project) {
        val extensionConfig = project.extensions.create(EXTENSION_NAME, CloudRunDeployPluginExtension::class.java)
        project.tasks.register<CloudRunDeployTask>(TASK_NAME) {
            deployedName.set(extensionConfig.deployedName)
            image.set(extensionConfig.image)
            imageTag.set(extensionConfig.imageTag)
            deployedRegion.set(extensionConfig.deployedRegion)
            env.set(extensionConfig.env)
        }
    }
}

Applying the custom Gradle plugin

With a plugin class created, we need to define the plugin metadata so our main :service project can locate our Plugin. We can do this by leveraging the java-gradle-plugin from the :cloudrun-deploy-plugin/build.gradle.kts buildscript:

// :cloudrun-deploy-plugin/build.gradle.kts

version = "1.0.0"
gradlePlugin {
    plugins.create("cloud-run-deploy") {
        id = "cloud-run-deploy"
        implementationClass = "dev.goobar.cloudrun.deploy.CloudRunDeployPlugin"
    }
}

All we need to do now before we configure the plugin is to apply it to our project. The exact method of applying it to your project will depend on how you’ve structured your project(s). In my case, I’ve defined the coordinates of my custom plugin in my version catalog file:

// gradle/libs.versions.toml

[versions]
cloudRunDeployPluginVersion = "1.0.0"

[plugins]
cloudRunDeploy = { id = "cloud-run-deploy", version.ref = "cloudRunDeployPluginVersion" }

Now, I can apply the plugin to my Ktor service buildscript as follows:

// :service/build.gradle.kts

plugins {
    application
    ...
    alias(libs.plugins.cloudRunDeploy)
}

Configuring the Cloud Run Deployment Plugin

We now have a custom Gradle task, a custom Gradle plugin to register the task, and a registered extension interface.

With that, we can configure our deployment from our module-level build.gradle.kts script:

// :service/build.gradle.kts

cloudRunDeploy {
    deployedName.set(deployedImageName)
    deployedRegion.set(deploymentRegion)
    image.set(artifactRegistryConfig.toImage)
    imageTag.set(deployedImageTag)
    env.set(deploymentEnv)
}

Deploying the Cloud Run Service

With our cloudrun-deploy plugin applied to our project and configured for deployment, deploying the service to Cloud Run is simple. We simply run the deployToCloudRun gradle task we registered from our plugin.

This task relies on gcloud under the hood. Whether running from a local development machine or a CI machine, we need to be authenticated with gcloud.

Conclusion

In this post, we’ve explored how to streamline a Cloud Run deployment for a Ktor service by using a custom Gradle plugin. Building on the concepts from previous posts, we were able to define a Gradle task that takes in the necessary configuration data to deploy a container from Artifact Registry to Cloud Run.

By consolidating this logic in Gradle tasks, we reduce the complexity of building and deploying our Kotlin service. This also allows us to leverage the Ktor plugin’s support for building and publishing Docker containers for our service so we get Gradle task avoidance for free; further simplifying our build and deployment pipeline.

Want to Learn More?

If you want to learn more about Kotlin language basics, check out my playlist on YouTube

And if you’re ready to learn more about Kotlin for server-side development, take a look at my LinkedIn Learning course Transitioning from Java to Kotlin for the Web.

Deploy a Ktor Service Docker Image from Artifact Registry to Cloud Run Using Gradle was originally published in goobar on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this post, we’ll walk through the configuration of the Ktor plugin to deploy a Ktor Docker image to Artifact Registry using Gradle. We’ll also explore several other Gradle tasks that make working with a containerized version of our Ktor service easier.

Ktor is a framework from JetBrains enabling developers to build client and/or server applications using Kotlin. For organizations using Kotlin for server-side development, Ktor is an appealing framework as it’s built from the ground up to leverage Kotlin language features such as coroutines.

In many organizations today, especially those undergoing “cloud transformation”, containers are a popular choice for packaging, sharing, and deploying applications. Those containers can be used with tools like Docker to simplify the development, testing, and coordination of services.

Containers are a key aspect of many serverless deployment workflows within Google Cloud Platform. Serverless solutions like Cloud Run rely on containers to respond quickly to increases in server load by deploying new containers, when needed, to handle the incoming traffic, and then spin down those containers when traffic subsides.

If using Ktor for serverless applications, you’ll likely need to create a container image for your application. Once you create an image, you’ll need to deploy it to an external registry so it can eventually be deployed to a container and start receiving traffic.

It’s possible to do this in an entirely manual fashion; defining our own Dockerfile, building the image locally, and then pushing it to some external registry such as Google’s Artifact Registry.

However, the Ktor plugin provides mechanisms for simplifying this process in a way that makes it easy for those without extensive Docker, container, or GCP knowledge to leverage.

If you need to publish your Ktor Docker image to Container Registry, check out this related post.

Prerequisites

Since we’ll be deploying our Ktor service container to Artifact Registry, that comes with a new Google Cloud Platform (GCP) prerequisites.

We need to have access to a GCP organization

1. How to setup a GCP organization?
2. You’ll eventually need to set up a billing account for your organization. To try out GCP for learning purposes, there are often ways to find free, or discounted, billing credits
   - Trial period with $300 free credits for new users
   - GCP credits for faculty and students
   - Google Cloud Innovators Plugs program

We need to have a project set up within that organization

• How to create and manage GCP projects?

Artifact Registry must be enabled within that project

• How to enable Artifact Registry for a GCP project?

How to configure the Ktor plugin to build a Docker image?

To configure a Docker image, and publishing tasks, for our Ktor project, we’ll use the Ktor plugin dsl in our project’s Gradle buildscript. This will leverage the jib Gradle plugin under the hood to configure and build images for your project.

// build.gradle.kts

// build.gradle.kts
ktor {
   docker {
      // container configuration goes here
   }
}

To start, we’ll specify a Java Runtime Environment (JRE) version to use when constructing the container image.

ktor {
   docker {
       // set a JRE version that will be used to build the container
       jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)
   }
}

Next, we’ll configure port mappings for the image.

ktor {
   docker {
       // set a JRE version that will be used to build the container
       jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)

       // setup port mapping for the container/service
       portMappings.set(listOf(
           io.ktor.plugin.features.DockerPortMapping(
               outsideDocker = 8080,
               insideDocker = 8080,
               io.ktor.plugin.features.DockerPortMappingProtocol.TCP
           )
       ))
   }
}

And finally, we’ll configure the external registry used when publishing the container. In our example, we’ll configure it to use Artifact Registry.

To configure the external registry, we need to create an instance of DockerImageRegistry. Ktor provides implementations of this interface for Docker Hub and Google Container Registry, but not Artifact Registry.

However, the interface is public, so we can create our own implementation.
The most important thing is to implement the toImage property to return a string representing the tag needed to publish our image to Artifact Registry. That tag structure takes the following form: <region>-docker.pkg.dev/<project>/<repository>/<image name>.

class ArtifactRegistry(
   projectName: Provider<String>,
   regionName: Provider<String>,
   repositoryName: Provider<String>,
   imageName: Provider<String>,
) : DockerImageRegistry {
   override val username: Provider<String> = provider { "" }
   override val password: Provider<String> = provider { "" }
   override val toImage: Provider<String> = provider {
       val region = regionName.get()
       val project = projectName.get()
       val repository = repositoryName.get()
       val image = imageName.get()

       "$region-docker.pkg.dev/$project/$repository/$image"
   }
}

There are several key pieces of information needed for this Artifact Registry configuration:

1. projectName should be the name of your existing GCP project in which Artifact Registry has been enabled and where you want to upload the built container
2. regionName should be the region identifier for the GCP region in which your desired artifact repository is set up in GCP. ex: us-west1
3. repositoryName should be the name of the specific artifact repository where the images should be published. ex: my-org-docker-containers
4. imageName should be the name for a specific set of images within the repository.
5. A reasonable default here could be to use the name of your Ktor project. You can pull this from the Gradle project using project.name

Once we have our custom implementation of DockerImageRegistry we can finish our Docker configuration.

ktor {
   docker {
       // set a JRE version that will be used to build the container
       jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)

       // setup port mapping for the container/service
       portMappings.set(listOf(
           io.ktor.plugin.features.DockerPortMapping(
               outsideDocker = 8080,
               insideDocker = 8080,
               io.ktor.plugin.features.DockerPortMappingProtocol.TCP
           )
       ))

       // configure how the container should be deployed to Artifact Registry
       externalRegistry.set(
           ArtifactRegistry(
               projectName = provider { "<your gcp project name>" },
               regionName = provider { "<repository’s region name>" },
               repositoryName = provider { "<target repository name>" },
               imageName = provider { "<desired image name>" },
           )
        )
   }
}

Once this configuration is applied to the project, we can build our Docker image using the buildImage Gradle task. Check out this FAQ from the jib Gradle plugin repo for more details on what an equivalent Dockerfile would look like for the generated image.

How to deploy a Ktor Docker image to Artifact Registry?

With the Ktor Docker configuration applied to the project, we can publish a Docker image for our project using the publishImage Gradle task.

This requires us to authenticate against the specified GCP project and Artifact Registry repository.

To do this, we can run the following command: gcloud auth configure-docker <region name>-docker.pkg.dev to set up gcloud as a Docker credential helper.

Then, we just need to be sure we’ve authenticated with gcloud using gcloud auth login.

When an image with tag who’s start matches <region name>-docker.pkg.dev is pushed, it will look for this credential helper and use local gcloud authorization for the request.

Once we are authenticated, running the publishImage task should successfully publish the built image to Artifact Registry.

How to deploy a Ktor Docker image to local Docker service?

Using the Ktor Docker configuration, we can also streamline publishing of our project’s Docker image to a local Docker.

1. Start Docker on your development machine
2. Run the publishImageToLocalRegistry Gradle task

How to run the Ktor service container using generated Gradle tasks?

Using the Ktor Docker configuration, we can build an image and start running it with the local Docker daemon by running the runDocker Gradle task.

Conclusion

The Ktor plugin’s Docker configuration provides a streamlined workflow for developers working with Ktor services via Docker images.

By configuring the plugin to publish images to Google Cloud’s Artifact Registry, developers can publish Docker images for their service to Artifact Registry using a single Gradle task. This task can be used by all developers, and by CI/CD workflows to provide a consistent deployment workflow.

How to Publish a Ktor Docker Image to Artifact Registry Using the Ktor Plugin was originally published in goobar on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this post, we’ll walk through the configuration of the Ktor plugin for deploying a Ktor Docker image to Container Registry via a generated Gradle task. We’ll also explore several other Gradle tasks that make working with a containerized version of our Ktor service easier.

Ktor is a framework from JetBrains enabling developers to build client and/or server applications using Kotlin. For organizations wanting to use Kotlin for server-side development, Ktor is an appealing choice of framework as it’s built from the ground up to leverage Kotlin language features such as coroutines.

In many organizations today, especially those undergoing “cloud transformation” through adoption of serverless technologies, containers are a popular choice for packaging, sharing, and deploying applications. Those containers can be used with tools like Docker to simplify the development, testing, and coordination of services.

Containers are a key aspect of many serverless deployment workflows within Google Cloud Platform. Servless solutions like Cloud Run rely on containers to respond quickly to increases in server load by deploying new containers, when needed, to handle the incoming traffic, and then spin down those containers when traffic subsides.

If using Ktor for modern serverless development, it’s likely you’ll need to create a container image for your Ktor application and be able to deploy that image to some external repository where it may then be deployed and start receiving traffic.

It’s possible to do this in an entirely manual fashion; defining our own Dockerfile, building the container locally, and then pushing it to some external registry such as Google’s Container Registry.

However, the Ktor plugin provides mechanisms for simplifying this process in a way that makes it easy for those without extensive Docker, container, or GCP knowledge to leverage.

If you need to publish your Ktor Docker image to Artifact Registry, check out this related post.

Prerequisites

Since we’ll be deploying our Ktor service container to Container Registry, that comes with a new Google Cloud Platform (GCP) prerequisites.

We need to have access to a GCP organization:
- How to setup a GCP organization?
- You’ll eventually need to set up a billing account for your organization. To try out GCP for learning purposes, there are often ways to find free, or discounted, billing credits

• Trial period with $300 free credits for new users
• GCP credits for faculty and students
• Google Cloud Innovators Plugs program

We need to have a project set up within that organization:
- How to create and manage GCP projects?

Container Registry must be enabled within that project:
- How to enable Container Registry for a GCP project?

How to configure the Ktor plugin to build a Docker image?

To configure a Docker image, and publishing tasks, for our Ktor project, we’ll use the Ktor plugin dsl in our project’s Gradle buildscript. This will leverage the jib Gradle plugin under the hood to configure and build images for your project.

// build.gradle.kts

// build.gradle.kts
ktor {
   docker {
      // container configuration goes here
   }
}

To start, we’ll specify a Java Runtime Environment (JRE) version to use when constructing the container image.

ktor {
   docker {
       // set a JRE version that will be used to build the container
       jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)
   }
}

Next, we’ll configure port mappings for the image.

ktor {
   docker {
       // set a JRE version that will be used to build the container
       jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)

       // setup port mapping for the container/service
       portMappings.set(listOf(
           io.ktor.plugin.features.DockerPortMapping(
               outsideDocker = 8080,
               insideDocker = 8080,
               io.ktor.plugin.features.DockerPortMappingProtocol.TCP
           )
       ))
   }
}

And finally, we’ll configure the external registry used when publishing the container. In our example, we’ll configure it to use Container Registry.

ktor {
   docker {
       // set a JRE version that will be used to build the container
       jreVersion.set(io.ktor.plugin.features.JreVersion.JRE_17)

       // setup port mapping for the container/service
       portMappings.set(listOf(
           io.ktor.plugin.features.DockerPortMapping(
               outsideDocker = 8080,
               insideDocker = 8080,
               io.ktor.plugin.features.DockerPortMappingProtocol.TCP
           )
       ))

       // configure how the container should be deployed to Container Registry
       externalRegistry.set(
           DockerImageRegistry.googleContainerRegistry(
               projectName = provider { "<your gcp project name>" },
               appName = provider { "<desired repository name>" },
               username = provider { "" },
               password = provider { "" }
           )
       )
   }
}

There are several key pieces of information needed for this Container Registry configuration:

1. projectName should be the name of your existing GCP project in which Container Registry has been enabled and where you want to upload the built container
2. appName will be used to name the repository within Container Registry. If the repository doesn’t exist, it will be created when the container image is published.
   - A reasonable default here could be the name of your Ktor project. You could pull this from the Gradle project using project.name
3. username and password we leave blank here. In the next section we’ll discuss how authentication happens when deploying the container image

Once this configuration is applied to the project, we can build our Docker image using the buildImage Gradle task. Check out this FAQ from the jib Gradle plugin repo for more details on what an equivalent Dockerfile would look like for the generated image.

How to deploy a Ktor Docker image to Container Registry?

With the Ktor Docker configuration applied to the project, we can publish a Docker image for our project using the publishImage Gradle task.

This task requires some form of authentication against the GCP project we are attempting to access. Specifically, we need access to the following resource url based on our project configuration: gcr.io/<project name>/<app name>

As the Ktor Docker configuration uses the jib plugin under the hood, we can review documentation from jib on how to authenticate with different container registries.

For Container Registry, there straightforward options for authenticating the requests to publish our image:

1. Setup Google Cloud Application Default credentials on your machine using an account that has access to necessary GCP project
2. Setup gcloud as a credential helper for any Docker-supported registries

To setup auth via application default credentials, run the following terminal command before running the publishImage task: gcloud auth application-default login

To setup auth via gcloud Docker credential helper, run the following terminal commands before running the publishImage task:

• gcloud auth configure-docker gcr.io
• gcloud auth login

Once one of these (or another) authentication methods are applied, running the publishImage task should successfully publish the built image to Container Registry.

How to deploy a Ktor Docker image to local Docker service?

Using the Ktor Docker configuration, we can also streamline publishing of our project’s Docker image to a local Docker.

1. Start Docker on your development machine
2. Run the publishImageToLocalRegistry Gradle task

How to run the Ktor service container using generated Gradle tasks?

Using the Ktor Docker configuration, we can build an image and start running it with the local Docker daemon by running the runDocker Gradle task.

Conclusion

The Ktor plugin’s Docker configuration provides a streamlined workflow for developers working with Ktor services via Docker.

By configuring the plugin to publish images to Google Cloud’s Container Registry, developers can publish. This task can be used by all developers, and by CI/CD workflows to provide a consistent deployment workflow.

How to Publish a Ktor Docker Image to Container Registry Using the Ktor Plugin was originally published in goobar on Medium, where people are continuing the conversation by highlighting and responding to this story.

When evaluating a Kotlin Multiplatform solution, it was very important to our team that we understand how we could build a CI/CD pipeline for the project. It needed to be easy for our team to push changes and build a new version of the project.

Both our iOS and Android applications use GitHub Actions for our CI needs. So, in this post, we’ll walkthrough how to setup a GitHub Actions workflow for building a Kotlin Multiplatform Mobile project.

For this post, we are assuming that we already know how to build/integrate our shared code locally, and are focused on moving that process to CI.

For the tldr; you can jump to the end to see two sample workflows in their entirety. One for PR builds and one for Release builds.

One implementation note on the Swift Package distribution approach detailed in this post.

This approach is fairly straightforward as it simply publishes a built XCFramework, along with a Package.swift file to a GitHub repo. Xcode can then import this Swift package.

After using this approach for 1.5 years, we’ve identified a an issue with this approach. Xcode does not perform a shallow clone of the Swift Package repo, so when syncing new versions of our Swift Package, Xcode may need to fetch many versions of the built XCFramework which are stored in git history. After 1.5 years this became prohibitively slow.

Our team is currently migrating to a remote url distribution model for our Swift Package, and will write another post on that approach once completed. Additionally, the great folks over at Touchlab are working on some tooling to help make this process easier for more teams to setup from the beginning.

This is a part of an ongoing series on our usage of Kotlin Multiplatform at Premise:

• Part 1: Kotlin Multiplatform at Premise
• Part 2: Kotlin Multiplatform Project Structure for Integrating with Brownfield Applications
• Part 3: Building a CI Pipeline for Kotlin Multiplatform Mobile Using GitHub Actions
• Part 4: Publishing Kotlin Multiplatform Swift Packages to Google Cloud Storage
• Part 5: Generating BuildConfig Files for a Kotlin Multiplatform Library — Coming Soon
• Part 6: Optimizing Local Build Times for Kotlin Multiplatform Mobile Projects — Coming Soon

Creating a GitHub Actions Workflow for Kotlin Multiplatform Mobile

To start, we’ll create a new workflow file at .github/workflows/build-release.yml. In this file, we’ll defined our GitHub Actions workflow.

// .github/workflows/build-release.yml

name: Build Release

In our case, we only trigger this workflow when a developer manually kicks off a build from the Actions tab. This is enabled by adding the workflow_dispatchtrigger to our workflow file.

// .github/workflows/build-release.yml

name: Build Release

on:
  workflow_dispatch:

If you wanted to publish a new version on any change to your main branch, you could add a push trigger with a filter for the main branch.

// .github/workflows/build-release.yml

name: Build Release

on:
  workflow_dispatch:
  push:
    branches: 'main'

Setting Up Our Workflow Environment

Next up, we’re going to set some environment variables for our build.

...

env:
  XCODE_VERSION: '13.4.1'
  JAVA_VERSION: '14'
  ARTIFACTORY_URL: ${{ secrets.ARTIFACTORY_URL }}
  ARTIFACTORY_REPO: ${{ secrets.ARTIFACTORY_REPO }}
  ARTIFACTORY_USERNAME: ${{ secrets.ARTIFACTORY_USERNAME }}
  ARTIFACTORY_PASSWORD: ${{ secrets.ARTIFACTORY_PASSWORD }}
  ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}

Let’s walk through why we’re using each of these.

We define constants for the Java and Xcode versions we’ll use for the build simply to pull those to the top of thee file and make theme easier to find.

XCODE_VERSION: '13.4.1'
JAVA_VERSION: '14'

The Artifactory variables are setup so we can make those values available to our JVM/Android artifact publication tasks. The actual values are pulled from GitHub Secrets so they can be hidden from the repo and rotated when needed.

ARTIFACTORY_URL: ${{ secrets.ARTIFACTORY_URL }}
ARTIFACTORY_REPO: ${{ secrets.ARTIFACTORY_REPO }}
ARTIFACTORY_USERNAME: ${{ secrets.ARTIFACTORY_USERNAME }}
ARTIFACTORY_PASSWORD: ${{ secrets.ARTIFACTORY_PASSWORD }}

And finally, we setup an access token that has read/push access to an additional GitHub repo within our org that we will ultimately used to serve our Swift Package artifact to the iOS application.

ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}

Configuring a Job to Build a Kotlin Multiplatform Mobile Project

Now, we can setup the Job that will actually build our project when our workflow is run.

The most important thing to call out here is that we’ll need to use a Mac OS runner if we want to build any iOS target. This is because we’ll need the Xcode Build Tools for our iOS compile/linking Gradle tasks.

Here, we’ve explicitly chosen the macos-12 runner, but you might also consider using macos-lateset if it has the tooling needed for your build.

You can find more details about available GitHub Actions runners here.

...

jobs:
  build:
    name: Build & Test
    runs-on: macos-12
    steps:
      ...

Next, we can start adding individual build steps to our job.

Checking Out Kotlin Multiplatform and Swift Package Repos

The first build steps will be to checkout the two repos needed for the build:

• Repo 1 is our mobile-shared, Kotlin Multiplatform, project. This is the repo we will actually build and generate output artifacts from.
• Repo 2 is our mobile-shared-swift-package project. This repo only exists as a location for us to publish the XCFramework and Package.swift file generated from the Kotlin Multiplatform build process

To checkout each repo, we’ll use the standard actions/checkout GitHub Action.

First, we checkout the primary KMP repo. Because we’re going to checkout a second repo, we add explicit repository and path parameters here to clearly indicate which repo we are checking out, and where on disk it should be copied to. Knowing this location will be needed later when we start adding tags and committing changes to our Swift Package repository.

steps:
  - name: Checkout
    uses: actions/checkout@v3
    with:
      repository: premisedata/mobile-shared
      path: mobile-shared

Next, we check out the repo that stores our Swift Package artifact. Again, here we are explicit about the repository and path so we can differentiate between it, and the primary repo that was checked out, when we need to push our new Swift Package to the repo.

We also explicitly pass a token to the checkout action. This is the token that was setup with our other environment variables. This extra token is needed to authorize requests to this second repo, because a workflow only has access to the repo its defined within by default.

steps:
  - name: Checkout
    uses: actions/checkout@v3
    with:
      repository: premisedata/mobile-shared
      path: mobile-shared

  - name: Checkout
    uses: actions/checkout@v3
    with:
      repository: premisedata/mobile-tutorials-swift-package
      path: mobile-tutorials-swift-package
      token: ${{env.ACCESS_TOKEN}}

Setting up Java and Xcode Tooling

We want to be explicit in the versioning we use for our CI tooling so we get repeatable results. To help with this, we explicitly setup both the JVM and Xcode tooling versions on our runner.

steps:
  ...

  - uses: actions/setup-java@v1
    with:
      java-version: ${{ env.JAVA_VERSION }}

  - uses: maxim-lobanov/setup-xcode@v1
    with:
      xcode-version: ${{ env.XCODE_VERSION }}

Extracting Project Version for Git Tagging

Once our XCFramework and Package.Swift file are generated, we ultimately want to consume them as a Swift Package from Xcode. To do this, we publish both files to our second mobile-shared-swift-package repo.

To resolve a given version of this package, Xcode will match a specified version code to a tag in the repo.

This means that any project version that is released needs to have a matching tag in pushed to the Swift Package repo.

Our version is defined as a Gradle project property within the project’sgradle.properties file.

// gradle.properties

libraryVersion=1.0.0

To extract this version, and make it available to future workflow steps, we define a build step prints available Gradle properties, extracts the version using grep and cut, and stores it as a env variable in the workflow.

• Getting Gradle properties from the command line
• Setting output as env variables

steps:
  ...

  - name: Extract Version Info
    run: |
      cd mobile-shared
      export ARTIFACT_VERSION=$(./gradlew properties | grep ^version: | cut -c 10-)
      echo "ARTIFACT_VERSION=$ARTIFACT_VERSION" >> $GITHUB_ENV

In a later build step, we’ll access the saved env variable when tagging a new version of thee Swift Package.

git tag -a ${{ env.ARTIFACT_VERSION }}

Building the Artifacts

Now, it’s finally time to build the actual artifacts.

To do this, we’ll use gradle/gradle-build-action to run our Gradle tasks to build and test the project.

We explicitly define build-root-directory to point to our primary KMP repo when running the Gradle tasks. And the arguments parameters define the actual tasks we want to run; in this case build and createSwiftPackage

steps:
  ...

  - name: Build Artifacts
    uses: gradle/gradle-build-action@v2
    with:
      build-root-directory: mobile-shared
      arguments: build createSwiftPackage

• The build task will compile and test the code
• The createSwiftPackage task coordinates the generation of a Package.swift file and and zipping/copying of the XCFramework into a more findable location

Publishing the JVM Artifacts

Once we’ve built the project, we can publish our artifacts.

The JVM/Android artifacts are published to our internal Maven repository; currently managed with an internal Artifactory instance. Here, we use those Artifactory env variables we defined initially to configure the publication tasks so they are uploaded to the correct locations.

steps:
  ...

  - name: Publish JVM Artifacts
    uses: gradle/gradle-build-action@v2
    with:
      build-root-directory: mobile-shared
      arguments: artifactoryPublish artifactoryDeploy -PARTIFACTORY_URL=${{env.ARTIFACTORY_URL}} -PARTIFACTORY_REPO=${{env.ARTIFACTORY_REPO}} -PARTIFACTORY_USERNAME=${{env.ARTIFACTORY_USERNAME}} -PARTIFACTORY_PASSWORD=${{env.ARTIFACTORY_PASSWORD}}

Publishing the Swift Package

Publishing our Swift Package artifact is more involved than on the JVM side.

First, we run a build step that removes the existing XCFramework file from our mobile-shared-swift-package repo. This is done so there’s only one version of the framework in the repo for any given commit.

steps:
  ...
  - name: Prepare iOS Artifacts
    run: |
      rm -f mobile-shared-swift-package/mobileshared-*
      cp -r mobile-shared/shared/swiftpackage/* mobile-shared-swift-package

Next, we “publish” the new Swift Package to our repo by committing the updated XCFramework and Package.swift file. In addition to committing the changes, we must be sure to tag the commit with the project version so Xcode can resolve this new named version of our Kotlin Multiplatform code.

steps:
  ...
  - name: Prepare iOS Artifacts
    run: |
      rm -f mobile-shared-swift-package/mobileshared-*
      cp -r mobile-shared/shared/swiftpackage/* mobile-shared-swift-package

  - name: Publish iOS Artifacts
    run: |
      cd mobile-shared-swift-package
      git add --all
      git commit -m "Update to version ${{ env.ARTIFACT_VERSION }}"
      git tag -a ${{ env.ARTIFACT_VERSION }} -m "mobile-shared v${{ env.ARTIFACT_VERSION }}"
      git push
      git push origin ${{ env.ARTIFACT_VERSION }}

The last step here is to tag the Kotlin Multiplatform project itself so the source code has a tag matching the release tag. This step isn’t necessary for integration, but helps us match versions to the code that generated them.

steps:
  ...
  - name: Prepare iOS Artifacts
    run: |
      rm -f mobile-shared-swift-package/mobileshared-*
      cp -r mobile-shared/shared/swiftpackage/* mobile-shared-swift-package

  - name: Publish iOS Artifacts
    run: |
      cd mobile-shared-swift-package
      git add --all
      git commit -m "Update to version ${{ env.ARTIFACT_VERSION }}"
      git tag -a ${{ env.ARTIFACT_VERSION }} -m "mobile-shared v${{ env.ARTIFACT_VERSION }}"
      git push
      git push origin ${{ env.ARTIFACT_VERSION }}

  - name: Tag Release
    run: |
      cd mobile-shared
      git tag -a ${{ env.ARTIFACT_VERSION }} -m "mobile-shared v${{ env.ARTIFACT_VERSION }}"
      git push
      git push origin ${{ env.ARTIFACT_VERSION }}

Once these commits and tags are pushed to the mobile-shared-swift-package repo, Xcode should be able to resolve the new version of our library.

Reviewing the Final Release Workflow Solution

To help visualize everything together, here’s a complete look at our solution. Some bits of this have been modified for brevity, but all the core steps are here.

// .github/workflows/build-release.yml

name: Build Release

on:
  workflow_dispatch:
  push:
    branches: 'release'

env:
  XCODE_VERSION: '13.4.1'
  JAVA_VERSION: '14'
  ARTIFACTORY_URL: ${{ secrets.ARTIFACTORY_URL }}
  ARTIFACTORY_REPO: ${{ secrets.ARTIFACTORY_REPO }}
  ARTIFACTORY_USERNAME: ${{ secrets.ARTIFACTORY_USERNAME }}
  ARTIFACTORY_PASSWORD: ${{ secrets.ARTIFACTORY_PASSWORD }}
  ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}

jobs:
  build:
    name: Build & Test
    runs-on: macos-12
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          repository: premisedata/mobile-shared
          path: mobile-shared

      - name: Checkout
        uses: actions/checkout@v3
        with:
          repository: premisedata/mobile-shared-swift-package
          path: mobile-shared-swift-package
          token: ${{env.ACCESS_TOKEN}}

      - uses: actions/setup-java@v1
        with:
          java-version: ${{ env.JAVA_VERSION }}

      - uses: maxim-lobanov/setup-xcode@v1
        with:
          xcode-version: ${{ env.XCODE_VERSION }}

      - name: Extract Version Info
        run: |
          cd mobile-shared
          export ARTIFACT_VERSION=$(./gradlew properties | grep ^version: | cut -c 10-)
          echo "ARTIFACT_VERSION=$ARTIFACT_VERSION" >> $GITHUB_ENV

      - name: Build Artifacts
        uses: gradle/gradle-build-action@v1
        with:
          build-root-directory: mobile-shared
          arguments: build createSwiftPackage

      - name: Publish JVM Artifacts
        uses: gradle/gradle-build-action@v1
        with:
          build-root-directory: mobile-shared
          arguments: artifactoryPublish artifactoryDeploy -PARTIFACTORY_URL=${{env.ARTIFACTORY_URL}} -PARTIFACTORY_REPO=${{env.ARTIFACTORY_REPO}} -PARTIFACTORY_USERNAME=${{env.ARTIFACTORY_USERNAME}} -PARTIFACTORY_PASSWORD=${{env.ARTIFACTORY_PASSWORD}}

      - name: Prepare iOS Artifacts
        run: |
          rm -f mobile-shared-swift-package/mobileshared-*
          cp -r mobile-shared/shared/swiftpackage/* mobile-shared-swift-package

      - name: Publish iOS Artifacts
        run: |
          cd mobile-shared-swift-package
          git add --all
          git commit -m "Update to version ${{ env.ARTIFACT_VERSION }}"
          git tag -a ${{ env.ARTIFACT_VERSION }} -m "mobile-shared v${{ env.ARTIFACT_VERSION }}"
          git push
          git push origin ${{ env.ARTIFACT_VERSION }}

      - name: Tag Release
        run: |
          cd mobile-shared
          git tag -a ${{ env.ARTIFACT_VERSION }} -m "mobile-shared v${{ env.ARTIFACT_VERSION }}"
          git push
          git push origin ${{ env.ARTIFACT_VERSION }}

Reviewing a PR Workflow Solution

We’ve walked through a version of our release workflow that builds, tests, and publishes artifacts for our Kotlin Multiplatform project. Because it’s doing a full release, it has some extra build steps that are not required if you simply want to setup a CI workflow for your PRs.

To round out our discussion, here’s an example of a PR workflow for a Kotlin Multiplatform project.

// .github/workflows/pull-request.yml

name: Build Pull Request

on:
  pull_request:
    branches:
     - 'main'
     - 'dev'
     - 'release'

env:
  XCODE_VERSION: '13.4.1'
  JAVA_VERSION: '14'

jobs:
  build:
    name: Build & Test
    runs-on: macos-12
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - uses: actions/setup-java@v1
        with:
          java-version: ${{ env.JAVA_VERSION }}

      - uses: maxim-lobanov/setup-xcode@v1
        with:
          xcode-version: ${{ env.XCODE_VERSION }}

      - name: Build
        id: gradle
        uses: gradle/gradle-build-action@v2
        with:
          arguments: check

Final Thoughts

These workflows have worked well for us in the past 1.5 years. Our publishing model for our Swift Package is evolving, but the workflow itself has worked great.

As linking native binaries for the XCFrameworks is slow, and running CI jobs is not free, it’s a good idea to ensure you’re only building native targets for iOS platforms you actually need. For example, you might find that you no longer need an X64 version of your code, and could stop building that in CI to save money.

Premise is constantly looking to hire top-tier engineering talent to help us improve our front-end, mobile offerings, data and cloud infrastructure. Come find out why we’re consistently named among the top places to work in Silicon Valley by visiting our Careers page.

Building a CI Pipeline for Kotlin Multiplatform Mobile Using GitHub Actions was originally published in Engineering at Premise on Medium, where people are continuing the conversation by highlighting and responding to this story.

Most Kotlin Multiplatform Mobile sample projects follow the same project structure — an Android project directory, an iOS project directory, and a common module shared between them in a single mono repo.

This makes sense for a greenfield project.

I think this also makes a lot of sense for sample projects since it reduces project complexity and setup times; both of which are tremendously valuable in learning content for beginners.

However, this greenfield monorepo world is not the world that most teams are living in when evaluating whether or not Kotlin Multiplatform is a viable code sharing solution.

When our team began evaluating Kotlin Multiplatform, we knew we needed a solution that did not require us to merge our separate Android and iOS repos into a new monorepo structure. Instead, we took the approach of treating our shared code as an internal, first-party library that would be published to, and consumed from, some internal location.

we took the approach of treating our shared code as an internal, first-party library

In this paradigm, we end up with three separate repos- an Android repo, an iOS repo and our shared code in a new Kotlin Multiplatform repo.

In this post, we’ll detail our exploration of both a single-feature project structure, and a structure that has supported us as we adopt Kotlin Multiplatform for additional functional areas over time.

This is a part of an ongoing series on our usage of Kotlin Multiplatform at Premise:

• Part 1: Kotlin Multiplatform at Premise
• Part 2: Kotlin Multiplatform Project Structure for Integrating with Brownfield Applications
• Part 3: Building a CI Pipeline for Kotlin Multiplatform Mobile Using GitHub Actions
• Part 4: Publishing Kotlin Multiplatform Swift Packages to Google Cloud Storage
• Part 5: Generating BuildConfig Files for a Kotlin Multiplatform Library — Coming Soon
• Part 6: Optimizing Local Build Times for Kotlin Multiplatform Mobile Projects — Coming Soon

Single Feature Module Structure

How do you structure a Kotlin Multiplatform project with a single module?

The most straightforward project structure for this first-party library integration model is to have a single Gradle module within the Kotlin Multiplatform project. This module contains all shared code.

Internally, this module may be organized in several ways.

There could be a single `commonMain` source set that defines shared code for all targets.

There could be a `commonMain` plus several other source sets-one for each build target allowing for platform-specific code where needed.

And within any of these source sets, we are free to define whatever package structure we may like.

Build Output

With this structure, our build would generate a single output artifact for each target. That artifact would contain all the code exposed to a given platform.

If the project targets Android and iOS, then we would end up with

• a .jar or .aar
• an XCFramework

Publishing JVM/Android Artifacts

To generate Maven publications for our Android and JVM targets, we can add the maven-publish plugin.

// shared/build.gradle.kts

plugins {
    id("maven-publish")
}

kotlin {
  android()
}

And to ultimately publish those artifacts to an internal Artifactory server we configure something like the following:

// shared/build.gradle.kts

...

artifactory {
    setContextUrl(properties["ARTIFACTORY_URL"])
    publish(delegateClosureOf<org.jfrog.gradle.plugin.artifactory.dsl.PublisherConfig> {
        repository(delegateClosureOf<org.jfrog.gradle.plugin.artifactory.dsl.DoubleDelegateWrapper> {
            setProperty("repoKey", properties["ARTIFACTORY_REPO"])
            setProperty("username", properties["ARTIFACTORY_USERNAME"])
            setProperty("password", properties["ARTIFACTORY_PASSWORD"])
            setProperty("maven", true)
        })
        defaults(delegateClosureOf<groovy.lang.GroovyObject> {
            invokeMethod("publications", arrayOf("androidRelease", "kotlinMultiplatform"))
        })
    })
}

Building iOS Artifacts

For iOS, building the XCFramework is pretty straightforward in the single-module approach.

We can configure our iOS targets to generate an XCFramework in our Gradle buildscript.

// shared/build.gradle.kts
...

kotlin {
    val xcf = XCFramework()

    ios {
        binaries.framework {
            baseName = "shared"
            xcf.add(this)
        }
    }
}

Adding this XCFramework() configuration will result in an assembleXCFramework task being created that will generate the XCFramework for us.

Consuming this XCFramework can be done in several ways. Our team publishes the framework as a Swift Package. That process is a bit more involved, and is described in more detail in this post:

https://engineering.premise.com/kotlin-multiplatforrm-github-actions-workflow-3e5e0fcb7081

Benefits to a single-module approach

This approach is the easiest to setup as it requires the least amount of Gradle configuration. A single module also reduces any cognitive load in considering whether or not to add new code in a new module or not; everything just goes into the same place (maybe separated by package).

A single module also means a single output artifact per platform, which means fewer artifacts to version, publish, and consume.

Drawbacks to a single-module approach

Putting all code into a single module can lead to unintended dependencies between individual components as the lines between functional areas blur. A single module can also have a negative impact on build times as the project grows and any change requires the whole project to be rebuilt.

Organizing around a single module also means likely having a single library version for everything. This may, or may not, be a problem based on how you envision your project growing and your team working on it. If there’s a chance that separate functional areas may need to develop at a different pace, then a single module is not the ideal structure.

And finally, if all the shared code exists within, and is exported to, a single build output per target, there’s no way to consume only specific functional areas. For example, if one project needed both DTOs and Analytics and another only needed Analytics there would be no way to separate the use cases; both projects would receive all shared code at all times. As we’ll see in the next section, this is likely to be the case for iOS no matter what, but on the JVM/Android side we have more options.

Multi-Feature Module Structure

As we evaluated Kotlin Multiplatform, we initially started with only a shared analytics use case. However, we also envisioned an N+1 world in which we wanted to add more functional areas to our shared code.

Imagining that we might want multiple features such as analytics, dtos, deeplinking, etc lead us to explore a project structure in which each functional area could be a separate Gradle module within our Kotlin Multiplatform project.

This approach is straightforward for JVM and Android targets.

Each functional module may define it’s own publishing config in the same way as a single-module approach would. And ultimately those publishing configs will be used to generate a .jar or .aar for each module.

However, on the iOS side there is additional project configuration needed to support individual functional modules. A current limitation of the Kotlin Multiplatform tooling is that a single project can only produce a single output XCFramework.

This means we can’t generate one XCFramework per module as we can for JVM/Android targets.

Instead, we must define another Gradle module to act as a wrapper. This sharedmodule defines each functional module as a dependency and then configures the output XCFramework to include them all.

// shared/build.gradle.kts

plugins {
    kotlin("multiplatform")
}

kotlin {
    val xcf = XCFramework(rootProject.name)

    val nativeTargets = listOf(iosSimulatorArm64(), iosArm64())
    nativeTargets.forEach { target ->
        target.binaries.framework {
            baseName = rootProject.name
            xcf.add(this)

            export(project(":analytics"))
            export(project(":network"))
            export(project(":data"))
            transitiveExport = true
        }
    }

    sourceSets {
        val iosSimulatorArm64Main by getting
        val iosArm64Main by getting
        val iosMain by creating {
            iosSimulatorArm64Main.dependsOn(this)
            iosArm64Main.dependsOn(this)
        }
    }
}

The resulting project structure looks something like the following:

Why use separate modules?

Separating functional areas into their own benefits has several benefits in our estimation:

• separation of concerns
• improved build times when changing a single module
• separate output artifacts for JVM/Android

We felt these attributes would scale well if we continued to invest in KMP. In particular, it meant that we could integrate functional modules on a case-by-case basis. For example, our Android app could consume every shared feature, while our mobile-proxy backend service could consume only the Analytics module if desired.

Drawbacks to a multi-module approach?

The main drawback here is probably the added complexity.

We must configure an extra wrapper module for the iOS output. We must add additional JVM/Android publication configurations for each functional module. We must work harder to keep dependency versions synced across each module. There’s extra work involved in adding a new functional area to the project due to the cost of setting up a new module.

All of these are very solvable problems, but in our experience, has required members of the team that are comfortable with non-trivial Gradle configuration.

Multi-Project Structure

We’ve focused the discussion so far on how to structure a single project for one, or more, output artifacts. However, nothing is stopping us from just creating separate KMP projects altogether.

This would provide a means of publishing/consuming separate XCFrameworks for iOS projects.

For a team with a single iOS app, this probably adds extra complexity for zero gain. But if your team maintains multiple iOS projects, having shared functionality split into separate artifacts could be beneficial.

For JVM/Android applications, this approach would likely provide little benefit to consumers. However, if development of individual functional areas was owned by separate teams/individuals, then separate projects could increase development velocity if those projects are not interdependent.

Closing Thoughts

Choosing the right project structure for you applications will depend on the applications themselves, your team structures, what code you’re trying to share, etc.

However, if you’re integrating your Kotlin Multiplatform code into existing applications, the chances are high that you’ll want to structure your KMP project using one of the approaches detailed above.

We’ve maintained a single KMP project, multi-module structure for over a year and have found it scaling well as we integrate additional functional areas and onboarding more developers.

Diving deeper into how we are leveraging Kotlin Multiplatform

Once you’ve settled on a project structure, you’ll likely move on to new challenges such as how to build your Kotlin Multiplatform project in CI and how to integrate your XCFramework into your iOS project.

• Building A Kotlin Multiplatform Mobile CI Pipeline Using GitHub Actions
• Kotlin Multiplatform at Premise

Premise is constantly looking to hire top-tier engineering talent to help us improve our front-end, mobile offerings, data and cloud infrastructure. Come find out why we’re consistently named among the top places to work in Silicon Valley by visiting our Careers page.

Kotlin Multiplatform Project Structure for Integration with Brownfield Applications was originally published in Engineering at Premise on Medium, where people are continuing the conversation by highlighting and responding to this story.

In January of 2021, we pushed the initial commit for our mobile-shared repo — a new Kotlin Multiplatform (KMP) initiative within the mobile engineering team here at Premise.

Fast forward to 2022, and our usage of Kotlin Multiplatform has increased along with our confidence in the choice to invest in Kotlin Multiplatform as a code sharing solution.

Much has been written comparing Kotlin Multiplatform to other code sharing solutions — debating the merits of one choice or another and often framed within the context of greenfield development or toy sample projects. While these resources are helpful, they’re often lacking a pragmatic grounding in the realities of scaling a production brown-field project.

The discussion is often focused on the core technology rather than on why Kotlin Multiplatform was chosen over other solutions, and how Kotlin Multiplatform was actually then adopted, integrated, and scaled across multiple teams and projects.

This is a part of an ongoing series on our usage of Kotlin Multiplatform at Premise:

• Part 1: Kotlin Multiplatform at Premise
• Part 2: Kotlin Multiplatform Project Structure for Integrating with Brownfield Applications
• Part 3: Building a CI Pipeline for Kotlin Multiplatform Mobile Using GitHub Actions
• Part 4: Publishing Kotlin Multiplatform Swift Packages to Google Cloud Storage
• Part 5: Generating BuildConfig Files for a Kotlin Multiplatform Library — Coming Soon
• Part 6: Optimizing Local Build Times for Kotlin Multiplatform Mobile Projects — Coming Soon

I want to focus both on high-level thinking and decision making, and also highlight some of the key technical challenges we had to solve to begin building with Kotlin Multiplatform across both our iOS and Android teams.

We’ll start with how we identified the need for code sharing and why we choose KMP as our solution. And we’ll work our way through the validation, education, and scaling process we’ve gone through over this past year.

This post is the high-level story of our Kotlin Multiplatform journey — the strategy. Throughout this post, we will highlight key technical challenges, and link to other, more focused, posts from the series on the implementation of how we overcame those challenges.

Why code sharing? And why now?

Our code sharing story starts not with Kotlin Multiplatform, and not in comparing KMP to solutions like Flutter or React Native. In fact, it doesn’t start from a strictly technical perspective at all.

It all started because we wanted more accurate and actionable analytics across our mobile platform.

Tip #1 — Let the problem define the solution

Standardizing analytics across iOS and Android

In early 2021, analytics across our mobile team was a bit of a mess.

Both iOS and Android cared about analytics, and tracked events to help understand our system. However, it was a bit like the wild west. Naming was very haphazard. Event types weren’t standardized. And there was significant room for error when implementing any given set of events.

This inconsistency, and uncertainty, hampered our ability to make data-driven decisions.

We knew we needed to change this as we scaled so data-driven decisions could be made for higher-impact with less cost.

To solve this problem, we formed a small team of engineering leaders, to standardize our thinking around analytics for the mobile team. These discussions were quite interesting in their own right, but not central to this story. What was important was the output of those discussions.

Our analytics working group generated several things:

• clear definitions for unique types of analytics event
• specific naming conventions for each event type
• enumerated value sets for key event name components

This thinking was consolidated into a doc to act as a source of truth for developers looking to implement analytics in any new work. We had effectively developed “cross platform thinking” around analytics for our team.

As we reached the end of this process, our strike team had developed two new questions:

1. Could we codify these rules so they’re strongly enforced in code rather than via a doc?
2. Could that enforcement be shared across both iOS, Android, and maybe even other teams?

We had finally arrived at the question of code sharing.

Sharing code for shared thinking

Code sharing, particularly for mobile development, is a bit of a “holy grail.” Most would probably agree that some theoretical ideal solution would allow for 100% code reuse across all mobile platforms — while still offering peak performance and beautiful platform-native experiences.

Most that have walked this path would probably also agree that this fantasy solution rarely works out in practice; and certainly not without tradeoffs.

For our use case though, we didn’t need, or want, a 100% code sharing solution. We wanted something much more focused. Honestly, simply sharing event name constants would have been a viable v1 solution and was discussed as such.

So, our decision making backdrop at this point was

• We have a clear set of rules and definitions for analytics events.
• We want to share this logic across iOS, Android (and maybe beyond).
• We don’t need shared UI.
• We need to share code using a tech stack that at least a significant portion of our team is familiar with.

Tip #2 — When evaluating a code sharing solution, consider both the technical, and human factors of each option.

Choosing Kotlin Multiplatform

We chose Kotlin Multiplatform as our code sharing solution for serval reasons.

• KMP focuses on business logic — not UI.
• KMP is relatively easy to integrate into a brown-field project.
• We had a great deal of Kotlin knowledge on our Android team.
• We had interest in KMP on our Android team.
• We had equal interest in KMP from our iOS team.

Kotlin Multiplatform was an obvious choice for us. Both our iOS, and Android tech leads arrived at this thought on their own before even having the full discussion.

It fit our use case of business logic sharing.

It was considered relatively low risk. If, for whatever reason, the KMP code didn’t work, the Android team could easily move the Kotlin code into our repo, and the iOS could translate it into Swift and bring it into their project.

At this point, we were comfortable enough to invest in a proof of concept validation.

Validating Kotlin Multiplatform for brown-field projects

Before we could realistically consider long-term investment into Kotlin Multiplatform, we needed to identify key challenges and risks.

We needed to understand how to solve the challenges and mitigate the risks before we felt comfortable adopting the proposed solution.

Some of these primary questions included:

• Where to store this new Kotlin Multiplatform project?
• What multiplatform targets to choose?
• How to structure a Kotlin Multiplatform project?
• How to provide a productive local development experience?
• How to ensure low-cost deployment via CI?
• How to integrate the KMP code into both the existing iOS and Android projects?
• How to onboard other developers?

Structuring our project

Many Kotlin Multiplatform examples focus on the idea of building an app from scratch. In this use case, it’s common to structure your KMP project as a monorepo with the Shared, Android, and iOS projects all living side-by-side.

However, we already had existing iOS and Android projects in their own repos. So this typical project structure wouldn’t work for us.

We realized we had to shift our thinking a bit.

Rather than thinking of it as building a Kotlin Multiplatform app, we found it more helpful to think of the project as building a separate Kotlin Multiplatform library to then be consumed by each of the existing projects mobile projects.

With that in mind, we created a new Kotlin Multiplatform project named mobile-shared with a single :analytics module to start.

This structure would allow us to consolidate all the shared code into this specific repo, which would then need to be built, published, and consumed by our iOS and Android projects.

Tip #3 — For brownfield projects, think of building a multiplatform library rather than a new multiplatform app

Choosing which platforms to support

We needed our shared code to work for both our iOS and Android apps. Our first thought was that we could simply choose iOS and Android targets.

However, that’s not where we landed.

We envisioned a world in which our multiplatform experiment went well and in which we might want to share our business event logic with other projects. Many of our services, including our primary mobile-proxy service, are JVM-based — being a mix of Scala, Java, and Kotlin. We thought it would be interesting to see if we could generate an artifact that could be consumed by those projects as well.

So, rather than using an Android target for our proof-of-concept, we chose a JVM target. This allows us to still consume the JVM artifact in our Android app, but gives us the option to potentially reuse the code in other services down the line.

We were able to make this choice, because our shared analytics strategy was platform agnostic. We weren’t planning to rely on any Android-specific apis, or SDKs for what we were planning to build.

Local development

Our mobile team very much cares about developer productivity, and the overall developer experience of our projects. Because of this, it was vital for us to understand how developers could efficiently build, test, and consume the KMP project locally.

We envisioned developers working on a new feature, opening up the mobile-shared project, building and testing their code locally, before then eventually pushing back to GitHub and creating a PR.

These seem like relatively straightforward tasks on the surface, but they raised a handful of interesting questions for us.

Tip #4 — Invest in the developer experience for any code sharing solution

How do developers build the Kotlin Multiplatform locally?

Kotlin Multiplatform relies on the Gradle build tool, and by extension, the Java Virtual Machine (JVM). The Android target requires Android build tooling, and the iOS target requires XCode build tooling.

This means that whether a dev was on the iOS or Android team, they likely had some tooling to install and configure before the KMP project could be built.

For iOS devs, they had to:

• Install the Android SDK.
• Install a JDK.

For Android devs they had to:

• Install XCode build tools.

Once the tooling was installed and configured properly, we could run our KMP project’s build task using the gradle-wrapper to assemble and test the project.

This left us with two primary artifacts:

• A .jar file for the JVM target
• A XCFramework for the iOS target

Now that we had these output artifacts, the next step was to understand how to actually consume those from within the iOS and Android projects.

How do developers integrate the Kotlin Multiplatform artifacts locally?

The fastest way to validate that our XCFramework and .jar could be integrated was to simply move those files into the iOS and Android project directories, and point the projects at them.

Both XCode and Gradle make it pretty easy to consume a framework or jar as a dependency.

This let us validate that our sample shared code could, in fact, be consumed from both iOS and Android using a single build process.

The next step was a move towards scalability.

Manually copying files around is not a great everyday development experience.

So, for the Android team, we explored a local deployment flow in which we could build our .jar, publish to our local Maven repository, and consume in our Android project like we would any other external dependency.

We applied the Maven Publish Plugin to our :analytics module to gain access to the publish and publishToMavenLocal tasks.

• The publishtask generates metadata and associates it with a build artifact for publishing to some maven repository.
• The publishToMavenLocal task deploys the maven publication (artifact + metadata) to the local Maven repository.

Once the JVM output artifact was published to our local Maven repository, we could access that maven coordinate from the mavenLocal() repository and integrate the dependency into our project as if it were coming from mavenCentral() or an internal Maven repository.

This flow is really powerful for the team as it allows for efficient local dev/deploy/test feedback cycles without having to publish anything to our internal Maven repository.

CI / CD

Building and deploying as part of our CI/CD pipeline faced similar challenges to the local build/deploy workflow.

Building a Kotlin Multiplatform project using GitHub Actions

Our mobile teams use GitHub Actions for our CI builds. We use the GitHub-hosted runners, so we needed to make sure that whatever runner we chose included the necessary build dependencies on the VM.

This was actually a pretty easy decision though. Because we need the XCode build tools to build the iOS target of our KMP project, we need to use the MacOS runner.

This was generally fine. The list of pre-installed dependencies is quite long, and included everything we needed. The only real concern was the cost. The macOS runner is more expensive than the ubuntu-runner our Android team typically uses.

After running our first builds on that macOS runner though, it was determined that it was not prohibitively expensive since the builds were fast on such a small project.

This build step included building both the JVM and iOS artifacts when running the build Gradle task.

Monitoring build performance using Gradle Enterprise

From that early point in the project, we also integrated Gradle Enterprise so we could collect buildscans and monitor the build performance over time. This was done to keep the KMP builds as fast as possible and lessen the overall cost of using the macOS runner.

Deploying Kotlin Multiplatform artifacts using GitHub Actions

Deploying of the JVM artifact was quite simple. We wanted to deploy the .jar as a maven artifact within our internal Maven server. To do this, we used the Gradle Artifactory Plugin. This plugin works in conjunction with the Maven Publish Plugin to deploy any generated Maven publications to some external Maven server.

Using the Artifactory plugin, we configured:

• the url of our internal Artifactory server
• the publication credentials
• which publications to include (jvm, kotlinMultiplatform)

Once the plugin was applied to our :analytics module and configured, we could use the artifactoryPublishand artifactoryDeployGradle tasks to publish our .jar file to our internal Artifactory server. Once that was done, we could access the desired dependency from the internal server when building our Android project.

Deploying the iOS artifact was more challenging and much more interesting. Our goal, was to consume the iOS code as a Swift Package.

To support the deployment, some changes were made to the project outside of any CI flow:

• We setup an additional GitHub repo to store a Swift Package and associated metadata.
• We integrated the Multiplatform Swift Package plugin which provides a Gradle task that packages a XCFramework as a Swift Package

Once those changes were in place, the general CI flow for publishing the Swift Package was:

• build the JVM and iOS targets
• generate the Swift Package from the XCFramework
• publish the Swift Package, and metadata, to the external repo by pushing a commit

Once the Swift Package is committed to the mobile-shared-swift-package repo, the XCode build can be pointed to that repo, with proper credentials, and can consume that package during the iOS CI build.

Once we reached this point, we had successfully validated all our questions and concerns. We had a working, end-to-end solution that made us feel comfortable continuing on with our investment into Kotlin Multiplatform. The next step was to present the findings to our team, and to ramp up in developer education.

Developer education

Once we had our working proof-of-concept, we wanted to share the findings with the team, and discuss our proposal for moving into the next phase of the project.

For this, we did several things:

• We prepared documentation on how to setup and build the project.
• Multiple calls were scheduled with the team to walk through the technical aspects of the proposed solution.
• A doc was prepared for review; outlining how our Analytics naming rules would be codified and enforced using Kotlin Multiplatform.
• We hosted several working sessions with developers to help setup the project, and get ramped up on how to consume the multiplatform code.
• Several working sessions were dedicated to rapid prototyping of our v1 release.

Through these efforts, we moved towards a “live beta” phase of the project where we then continued to invest and expand on our usage of Kotlin Multiplatform.

Testing Kotlin Multiplatform in production

Even in early 2021, Kotlin Multiplatform was quite stable for use cases not requiring multithreading in native build targets such as iOS.

For our use case, we were generally only defining sealed classes, enums, constants, and a few interfaces to wire everything together. So, we viewed our shipping of KMP in production as a “low risk” endeavor.

That said, we still took a very measured approach to shipping Kotlin Multiplatform code into our iOS and Android apps — just in case something did go wrong.

iOS shipped Kotlin Multiplatform first. They added a single event that shadowed an existing event from the previous analytics implementation. This event was only fired in one single place, making it a low-cost investment. After several weeks in production, no issues were found, and it was deemed safe to continue adding more events from the shared code.

On Android, the experience was much the same. A single, commonly sent, event was added using shared code, and after weeks of observation, the endeavor was deemed safe for production.

Revisiting our Kotlin Multiplatform investment one year later

The process outlined in this post took several months from first conception to shipping our first Kotlin Multiplatform in production.

After a year, we’re still pretty bullish on Kotlin Multiplatform — at least for non user-facing use cases.

Our investment into the shared analytics use case has been very much a success. The ability to strongly enforce even naming consistency across both iOS and Android has been a win on its own. Being able to share the event definitions when teams work on projects at different times is another big win. Through developing our shared apis, we also took the opportunity to improve our thinking around event properties as well. In particular, we marked all events with a schema property making event schemas easier to identify in Amplitude.

There have been challenges as well. In particular, the iOS developer experience is not as polished as it is for the JVM and Android dev experience.

The two main challenges our iOS team have faced have been:

1. Kotlin Multiplatform generates Objective-C code rather than Swift. So, some of the apis are difficult to work with from Swift when being consumed via Swift Package based on an XCFramework.
2. Syncing XCFrameworks (as Swift Packages) became slower and slower over time. This was due to committing the built XCFramework directly to our publication repo which causes Xcode’s deep clone of the repo to fetch a multi-gigabyte repo on fresh syncs.

The issue with consuming KMP code from Swift is a real one. Our iOS team has started to build some small wrapper layers around the KMP code to make it easier to work with in Swift. This largely revolves around mapping Objective-C types into the desired Swift types as outlined in the KMP Objective-C/Kotlin interop guide. There are additional tools and features coming online to help with this problem, so hopefully this part of the developer experience continues to evolve for the better.

The issue with slow Swift Package syncs is something we’re actively working on fixing at the moment. The solution in a nutshell is to move away from a binary distribution approach for the Swift Package to a remote url distribution approach. This allows us to store the XCFramework as a zip file outside of any git repository. In doin this, it means Xcode only ever has to sync a single version of the XCFramework so syncs stay fast over time.

What’s next?

Thanks to the success of the analytics use case, we’ve turned to Kotlin Multiplatform for new use cases including:

• Deeplinking
• Performance monitoring
• Recommendations
• Networking

There has been some experimentation into using Kotlin Multiplatform, in conjunction with Compose Multiplatform, to build some internal tools, but nothing significant has come from those efforts yet.

Will we ever embrace KMP to the level of shared presenters or even shared UI? I doubt it. But, we’re excited to see where the technology, and developer community are going.

I think we’ll continue to invest heavily into our analytics and deeplinking use cases, and think we’ll likely add new features into shared code as they come up.

For more on how we’ve been using Kotlin Multiplatform at Premise, check out Part 2 in our series:

Kotlin Multiplatform Project Structure for Integrating with Brownfield Applications

Premise is constantly looking to hire top-tier engineering talent to help us improve our front-end, mobile offerings, data and cloud infrastructure. Come find out why we’re consistently named among the top places to work in Silicon Valley by visiting our Careers page.

Kotlin Multiplatform at Premise was originally published in Engineering at Premise on Medium, where people are continuing the conversation by highlighting and responding to this story.

Our mobile team here at Premise uses Firebase for a number of things, the most heavily used feature being Firebase Remote Config. We use Remote Config to control feature flags. We also use it in our internationalization efforts as well as in conjunction with A/B Testing to gather insights into user behavior and application performance.

With this usage comes a fair bit of operational overhead.

As the number of Remote Config parameters and conditions increases, it becomes more difficult to keep track of individual values. This is exacerbated by increasing team sizes and more regular release cadences.

To help with this, we’ve been using Google Cloud Functions to monitor for changes in Remote Config values and to send notifications of those changes to Slack. This system has worked well, but over time, we’ve identified some potential improvements.

We recently built out a v2 of this Remote Config notification system, which included the following goals:

• Build the new Cloud Function in Kotlin so it’s easier for the Mobile team to maintain
• Notify on changes to Remote Config parameters
• Notify on changes to grouped Remote Config parameters
• Notify on changes to Remote Config conditions
• Clearly highlight diffs between parameter/condition versions

In this post, we’ll walk through the system design, project setup, code, and basic deployment that we’ve used to monitor Remote Config changes in our DEV, QA, and PROD environments. You should be able to easily adapt this sample code to your team’s needs or even deploy it to your Google Cloud project as is and start monitoring Remote Config changes immediately.

In the end, you should end up with something similar to this:

If you want to skip right to the code, you can check out the sample project on GitHub.

mobile-tutorials/002-remote-config-monitor-cloud-function at main · premisedata/mobile-tutorials

Designing the Remote Config Notification System

The end-to-end flow of our notification system has several steps:

• Changes to Firebase Remote Config trigger the remoteconfig.update event within GCP.
• That event triggers our monitoring Cloud Function.
• The Cloud Function parses the updated Remote Configversion from the event payload.
• The previous, and new, Remote Config templates (sets of values) are loaded using the Firebase Admin SDK.
• Those changes are compared to identify additions, removals, updates, etc.
• Changes are sent to Slack using the Slack SDK.

This workflow could be customized to suit your team’s needs in several ways:

• If you don’t care to see diffs in your Slack messages, then you could skip the use of the Firebase Admin SDK, and simply notify Slack of the updated version number any time the Cloud Function is run.
• The comparison of new and old templates could be simplified, or made more complex, depending on specific output needs.
• The notification could be sent to any other service, email, etc. It would be straightforward to send to multiple destinations as well.

Creating the Kotlin Cloud Function Project

We’re going to use IntelliJ to create a new Kotlin project. Cloud Functions supports the Java 11 runtime, and we can include the required dependencies using either Maven or Gradle. We’re using Maven as the setup is a bit simpler and most of the Google docs use Maven as well.

Once we’ve created our project template, we need to add a dependency on the Cloud Functions API so we can define our entry point class.

// pom.xml

<dependencies>
    <dependency>
        <groupId>com.google.cloud.functions</groupId>
        <artifactId>functions-framework-api</artifactId>
        <version>1.0.1</version>
    </dependency>
    ...
</dependencies>

Google Cloud Functions may be triggered in response to HTTP requests and/or some GCP event coming from a Pub/Sub topic, a Cloud Storage change, or a variety of other events such as changes to Firebase Remote Config.

In our case, we only need to respond to the remoteconfig.update event, so we will implement one of two available interfaces when defining our Cloud Function:

• RawBackgroundFunction
• BackgroundFunction<T>

Both interfaces provide an accept() method that we may override to defined the behavior of our Cloud Function. In the case of RawBackgroundFunction, the event payload will be returned as raw JSON, whereas the BackgroundFunction interface will return to us a typed object of our choosing.

We will use RawBackgroundFunction as it gives us more flexibility in how we log and parse the incoming event.

We’ll create a RemoteConfigUpdateEvent data class to hold the remoteconfig.update event payload.

import com.google.gson.annotations.SerializedName

data class RemoteConfigUpdateEvent(
    @SerializedName("updateType")
    val type: String,
    @SerializedName("updateOrigin")
    val origin: String,
    @SerializedName("versionNumber")
    val version: Long,
    @SerializedName("updateUser")
    val user: User
) {
    data class User(val email: String)
}

With this event, we can now define our Cloud Function.

class RemoteConfigMonitorService
    : RawBackgroundFunction> {
    override fun accept(
        json: String,
        context: Context
    ) {
        TODO("Not yet implemented")
    }
}

Now, we can add some basic logging to help identify changes in our Cloud Function logs. This also helps familiarize ourselves with the structure of the event payload before we start using it for more complex things.

class RemoteConfigMonitorService
    : RawBackgroundFunction {

    private val logger: Logger = Logger.getLogger(RemoteConfigMonitorService::class.java.name)

    private val gson: Gson = Gson()

    override fun accept(
        json: String,
        context: Context
    ) {
        val event: RemoteConfigUpdateEvent = 
            gson.fromJson(json, RemoteConfigUpdateEvent::class.java)        
        logger.info("Remote Config updated to version ${event.version} by ${event.user.email}")
    }
}

With our function in place and the update event data available to us, we can start working on notifying Slack of the changes.

Notifying Slack Of Remote Config Version Changes

For our first pass here, we’ll send a notification message to Slack that simply includes the new version number and the email address of the person who made the change.

For this, we’ll use the Slack SDK for Java.

// pom.xml

<dependencies>
    ...
    <dependency>
        <groupId>com.slack.api</groupId>
        <artifactId>slack-api-client</artifactId>
        <version>1.21.1</version>
    </dependency>
</dependencies>

To help work with the Slack SDK, we’ll create a small class SlackClient.

import com.slack.api.Slack
import com.slack.api.methods.MethodsClient
import com.slack.api.methods.response.chat.ChatPostMessageResponse

class SlackClient(token: String, private val channelId: String) {

    private val methods: MethodsClient = Slack.getInstance().methods(token)

    fun notify(message: String): ChatPostMessageResponse = methods.chatPostMessage { req ->
        req.channel(channelId).text(message).mrkdwn(true)
    }
}

This class will manage the instantiation of a Slack MethodsClient and the sending of our notification message to the desired Slack channel. The authentication and routing of these calls is controlled via the token and channelId values passed to the SlackClient.

The Slack token must be authorized to post messages to the provided channelId. How to do that is beyond the scope of this post. I recommend reviewing the “Building a Slack app” guide on how to get started building a Slack bot that can post messages on your service’s behalf.

We will access the token and channelId as runtime environment variables. We’ll examine how to pass those values as part of the deployment section of this post.

Once we’ve retrieved the necessary Slack credentials, we can create a new instance of our SlackClient and notify the channel when the Remote Config version has been changed.

class RemoteConfigMonitorService
    : RawBackgroundFunction {

    private val slackChannelId: String =
        System.getenv("SLACK_CHANNEL_ID")
    private val slackToken: String =
        System.getenv("SLACK_TOKEN")

    private val slack =
        SlackClient(token = slackToken, channelId = slackChannelId)
    private val logger: Logger =
       Logger.getLogger(RemoteConfigMonitorService::class.java.name)

    private val gson: Gson = Gson()

override fun accept(
        json: String,
        context: Context
    ) {
        val event: RemoteConfigUpdateEvent = 
            gson.fromJson(json, RemoteConfigUpdateEvent::class.java)
        val msg = "Remote Config updated to version ${event.version} by ${event.user.email}"
        logger.info(msg)
        slack.notify(msg)
    }
}

At this point, we should have a functional (if still limited) notification service ready to be deployed to GCP.

Deploying The Cloud Function From A Local Development Machine

To actually put our new Cloud Function into service, we need to deploy it to the Google Cloud project that our Firebase project is in. If you’re not sure which GCP project backs your Firebase instance, you can review your Firebase Project Settings.

To deploy our Cloud Function from our local development machine, we need to do several things:

• Install gcloud cli
• Authenticate gcloud with our desired GCP project
• Run the functions deploy command with any necessary arguments

The general deployment command format looks something like the following:

gcloud functions deploy <desired deployment name> \
--entry-point <class name> \
--runtime <target runtime> \
--trigger-event <target event>

We need to indicate the desired deployment name of our function, which class to load in response to an event, which event(s) should trigger the function, and which runtime environment to use for the deployment.

Beyond the basics, we can further customize the deployment by passing any necessary environment variables or any number of other properties.

In our case, the deployment command should look something like the following:

gcloud functions deploy remote-config-monitor-service \
--entry-point com.premise.RemoteConfigMonitorService \
--runtime java11 \
--trigger-event google.firebase.remoteconfig.update \
--set-env-vars GOOGLE_CLOUD_PROJECT=<target gcp proejct>,SLACK_TOKEN=<Slack Bot Token>,SLACK_CHANNEL_ID=<Id for target Slack channel>

The GOOGLE_CLOUD_PROJECT env variable will be used internally to ensure that the Cloud Function is authorized to interact with other GCP services such as Firebase. This variable is set automatically by some Cloud Function runtimes, but must be explicitly set when using the Java runtime.

Once this command has been run, it typically takes a couple of minutes to deploy to the target GCP project. Once deployed, you should be able to make a change in Remote Config and see a Slack notification within a few seconds.

Sending Remote Config Diffs With Slack Notification

At this point, our notification simply shows the new versionand the email of whoever made the change. It’s not particularly useful yet. So let’s dive deeper and take a look at how we can generate detailed diffs of each template change.

Loading Remote Config template versions

The remoteconfig.update event payload does not include the actual changes to the Remote Config template. Instead, the payload gives the version of the updated template. If we want to notify of the specific changes to our Remote Config parameters and conditions, we need to use the version to fetch that information.

We can use the Firebase Admin SDK to load this information from our Firebase project, and we’ll use coroutines to load both the old and new versions in parallel.

// pom.xml

<dependencies>
    <dependency>
        <groupId>com.google.firebase</groupId>
        <artifactId>firebase-admin</artifactId>
        <version>8.1.0</version>
    </dependency>
    <dependency>
        <groupId>org.jetbrains.kotlinx</groupId>
        <artifactId>kotlinx-coroutines-core</artifactId>
        <version>1.6.1</version>
        <type>pom</type>
    </dependency>
</dependencies>

With these dependencies added, we can now load the updated Remote Config version and version — 1 to get the complete set of parameters and conditions for each version.

private val projectId:String = System.getenv("GOOGLE_CLOUD_PROJECT")

init {        
    FirebaseApp.initializeApp()    
}

override fun accept(
    event: RemoteConfigUpdateEvent,
    context: Context
): Unit = runBlocking {
    val event: RemoteConfigUpdateEvent = 
            gson.fromJson(json, RemoteConfigUpdateEvent::class.java)

    val newConfig = async(Dispatchers.IO) {
        FirebaseRemoteConfig
            .getInstance()
            .getTemplateAtVersion(event.version)
    }
    val previousConfig = async(Dispatchers.IO) {
        FirebaseRemoteConfig
            .getInstance()
            .getTemplateAtVersion(event.version - 1)
    }

    val msg = "Remote Config updated to version ${event.version} by ${event.user.email}"
    logger.info(msg)
    slack.notify(msg)
}

Notice here that we are using runBlocking{} to provide a CoroutineScope and to block completion of accept() until all coroutines have completed. We then use the async{} coroutine builder to load both Remote Config template versions in parallel.

Diffing Remote Config template versions

Now that we have the full set of Remote Config parameters and conditions, we can diff the two versions, and use that diff to build a more helpful notification message.

We’ll start by creating several model classes to assist with the diff operations.

/**
 * Holds a diff between two Parameters or Conditions
 */
data class RemoteConfigDiff<T>(
    val key: String,
    val original: T,
    val updated: T
)

/**
 * A container for a set of changes to Parameters or Conditions
 */
data class RemoteConfigDiffContainer<T : Any>(
    val label: String,
    val removed: List<RemoteConfigDiff<T>>,
    val added: List<RemoteConfigDiff<T>>,
    val updated: List<RemoteConfigDiff<T>>,
) {
    val numberOfChanges: Int = removed.size + added.size + updated.size
}

/**
 * A container for the overall diff between two Remote Config template versions
 */
data class RemoteConfigDiffCollection(
    val projectId: String,
    val update: RemoteConfigUpdateEvent,
    val parameters: RemoteConfigDiffContainer<Parameter>,
    val conditions: RemoteConfigDiffContainer<Condition>
) {
    val numberOfChanges = parameters.numberOfChanges + conditions.numberOfChanges
}

Next, we’ll write a function to diff the updates into a resulting RemoteConfigDiffCollection, which will then be used to format our output notification for Slack.

/**
 * Constructs the diffs for Parameters and for Templates
 */
fun getRemoteConfigDiff(
    projectId: String,
    update: RemoteConfigUpdateEvent,
    original: Template,
    updated: Template
) = RemoteConfigDiffCollection(
    projectId = projectId,
    update = update,
    parameters = constructDiff(
        label = "Parameters",
        originalValues = original.getAllParameters(),
        updatedValues = updated.getAllParameters(),
        getDefault = { Parameter() }
    ),
    conditions = constructDiff(
        label = "Conditions",
        originalValues = original.conditions.associateBy { condition -> condition.name },
        updatedValues = updated.conditions.associateBy { condition -> condition.name },
        getDefault = { Condition(" ", " ") }
    )
)

private fun Template.getAllParameters() = parameters + buildMap {
    parameterGroups.entries.forEach { entry ->
        putAll(entry.value.parameters)
    }
}

/**
 * Uses Set math to identify which keys were updated, added, or removed
 * Builds List<RemoteConfigDiff> for each of these sets of keys
 */
private fun <T : Any> constructDiff(
    label: String,
    originalValues: Map<String, T>,
    updatedValues: Map<String, T>,
    getDefault: () -> T
): RemoteConfigDiffContainer<T> {

    val keysInBothConfigVersion = originalValues.keys intersect updatedValues.keys
    val removedKeys = originalValues.keys subtract keysInBothConfigVersion
    val addedKeys = updatedValues.keys subtract keysInBothConfigVersion

    val removedDiffs: List<RemoteConfigDiff<T>> = removedKeys.map { key ->
        RemoteConfigDiff(
            key = key,
            original = originalValues.getOrDefault(key, getDefault()),
            updated = getDefault()
        )
    }

    val addedDiffs: List<RemoteConfigDiff<T>> = addedKeys.map { key ->
        RemoteConfigDiff(
            key = key,
            original = getDefault(),
            updated = updatedValues.getOrDefault(key, getDefault())
        )
    }

    val updatedDiffs: List<RemoteConfigDiff<T>> = keysInBothConfigVersion.map { key ->
        val originalParameter = originalValues.getOrDefault(key, getDefault())
        val updatedParameter = updatedValues.getOrDefault(key, getDefault())
        Triple(key, originalParameter, updatedParameter)
    }.filter { it.second != it.third }
        .map {
            RemoteConfigDiff(
                key = it.first,
                original = it.second,
                updated = it.third
            )
        }

    return RemoteConfigDiffContainer(
        label = label,
        removed = removedDiffs,
        added = addedDiffs,
        updated = updatedDiffs
    )
}

Now that we understand how the Remote Config values have changed, we can format that data into something easier to glance at and understand in Slack.

Formatting Remote Config diffs

We will use the Java Diff Utils library to format the diffs between Remote Config template versions. This library is particularly helpful because it can annotate changes using simple markdown, which allows us to do things like strikethrough removed values and bold added ones. This helps a lot when viewing the output in Slack.

// pom.xml

<dependencies>
    <dependency>
        <groupId>io.github.java-diff-utils</groupId>
        <artifactId>java-diff-utils</artifactId>
        <version>4.11</version>
    </dependency>
</dependencies>

With the dependency added, we can create a RemoteConfigDiffFormatter class that will use a passed RemoteConfigDiffCollection to format an output string including:

• The total number of changes made
• A link to the Firebase Remote Config dashboard
• The email address of the person that made the change
• Removed, Added, and Updated sections for Conditions
• Removed, Added, and Updated sections for Parameters

private const val REMOVAL_SLACK_EMOJI = ":github-changes-requested:"
private const val ADDITION_SLACK_EMOJI = ":github-check-mark:"

/**
 * Generates a markdown-formatted string for a given [RemoteConfigDiffCollection]
 */
class RemoteConfigDiffFormatter(
    private val diff: RemoteConfigDiffCollection,
    private val firebaseProjectUrl: String,
) {
    private val generator = DiffRowGenerator.create()
        .showInlineDiffs(true)
        .inlineDiffByWord(true)
        .oldTag { f: Boolean? -> "~" }
        .newTag { f: Boolean? -> "*" }
        .build()

    fun toMarkdownString(): String {
        return diff.run {
            buildString {
                appendLine("$numberOfChanges changes made to <$firebaseProjectUrl|Remote Config> in $projectId")
                appendLine("Updated by: ${update.user.email}")

                markdownFromContainer(conditions)
                markdownFromContainer(parameters)
            }
        }
    }

    private fun Any.toJson(): String {
        return Gson().toJson(this)
    }

    private fun <T : Any> StringBuilder.markdownFromContainer(container: RemoteConfigDiffContainer<T>) {
        removedSection(container)
        addedSection(container)
        updatedSection(container)
    }

    private fun <T : Any> StringBuilder.updatedSection(container: RemoteConfigDiffContainer<T>) {
        if (container.updated.isNotEmpty()) {
            appendLine("")
            appendLine("")
            appendLine("$ADDITION_SLACK_EMOJI *Updated ${container.label}:*")
            appendLine("")

            container.updated.forEach { diff ->
                val rows = generator.generateDiffRows(
                    listOf(diff.original.toJson()),
                    listOf(diff.updated.toJson())
                )
                appendLine("  • _${diff.key}_")
                appendLine("      • ${rows.first().oldLine}")
                appendLine("      • ${rows.first().newLine}")
            }
        }
    }

    private fun <T : Any> StringBuilder.addedSection(container: RemoteConfigDiffContainer<T>) {
        if (container.added.isNotEmpty()) {
            appendLine("")
            appendLine("")
            appendLine("$ADDITION_SLACK_EMOJI *Added ${container.label}:*")
            appendLine("")

            container.added.forEach { diff ->
                val rows = generator.generateDiffRows(
                    listOf(""),
                    listOf(diff.updated.toJson())
                )
                appendLine("  • _${diff.key}_: ${rows.first().newLine}")
            }
        }
    }

    private fun <T : Any> StringBuilder.removedSection(container: RemoteConfigDiffContainer<T>) {
        if (container.removed.isNotEmpty()) {
            appendLine("")
            appendLine("")
            appendLine("$REMOVAL_SLACK_EMOJI *Removed ${container.label}:*")
            appendLine("")

            container.removed.forEach { diff ->
                val rows = generator.generateDiffRows(
                    listOf(diff.original.toJson()),
                    listOf("")
                )
                appendLine("  • _${diff.key}_: ${rows.first().oldLine}")
            }
        }
    }
}

With our formatter in place, we can put it all together to send our more detailed notifications to Slack.

class RemoteConfigMonitorService
    : RawBackgroundFunction {

    private val projectId: String = 
        System.getenv("GOOGLE_CLOUD_PROJECT")
    private val slackChannelId: String =
        System.getenv("SLACK_CHANNEL_ID")
    private val slackToken: String =
        System.getenv("SLACK_TOKEN")
    private val firebaseProjectUrl: String = 
   "https://console.firebase.google.com/project/${projectId}/config"


    private val slack =
        SlackClient(token = slackToken, channelId = slackChannelId)
    private val logger: Logger =
       Logger.getLogger(RemoteConfigMonitorService::class.java.name)

    private val gson: Gson = Gson()

    init {        
        FirebaseApp.initializeApp()    
    }

    override fun accept(
        json: String,
        context: Context
    ): Unit = runBlocking {
        val event: RemoteConfigUpdateEvent = 
            gson.fromJson(json, RemoteConfigUpdateEvent::class.java)
        
        val newConfig = async(Dispatchers.IO) {
            FirebaseRemoteConfig
                .getInstance()
                .getTemplateAtVersion(event.version)
        }
        val previousConfig = async(Dispatchers.IO) {
            FirebaseRemoteConfig
                .getInstance()
                .getTemplateAtVersion(event.version - 1)
        }

        val diffCollection = getRemoteConfigDiff(
            projectId, 
            event, 
            previousConfig.await(), 
            newConfig.await()
        )

        val response = slack.notify(
            message = RemoteConfigDiffFormatter(
                diff = diffCollection,
                firebaseProjectUrl = firebaseProjectUrl
            ).toMarkdownString())

        when (response.isOk) {
            true -> logger.info("Slack channel $slackChannelId notified")
            false -> logger.info("Failed to notify Slack channel $slackChannelId: ${response.error}")
        }
    }
}

The resulting output in Slack should look something like this:

What’s Next?

This v2 of our Firebase Remote Config notification system has been a nice improvement so far, but we’re not done. Now that we have an improved framework on which to build, we’re looking for more ways to improve our monitoring of Remote Config changes.

The following are some of the planned improvements:

• Explicitly warn when a parameter’s type changes
• Automated deployment via GitHub Actions
• Improved notification formatting — possibly using Slack’s Block Kit APIs

You can find the full sample code for this post on GitHub:

mobile-tutorials/002-remote-config-monitor-cloud-function at main · premisedata/mobile-tutorials

Premise is constantly looking to hire top-tier engineering talent to help us improve our front-end, mobile offerings, data and cloud infrastructure. Come find out why we’re consistently named among the top places to work in Silicon Valley by visiting our Careers page.

Monitoring Changes In Firebase Remote Config Using Kotlin, Slack, and Google Cloud Functions was originally published in Engineering at Premise on Medium, where people are continuing the conversation by highlighting and responding to this story.

by Nate Ebel, Android Developer

The buildSrc directory of a Gradle project is a special directory that is treated as an included build and provides a convenient location to store common or complex build logic used throughout your Gradle project. A common example of this might be to declare statically typed dependency coordinates in a single location for across build scripts in a multi-module Gradle project.

Within the buildSrc directory, we can write custom build logic using both Groovy and Kotlin. This has the potential to make build scripts more approachable for developers that are unfamiliar with Groovy but comfortable with Kotlin — as is the case for most of the Android developers here at Premise.

When we define custom build logic in Kotlin within the buildSrc directory, we can easily interact with that code from any Gradle build scripts written in Kotlin — we simply import and access as we would from any other Kotlin file.

Recently, however, we wanted to reuse some Kotlin extensions in our Groovy-based build scripts. In doing so, we found it wasn’t quite as straightforward as reusing in Kotlin-based build scripts and thought we’d share a couple of tips in case others have run into the same hangups.

Check out our mobile-tutorials repo for the full working sample detailed throughout this post.

mobile-tutorials/001-buildsrc-kotiln-extensions-in-groovy at main · premisedata/mobile-tutorials

Let’s walk through a small example to illustrate this use case.

We’ll be working out of a basic “hello world” Android app. You can find the code in our mobile-tutorials repo.

Within the root directory of that project, we’ll add our buildSrc directory.

• /app
• /buildSrc
• settings.gradle
• build.gradle
• etc…

Now, let’s add some custom build logic to our new buildSrc directory.

We’ll add an extension function on Gradle’s Project type.

// buildSrc/src/main/.../extensions/ProjectPropertyExtensions.kt

fun Project.getProjectVersionName(): String? {    
  val versionName = findProperty("VERSION_NAME")    
  return if (versionName is String) versionName else null
}

From a Kotlin build script file, we can leverage this extension quite simply.

// app/build.gradle.kts

import com.premise.mobile.tutorials.extensions.getProjectVersionName
...

android {
  defaultConfig {
    versionName = getProjectVersionName()
  }
}

However, from a Groovy file, if we try to use the extension the same way, we’ll get a build error during the script evaluation phase.

// app/build.gradle

android {
  defaultConfig {
    // Error: groovy.lang.MissingMethodException
    versionName getProjectVersionName()
  }
}

Notice though, that in the Kotlin build script example, we had explicitly imported the extension function. So, maybe we just need to import our function?

Still not there. If we try explicitly importing the extension first we get a different compilation error during the configuration phase of our build.

// app/build.gradle

// Error: unable to resolve class getProjectVersionName
import com.premise.mobile.tutorials.extensions.getProjectVersionName

android {
  defaultConfig {
    versionName getProjectVersionName()
  }
}

How then are we supposed to reference this getProjectVersionName() extension function from our Groovy build scripts? Surely we don’t have to migrate all our build scripts to Kotlin, right?

To answer that, let’s start by finding the lowest common denominator between our Groovy scripts and our Kotlin scripts.

Both Kotlin and Groovy ultimately compile down to JVM-compatible bytecode.

If we examine the /build directory of our :buildSrc module, we can locate the generated code for our getProjectVersionName() extension.

What we find is ultimately not surprising if you’re familiar with Java<>Kotlin interop. Under the hood, the Kotlin extension function generates a Java class named for the file in which it’s defined ProjectPropertyExtensionsKt.class .

So, we have a Kotlin extension, which generates a Java class, which we ultimately then want to consume from our Groovy build script.

If we return to our last example within our Groovy build script, we might now notice a key clue.

// app/build.gradle

// Error: unable to resolve class getProjectVersionName
import com.premise.mobile.tutorials.extensions.getProjectVersionName

android {
  defaultConfig {
    versionName getProjectVersionName()
  }
}

When attempting to import the extension function like we did from the Kotlin build script, we get an error indicating that the compiler can’t resolve the class name getProjectVersionName . In other words, it’s trying to import a class, rather than a specific function name. In Kotlin, this is handled for us, but when mixing Groovy and Kotlin here we need to explicitly import the name of the class generated by the compiler in which our extension function resides at the bytecode level.

With that in mind, we can now update the usage of our extension to call it as a static method on the generated class. The receiver type, Project, is then passed as the first argument to the invocation.

// app/build.gradle

// import the generated class
import com.premise.mobile.tutorials.extensions.ProjectPropertyExtensionsKt

android {
  defaultConfig {
    // call the static function - passing in the project
    versionName ProjectPropertyExtensionsKt.getProjectVersionName(project)
  }
}

Ultimately, we end up invoking the extension using the same syntax we would if we were calling it from Java.

The same thinking applies for extension properties as well.

Here, we’ve defined an extension property on Project to indicate whether or not we should be sending analytics events.

// buildSrc/src/main/.../extensions/ProjectPropertyExtensions.kt

val Project.shouldLogAnalytics: Boolean
    get() {
        val shouldLogAnalytics = findProperty("LOG_ANALYTICS")
        return if (shouldLogAnalytics is String) {
            shouldLogAnalytics.toBoolean() 
        } else {
            false
        }
    }

Now, let’s say we want to use this extension in our Android project to generate a custom BuildConfig field.

// app/build.gradle

buildTypes {
    debug {
        buildConfigField("boolean", "SHOULD_LOG_ANALYTICS", ProjectPropertyExtensionsKt.getShouldLogAnalytics(project).toString())
    }
}

Notice a few things about our usage of the extension property here:

• Again, we access the extension using the generated class name ProjectPropertyExtensionskt.
• Rather than using Kotlin’s property access syntax, we must use the extension property through a generated getter.
• That getter prepends “get” to the property name to generate a method named getShouldLogAnalytics().
• We must again pass the Project instance receive as the first parameter to the invocation of the method.

This is a bit more complicated than with the extension function, but still mirroring how we would access an extension property from Java.

Understanding how to consume our Kotlin build script extensions from both Groovy and Kotlin allows us to reuse those extensions from any of our build scripts, achieving a greater level of consistency throughout our build.

It also allows us to migrate key build logic into these helper extensions without having to completely migrate our build scripts to Kotlin, which has both developer and build time costs.

Check out our mobile-tutorials repo for the full working sample detailed throughout this post.

mobile-tutorials/001-buildsrc-kotiln-extensions-in-groovy at main · premisedata/mobile-tutorials

Premise is constantly looking to hire top-tier engineering talent to help us improve our front-end, mobile offerings, data and cloud infrastructure. Come find out why we’re consistently named among the top places to work in Silicon Valley by visiting our Careers page.

Using :buildSrc Kotlin Extensions From Groovy-based Gradle Build Scripts was originally published in Engineering at Premise on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Mobile Engineering team here at Premise has made a concerted effort to improve developer efficiency over the past year.

One of the biggest areas of tech investment has been into understanding and reducing the Gradle build times of our Android project. We worked with engineers from Gradle to help us optimize and monitor our build using tools such as Gradle Enterprise and gradle-profiler. The results from this work have been terrific. We are seeing decreases in both local and CI build times of 50–80% from when we first started our work.

Now that we have our build under control, the story has changed a bit. Instead of a laser-focus on speeding up our builds, we now look towards adopting new build optimizations and towards maintaining existing build performance.

How can we monitor build trends over time? How do we debug local build issues for a team spread across multiple continents? What can we do to help identify and fix regressions?

It’s these questions that we are more often considering.

In this post, I want to walk you through a bit of a case study into this build monitoring process. We’re going to explore how we used Gradle Enterprise, and other tools to:

• Discover a build regression
• Identify the root cause of that regression
• Fix the issue
• Improve monitoring so the issue is easier to deal with in the future

If you want to jump to the final summary analysis, you can skip to the end.

If you want to walk through the full case study, then hopefully as you read through this, you might gain some insights into how you might identify Gradle build issues in your own projects and to leverage tools such as Gradle buildscans to diagnosis and fix those issues.

Discovering A Build Regression

The issue we uncovered was this:

• Over the course of a few days, our PR builds in GitHub Actions saw a 3x increase in Gradle configuration time
• This resulted in a ~20% increase in total PR build times in GitHub Actions
• Local build times didn’t seem to be impacted

How Did We Detect The Increase In Gradle Configuration Time?

We detected this issue by reviewing build trend data in our Gradle Enterprise dashboard. If you aren’t familiar, Gradle Enterprise is a tool to help collect and analyze Gradle build data for your team or organization.

On reviewing our median PR build times over the past several weeks, we noticed a significant increase had occurred and persisted over several days. Without our Gradle Enterprise server collecting data for all Android builds, we likely would not have discovered this issue on our own.

Digging deeper into performance data for individual builds showed a significant increase in configuration time for one of our Gradle modules. This happened to be the one module in our app with native C++ code.

The was our first clue that maybe the issue was related to the NDK somehow.

Identifying The Root Cause Of The Regression?

With several days of increased build times for our :nativecode module, how did we then identify the root cause?

We started running through our list of Gradle debugging tools.

Trying to reproduce locally

First, we ran ./gradlew help locally to try and replicate the problem.

Running the help task will run just the configuration phase for a Gradle build as was the first place for us to try and reproduce the issue locally.

Unfortunately, this did not help as we were unable to replicate the build regression during a local build. Great for individual developer productivity, but not helpful in identifying our mystery issue.

Identifying the causal commit

Our next step, was to try and identify when the build regression started, and to identify any commits around that time that seemed likely to impact build performance.

Again, here we turned to Gradle Enterprise. We were able to view buildscan data over the past two weeks, and identify the approximate date where the build times started going up.

Once we had a timeframe in mind, we turned to our git history in GitHub.

On investigating the changes from that period, one change stuck out as the likely culprit — the updating of AGP to version 4.1 and Gradle to version 6.7

Benchmarking build performance around possible problem commit

To determine if the build regression was due to our bumping of AGP/Gradle versions, we turned to gradle-profiler to locally benchmark build performance before, and after, the version bump commit.

If you aren’t familiar, gradle-profiler automates the running of Gradle tasks and the collection of build metrics to provide aggregate performance data and comparisons. By default, gradle-profiler will run a target task 10 times and summarize that build data for you to compare against other builds.

In our benchmarking, we identified several commits around the possible problem commit, and ran our benchmarks against each of them.

Given that we couldn’t reproduce the issue locally to begin with, it’s not surprising that the results of our benchmarking were inconclusive. We weren’t seeing any statistically significant difference in build times across any of the commits we tested.

Then, we realized we may be using gradle-profiler incorrectly.

We were measuring total build time, but our issue was specifically at configuration time. Maybe if we isolated configuration time, our benchmark results would turn up something more interesting?

As it turns out, gradle-profiler can be configured to include configuration time in its output.

After re-running our benchmarks with configuration time reported, sadly, we still saw nothing conclusive.

At this time, we considered admitting defeat and calling in reinforcements from the developer community. We decided to sleep on it, and give it one more try later that week.

Reviewing buildscan output

After taking a break, we decided to take a step back from the benchmarking and deep dives into commit history. Instead, we opened up one of the impacted buildscans and gave it a thorough examination.

We noticed something important upon review of the console output for that build. The output logs mentioned the download and installation of the NDK.

This struck us as odd.

Why are we downloading the NDK if our pinned version matches what is pre-installed on our virtual machine?

Examining the pre-installed tooling of the GitHub Actions virtual environment

We needed to double check the pre-installed version(s) of the NDK available in the GitHub Actions virtual environments.

The list of current GitHub Actions virtual environments is available on GitHub. We found the listing for ubuntu-latest which, at the time, was ubuntu-18.*

If you open the README for a specific virtual environment, you can find the pre-installed tooling descriptions. And in the case of our investigation, we searched for NDK to determine which NDK versions were available to us.

We compared the downloaded NDK version to the version pinned in our application module.

Why/how could this be?

Reviewing AGP changes

We checked the AGP release notes — specifically for NDK changes. We found that, sure enough, our updated version of AGP had a new default NDK version — a version not pre-installed on the GitHub Actions virtual environment.

Forming Our Hypothesis

We now knew what was happening, but not why?

The Android Gradle Plugin was downloading and installing a version of the NDK during the configuration phase of our build. The install was required, because that new version of the NDK was not available on the GitHub Actions virtual machine.

But why was AGP downloading a version different than what we had pinned in our application module?

We formed the following hypothesis:

Our desired NDK version was not set for all Android modules in the project — therefore leaving AGP to download whatever versions it needed if it didn’t detect a pinned version.

How Could We Reproduce And Test The Issue Locally?

To validate this hypotheses, we tried again to reproduce the issue locally.

We check the installed NDK versions on a local machine and found that the new default NDK had, in fact, been downloaded to the machine.

This would explain why local configuration times didn’t seem to be impacted. The NDK had been previously installed by either a dev or by AGP on a previous build.

We then deleted that version of the NDK from the local test machine, and ran ./gradlew help to replicate the issue.

This time, we saw that the task kicked off a download of the new NDK version.

Successful repro!!!

We then tested a possible fix. Again remove the new NDK version from the local machine. Then, we pinned our desired NDK version in each project module. Once again we ran the help task, and this time, the project built without downloading or installing any other version of the NDK.

Fixing The Issue For Impacted CI Builds

To fix this issue for good, so our CI builds were not facing increased build times, we needed to ensure we were pinning the same NDK version for each of our modules.

To accomplish this, we created a precompiled script plugin named android-library.gradle file to consolidate any configuration that should be commonly applied to our Android library modules.

This included a single pining of the NDK which was then reused across modules to ensure consistency.

android {
    ndkVersion "<our desired NDK version>"
}

We then applied this android-library.gradle file across our build.gradle files.

Validating The Fix

Once we consolidated our buildscript logic, we pushed a branch with our changes and generated several Gradle buildscans for the new branch.

We examined the buildscans in Gradle Enterprise and verified that configuration time once again looked normal and output logs no longer included NDK installation

Summary Analysis

The Android Gradle Plugin (AGP) has a default version of the Native Development Kit (NDK). This is the version of the NDK that will be used if your project requires the NDK, but doesn’t specific a specific version of the NDK.

If you don’t specifically pin an NDK version for each module in your app, AGP will automatically download the default NDK if it doesn’t exist on your local machine.

The virtual environments provided by GitHub Actions have pre-installed versions of the NDK available. And for a while, the pre-installed versions matched AGP’s default NDK version for our builds.

However, after updating our Android Gradle Plugin version, the default NDK version was no longer present on the GitHub Actions virtual machine and had to be downloaded by AGP on each run.

Because our CI builds were not caching any/all installed NDK versions due to the large download size, our CI builds were having to download this new NDK on each build.

This download/installation time occurs during the configuration phase of the Gradle build for an Android project which is what was leading to the 5x increase in configuration time for our CI builds.

To avoid this, ensure that you’re using a consistent NDK version across all modules in your app so you don’t start downloading new versions when you update AGP in the future.

How Were We Able To Improve For The Future?

At this point, there was some minor celebration and we took pleasure in our newfound understanding of how Android Gradle Plugin works.

But very quickly, we started asking ourselves how we could identify and fix this issue more efficiently in the future? In trying to answer that question, we identified a number of ideas:

Ensure NDK pinned properly across modules.

This is the approach we implemented and helps us ensure that we’re only dealing with a single NDK version.

Cleanup installed NDK versions

By keeping multiple NDK versions around on local dev machines, we run the risk of not finding these types of issues during local development in the future. We took it as an opportunity to cleanup unused versions.

Pay closer attention to the default NDK version when updating AGP

Now that we know that Android Gradle Plugin will download and install missing versions of the NDK, we pay closer attention to the versions of the NDK that are installed on our GitHub Actions virtual environments.

If either the environment changes, or AGP’s default NDK version changes, we run the risk of facing the same configuration time regression.

Benchmarking before merging

This experience led to a greater understanding of the gradle-profiler tool. We now have this as a tool at our disposal when we think a change might impact build times.

Before merging in major changes to our build, we can benchmark build performance and better understand if our build will be significantly impacted in any way.

Automated configuration checks

Because the impact to configuration time is so dramatic when a new version of the NDK is downloaded, it’s pretty easy to spot the change if you are looking for it.

Wit this in mind, we created a workflow in GitHub Actions to help monitor and identify this issue in the future.

On a regular schedule, we run a job that simply runs Gradle’s help task.

This does several useful things for us

• Generates regular buildscans for the help task — making trends easier to analyze in Gradle Enterprise
• Generates CI builds that have very consistent runtime characteristics. Because this job always completes in ~1 min, it’s very obvious that something is wrong with the job takes ~3–4 min.

We’ve found that simply reviewing the job duration in GitHub has been enough to identify this regression — without even needing to review data in Gradle Enterprise.

Suggesting improvements to Gradle Enterprise

Gradle Enterprise already breaks build trends into several categories like “serial execution” and “cache avoidance”. We recognized that if Gradle Enterprise surfaced “configuration time” in this same way, it may make detecting this kinds of issues even easier.

We suggested the idea to Gradle engineers as a feature request for Gradle Enterprise.

Takeaways for your own projects

Hopefully you’ve found this to be an interesting and practical exploration of how to debug an Android Gradle build regression.

I just want to highlight a few key things for you that may be most helpful for your own builds:

• Gradle Enterprise is a terrific tool for monitoring, debugging, and optimizing Gradle builds. It collects buildscan data from across your team or organization and helps identifies issues and trends that may otherwise go unnoticed.
• Gradle buildscans are an invaluable tool for learning about a specific build, and for diagnosing issues in that build. We rely on them heavily for maintaining/improving our builds. If you don’t use Gradle Enterprise, you can still capture a buildscan by adding --scan to the end of your Gradle wrapper invocation.
• gradle-profiler can be a very helpful tool for comparing the build impact of any project changes. You can test simple scenarios like our help task. Or, you can simulate complex scenarios like incremental builds across a multi-module project.
• Be aware that Android Gradle Plugin will download missing NDK dependencies in the background if the needed version is not present on the building machine. The NDK is multiple Gb in size, so downloading this on every CI build is likely a cost you’d rather not pay each time if you can avoid it.
• It’s a very good idea to use the same NDK version across all project modules — and to be explicit in setting that version across modules. Consolidating common build logic into a precompiled script plugin is one very viable approach for ensuring Android Gradle Plugin is configured properly across all your modules.

Premise is constantly looking to hire top-tier engineering talent to help us improve our front-end, mobile offerings and cloud infrastructure. Come find out why we’re consistently named among the top places to work in Silicon Valley by visiting our Careers page.

Using Gradle Enterprise to Detect Configuration Regressions In An Android Gradle Build was originally published in Engineering at Premise on Medium, where people are continuing the conversation by highlighting and responding to this story.