Amazon S3 Encryption and Replicating Data to On-Prem MinIO

Introduction

This post walks through how we replicated data from an Amazon S3 bucket to a self-hosted, on-premises MinIO deployment. Before getting into the migration

mechanics, it’s worth understanding two things that directly affect how the migration behaves: how Amazon S3 encrypts data, and how MinIO relates to S3.

Background: Amazon S3 and Its Encryption Options

Amazon S3 (Simple Storage Service) is AWS’s object storage service. It stores data as objects and lets you store and retrieve any amount of data over the

network.

To protect data at rest in S3, you have a few encryption options. Understanding which one your bucket uses is critical.

SSE-S3 (AES256)

AES256 in S3 refers to SSE-S3: Server-Side Encryption with S3-managed keys.

• S3 encrypts data at rest: when S3 receives an object, it encrypts it. When someone retrieves the object, S3 decrypts it and returns it.
• S3 manages and stores the encryption keys itself, alongside the data.
• Because key management is fully handled by S3, no additional permissions are needed to decrypt. S3 read permission is enough to retrieve the object.

```bash
aws s3api get-bucket-encryption --bucket BUCKET_NAME

SSE-KMS (aws:kms)

When a bucket uses SSE-KMS, S3 relies on the AWS Key Management Service (KMS) to encrypt and decrypt data.

How encryption works:

1. When S3 receives data, it encrypts the object using a unique data key.
2. KMS uses a master key (the KMS key) to encrypt that data key.
3. The encrypted data key is stored alongside the encrypted object.

How decryption works:

1. When someone requests the object, an API call goes to AWS KMS.
2. KMS decrypts the unique data key using its master key.
3. The decrypted data key is then used to decrypt the object, making it accessible.

Because decryption involves an API call to KMS, the requesting user must have the kms:Decrypt permission. Without it, the request is rejected. So to read data

from a bucket encrypted with SSE-KMS, you need S3 read permission plus kms:Decrypt.

```bash
aws s3api get-bucket-encryption --bucket BUCKET_NAME

SSE-C (Customer-Provided Keys)

SSE-C: Server-Side Encryption with Customer-Provided Keys.

• The application supplies the encryption key with each request.
• S3 uses that key to encrypt and decrypt the object.
• S3 does not store the key.

Client-Side Encryption

With client-side encryption, your application or client encrypts the data before it reaches S3. S3 receives already-encrypted data and stores it in the bucket as-is.

Important note on deny policies: Regardless of which encryption method is used, make sure the object is not affected by an explicit Deny in the S3 bucket policy or the KMS key policy. An explicit deny overrides everything, it will block the request even if you have admin permissions on the account.

What Is MinIO?

MinIO is an open-source, high-performance, S3-compatible object storage system. It’s a self-hosted alternative to Amazon S3: it provides S3-compatible APIs and ecosystem compatibility while giving you full control over your data.

MinIO can run on a single machine, inside a Kubernetes cluster, or across large distributed infrastructure, scaling from a single server up to petabytes of distributed storage.

The practical implication: if your application can talk to S3, it can generally talk to MinIO. In many cases the main change is pointing your client at a different endpoint (DNS/URL) instead of the AWS S3 endpoint.

Compatibility caveat: MinIO implements the S3 API, but not every S3 feature behaves identically. Server-side features that depend on AWS-specific services, most notably SSE-KMS backed by AWS KMS, do not map one-to-one to MinIO. Treat “S3-compatible” as “compatible at the API level,” and validate any feature that depends on AWS-native services.

Why We Chose MinIO

Cost efficiency. MinIO is open-source and free to use. Running it yourself avoids vendor lock-in and reduces cloud storage bills.

Get Dixit Rathi’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

S3 compatibility. MinIO speaks the Amazon S3 API, so tools and SDKs that already work with S3 also work with MinIO, with minimal relearning.

Community and open source. MinIO has an active open-source community and transparent development.

These are the reasons that applied to our use case. Whether they apply to yours depends on your operational capacity to run and maintain storage infrastructure yourself, which is a cost that self-hosting shifts onto your team rather than eliminating.

Migration Overview

At a high level, the migration copies objects from a source S3 bucket to a destination MinIO bucket using the MinIO client (mc) as an intermediary. The mc client machine needs network access to both the AWS S3 endpoint and the on-prem MinIO endpoint.

Prerequisites

Before starting, make sure the following are in place:

1. MinIO is already set up and running.
2. The mc client (MinIO’s CLI) is installed and configured on a machine that will drive the migration.
3. Network connectivity exists between all three components: the AWS S3 bucket, the MinIO server, and the mc client machine.
4. The source bucket has no explicit Deny in its bucket policy or KMS key policy that would block reads.

Step-by-Step Migration

Step 1: Create a scoped AWS user for the migration

Create an AWS IAM user that has permissions limited to the specific S3 bucket being migrated. Read permission is sufficient for the bucket itself. If the bucket uses SSE-KMS encryption, also grant kms:Decrypt so the migration user can read encrypted objects.

Then create an access key and secret key for this user.

Why scoped permissions: Limiting the migration user to read-only on a single bucket (plus kms:Decrypt if needed) follows least-privilege and reduces blast radius if the credentials leak.

Step 2: Configure mc aliases for source and destination

Configure the mc client with aliases for both the source S3 bucket and the destination MinIO server. An alias stores the endpoint URL and credentials under a short name you can reference in later commands.

# Source: AWS S3 (ap-south-1 region endpoint)
mc alias set source-s3 https\://s3.ap-south-1.amazonaws.com ACCESS_KEY SECRET_KEY
OUTPUT:- "Added `source-s3` successfully"

# Destination: on-prem MinIO
mc alias set destination-minio https\://on-prem.minio.example.com ACCESS_KEY SECRET_KEY
OUTPUT:- "Added `destination-minio` successfully"

• source-s3 / destination-minio are the alias names.
• The URL is the S3 or MinIO endpoint.
• ACCESS_KEY / SECRET_KEY are the credentials for each side (the AWS user from

After the aliases are set, validate connectivity to both by listing their contents:

mc ls source-s3
mc ls destination-minio

If both commands return without errors, the client can reach and authenticate against both endpoints.

Step 3: Migrate the data

Use mc mirror to replicate objects from the source S3 bucket to the destination MinIO bucket, then use mc diff to compare the two and confirm the copy is complete.

Copy objects from S3 to MinIO

mc mirror source-s3/bucket_name destination-minio/bucket_name

Compare source and destination to find any differences

mc diff source-s3/bucket_name  destination-minio/bucket_name

• mc mirror copies objects from the source bucket/prefix to the destination, making the destination match the source.
• mc diff reports objects that differ between source and destination. After a successful mirror, this should report no differences (or only expected ones).

Note on mc mirror: By default, mc mirror copies objects that are new or changed. It does not, by default, delete objects on the destination that no longer exist on the source unless you use the removal/overwrite flags. Confirm which behavior you relied on.

Validation / Testing

The migration was validated using mc diff to compare source and destination after mirroring.

Conclusion

Replicating data from Amazon S3 to on-premises MinIO is straightforward at the API level because MinIO is S3-compatible, the core of the migration is three

steps: create a scoped AWS user, configure mc aliases for source and destination, and run mc mirror followed by mc diff to validate.

The part that requires the most attention is encryption. The encryption method on your source bucket (SSE-S3, SSE-KMS, SSE-C, or client-side) determines the

permissions your migration user needs, and encryption settings do not carry over to MinIO automatically. Understand your source encryption first, grant the

right permissions (kms:Decrypt for SSE-KMS), and configure encryption on the destination separately if you need it.