Part 1 — Building a local Apache Iceberg v3 lakehouse
Why bother with v3?
Iceberg v2 solved row-level deletes with positional delete files: every `DELETE` or `UPDATE` in merge-on-read mode writes another small Parquet file listing which rows to skip. Over time a table accumulates thousands of them, and every read has to open and apply them all.
Format version 3 addresses the pain points that teams have lived with in production:
Get Sercan Bilgiç’s stories in your inbox
Join Medium for free to get updates from this writer.
v3 capabilities and Problem it solves :
- Deletion vectors: One compact bitmap per data file instead of many delete files. Far fewer files to open; compaction merges them away.
- Row lineage: Built-in `_row_id` / `_last_updated_sequence_number` on every row. Change detection without a custom `updated_at` column or CDC tooling.
- Column defaults: Add a column with a default; existing rows read it with zero rewrite.
- New types: `variant` for semi-structured data, nanosecond timestamps, geometry/geography.
The goal of this setup: write v3 tables with one engine, read them with another, and prove the features are real rather than slideware.
Architecture
The division of labour is deliberate and mirrors a common production shape — Glue/Spark writes, Athena reads:
- Spark performs v3 writes, `DELETE`/`UPDATE`/`MERGE`, and all maintenance procedures.
- Trino is the query engine. Athena is Trino, so the SQL dialect in the UI is identical to what you’d type into the Athena console.
- CloudBeaver is the browser UI in front of Trino.
- The REST catalog is the single source of truth both engines agree on. Neither talks to the other; they only share metadata.
Both engines run Iceberg 1.11.0 — the writer and the reader use the same library version, which matters for a format that’s still stabilising.
The Dremio decision
The original plan was Dremio as the UI. Checking before building saved a wasted afternoon:
- Dremio’s Iceberg v3 support shipped in Dremio Cloud (April 2026). Dremio Software is promised “later in 2026”.
- The dremio/dremio-oss Docker image has been frozen at 26.0.5 for months, with community questions about whether OSS images are still being published.
- It would have been unable to read the very tables this environment exists to test.
Trino 483 documents v3 as experimental, but it reads deletion vectors, creates v3 tables, and — as Part 2 shows — exposes the row-lineage columns. Good enough to learn on, and the same dialect as Athena.
Components and versions
Every image was verified to publish `linux/arm64` (Apple Silicon) as well as `amd64`.
Iceberg jars baked into the Spark image at build time (so it starts offline):
org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.11.0
org.apache.iceberg:iceberg-aws-bundle:1.11.0Prerequisites
Docker via Colima on macOS. The default Colima VM (**2 CPU / 2 GiB**) is too small — Trino alone wants a 2 GB heap.
colima stop && colima start - cpu 4 - memory 8Repository layout
https://github.com/blgcsrcn/iceberg-demo
├── docker-compose.yml # the five services
├── lake.sh # helper: up / down / reset / demo / spark / trino / sql / tsql
├── spark/
│ ├── Dockerfile # apache/spark + Iceberg 1.11.0 jars
│ ├── spark-defaults.conf # catalog "lake" -> REST -> MinIO
│ └── bin/sql, bin/sqlf # one-shot SQL wrappers
├── trino/
│ ├── config.properties # single-node coordinator
│ └── catalog/lake.properties # Iceberg connector -> same REST + MinIO
├── cloudbeaver/
│ └── initial-data-sources.conf # pre-registered Trino connection
└── sql/
├── 01_create_v3_table.sql # v3 table, DVs, row lineage
├── 02_variant_type.sql # VARIANT column
├── 03_maintenance.sql # compaction, expiry, orphans, manifests
└── 04_query_from_trino.sql # read everything back in Athena dialectStartup sequence
`depends_on` with health conditions enforces this order; nothing starts until its dependency is genuinely ready, not merely running.
Key configuration
Spark → catalog (`spark/spark-defaults.conf`)
spark.sql.extensions org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions
spark.sql.catalog.lake org.apache.iceberg.spark.SparkCatalog
spark.sql.catalog.lake.type rest
spark.sql.catalog.lake.uri http://rest:8181
spark.sql.catalog.lake.warehouse s3://warehouse/
spark.sql.catalog.lake.io-impl org.apache.iceberg.aws.s3.S3FileIO
spark.sql.catalog.lake.s3.endpoint http://minio:9000
spark.sql.catalog.lake.s3.path-style-access true
spark.sql.defaultCatalog lakeTrino → catalog (`trino/catalog/lake.properties`)
connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=http://rest:8181
iceberg.rest-catalog.warehouse=s3://warehouse/
iceberg.format-version=3
fs.native-s3.enabled=true
s3.endpoint=http://minio:9000
s3.region=us-east-1
s3.path-style-access=true
s3.aws-access-key=admin
s3.aws-secret-key=passwordThe v3 table itself
CREATE TABLE lake.demo.orders (
order_id BIGINT, customer_id BIGINT, status STRING,
amount DECIMAL(12,2), order_ts TIMESTAMP, region STRING
)
USING iceberg
PARTITIONED BY (region)
TBLPROPERTIES (
'format-version' = '3', - <- this is the v3 switch
'write.delete.mode' = 'merge-on-read', - required for deletion vectors
'write.update.mode' = 'merge-on-read',
'write.merge.mode' = 'merge-on-read'
);`format-version = 3` is the whole trick. Merge-on-read is what makes deletes produce deletion vectors instead of rewriting data files.
Verification — what was actually observed
It really is v3
SHOW TBLPROPERTIES lake.demo.orders;
A DELETE writes a deletion vector, not a delete file
SELECT content, file_format, record_count, file_path FROM
lake.demo.orders.files;
`content = 1` with `file_format = PUFFIN` is the v3 signature. On a v2 table this would be a `PARQUET` positional-delete file.
Compaction merges the DV away
After
ALTER TABLE lake.demo.orders EXECUTE optimize;
Trino reads a live deletion vector written by Spark
The strongest cross-engine test: Spark deletes a row (writing a DV), Trino — without compaction — returns the correct row count and lists the Puffin file in `”orders$files”`.
Writes work both ways
Trino: `CREATE TABLE from_trino (…) WITH (format_version = 3)` + `INSERT`. Spark reads it back.
