Partitioning vs. Liquid Clustering😮

Why One Requires a Rewrite and the Other Doesn’t?

Photo by Nick Brunner on Unsplash

Key Note: Partitioning fixes the physical directory structure, whereas Liquid Clustering lets you change the clustering rules without immediately rewriting all existing data.

Let’s understand why.

1. Why does changing partitioning require rewriting data?

Suppose you have a Delta table with 1 billion records partitioned by year.

CREATE TABLE sales (
    sale_id BIGINT,
    sale_date DATE,
    customer_id BIGINT,
    amount DOUBLE
)
USING DELTA
PARTITIONED BY (year);

Conceptually, the storage looks like this:

Now, six months later, your queries change. Instead of filtering by year, users frequently search for customer_id. You decide that partitioning by customer would be better.

PARTITIONED BY (customer_id)

Why can’t you change the partition column?

• Because the existing files are physically organised according to the old partition layout.
• A file in year=2025/ may contain records for…