Medium

Christian Hansen

In

Databricks Community Articles

by

Christian Hansen

·

Nov 2

Apache Spark 4.3: What Developers Need to Know

Spark 4.3 adds ASOF JOIN, Python UDFs compiled to SQL, JSON_TABLE, and SCD Type 2 Auto CDC: what works, what needs a flag, and what breaks

In

Databricks Community Articles

by

Christian Hansen

·

Oct 19

Databricks Multi-Statement Transactions Are GA: All Tables or None

Databricks multi-statement transactions commit several Delta tables together or roll them all back. How to build pipelines on BEGIN ATOMIC.

Databricks multi-statement transactions: three Delta tables committed together with BEGIN ATOMIC

In

Databricks Community Articles

by

Christian Hansen

·

Oct 11

Databricks ABAC Now Follows Your Data: External Engines and Views

Databricks ABAC row filters and column masks now reach external Apache Spark (GA) and Unity Catalog views (Beta)

Databricks ABAC in Unity Catalog now enforced for external engines and views

In

Databricks Community Articles

by

Christian Hansen

·

Oct 4

Databricks ai_decide: The AI Judge That Never Writes a Word

Use Databricks ai_decide() to grade every ai_query output in SQL and decide what gets sent, without a second LLM

Databricks ai_decide: The AI Judge That Never Writes a Word

In

Databricks Community Articles

by

Christian Hansen

·

Sep 23

Databricks Governance Hub: One View of Data, AI, and Cost Across Every Workspace

Databricks Governance Hub brings data, access, AI, cost, and tag governance into one account-level view. Here is what is in the Beta.

In

Databricks Community Articles

by

Christian Hansen

·

Sep 11

Delta Lake 4.3: Selective Data Replacement and the Unity Catalog Delta APIs

replaceUsing and replaceOn give you a better overwrite primitive, and the catalog now validates every operation, not just the commit.

In

Databricks Community Articles

by

Christian Hansen

·

Sep 7

Databricks Production Planning: How to Actually Use the Deployment Guide

A practical read of the 10-phase Databricks deployment guide: what to decide first, and how to use it on a platform you already run.

In

Databricks Community Articles

by

Christian Hansen

·

Sep 1

Databricks ai_classify: Classify Text in SQL

Databricks ai_classify() is GA: classify text in SQL with your own labels, plus v2.1 confidence scores and rationales

Databricks ai_classify: Classify Text in SQL

In

Databricks Community Articles

by

Christian Hansen

·

Aug 17

Databricks Lakehouse Replay: Testing the Next Runtime on Your Own Queries

How Databricks Lakehouse Replay reruns your read-only serverless queries on unreleased runtimes to catch regressions before they ship

Databricks Lakehouse Replay: Testing the Next Runtime on Your Own Queries

In

Databricks Community Articles

by

Christian Hansen

·

Aug 11

Databricks Pipeline Unit Testing Finally Arrives

Databricks pipeline unit testing hits Beta: pytest-style tests for Lakeflow Declarative Pipelines, with mock data and Auto CDC coverage.

Title card on a dark background reading DATABRICKS PIPELINES FINALLY GET UNIT TESTING, above a row of four status circles: three green check marks and one red cross.

Christian Hansen

Christian Hansen

Friend of Medium

Databricks Champion, Lead Data & Machine Learning Engineer at Halfspace LinkedIn: https://www.linkedin.com/in/cenh/

Following

Help

Status

About

Careers

Press

Blog

Store

Privacy

Rules

Terms

Text to speech