Apache Spark 4.3: What Developers Need to Know

Spark 4.3 adds ASOF JOIN, Python UDFs compiled to SQL, JSON_TABLE, and SCD Type 2 Auto CDC: what works, what needs a flag, and what breaks

Apache Spark 4.3.0 shipped on October 30, 2026, and it is a different kind of release. It is the first on Spark’s new quarterly train: feature releases now land every three months, each gets six months of maintenance, and the long-term support release for the 4.x line will be 4.5, not 4.3. If you run production pipelines on open-source Spark, that changes how you plan upgrades before you read a single feature.

The features are worth reading, though. Apache Spark 4.3 finally gives SQL a point-in-time join, can turn simple Python UDFs into native SQL expressions, adds the SQL-standard JSON_TABLE, lets you edit VARIANT values in place, and teaches Declarative Pipelines to keep SCD Type 2 history. It also changes a handful of defaults that will surprise you on the first run.

The release notes link to a long JIRA list and say little else, so I installed pyspark==4.3.0, read the migration guides, and ran every SQL and DataFrame example below; the pipeline and real-time sections rely on the 4.3.0 docs, API, and error messages. That matters, because several headline features ship disabled. The examples use a sample company, CH Enterprise, and temporary views that the companion notebook…