InDatabricks Community ArticlesbyVenugopal Dabbara·1d agoSpark 4.3 AUTO CDC: SCD2 Made SimplerWhat’s new in upstream Spark SQL, how selective history tracking works, and why Databricks users may already be familiar with it.
Venugopal Dabbara·Nov 9Databricks AI Function Jobs (Beta): Parse, Extract & Classify in Lakeflow JobsTurn unstructured data into structured outputs with AI-powered tasks — no model or endpoint management required.
Venugopal Dabbara·Nov 5Apache Spark 4.3: Goodbye ZIP Extraction, Hello Direct Archive Reading!Hands-on experiments with CSV, JSON, XML, archive filtering and file lineage.A response icon1A response icon1
Venugopal Dabbara·Nov 3Beyond Micro-Batching: Exploring Real-Time Mode in DatabricksFrom Kafka transactions to operational decisions: a hands-on experiment using Databricks serverless Spark Declarative Pipelines, and…
Venugopal Dabbara·Oct 29Exploring Databricks Unity AI Gateway ObservabilityUnderstanding how Unity AI Gateway brings model usage, performance, token consumption, cost insights, MCP tools and coding-agent activity…A response icon2A response icon2
Venugopal Dabbara·Oct 28Databricks Serverless DataFrame CachingServerless caching is now in Public Preview. But if you’re familiar with Spark caching on classic compute, a few behaviours might surprise…
Venugopal Dabbara·Oct 23Databricks Genie Code’s Compute Migration AgentFrom classic-compute configuration to a working serverless job using Genie Code’s /compute command.
Venugopal Dabbara·Oct 19Databricks Governed Tag Propagation (Beta): What Worked and What I FoundAs data moves from Bronze to Silver to Gold, keeping governance metadata consistent across tables can become tedious. If a column is tagged…