Databricks has released major updates to its AI ecosystem, highlighting the GA of Unity Gateway APIs, new governance and tracing features for AI coding agents and enhanced capabilities in ai_search and document processing.

Unity Gateway Updates

• Enforce third party guardrails on Unity Gateway Traffic. The external service policy sends requests and responses content to your guardrail vendor and blocks the call when the vendor denies it.
• Claude Code auto mode for Anthropic and non anthropic models support the auto mode reviews to reduce routine permissions.

ug claude — — permission-mode auto

• Amazon Bedrock Mantle are available through Unity Gateway with the permissions, rate limits, budgets and inference tables.
• Custom Policies verify caller supplied request tags before a model request or MCP toll call proceeds.
• Skills API are beta with the possibility to create, read, list, delete and finalize support across Databricks Developer Tools (Cli, Python, Go; Java and JS SDK).
• Users with Consumer Access entitlement can query model services and model provider services shared with them or to use Unity Gateway coding agents features.
• Unity Gateway usage Dashboard is available. Exploring the usage, coding agent adoption, harness usage and estimated session spend become easy.

• Harnesses can connect to Databricks MCP Servers using default OAuth Client IDs.
• Tracking Skill usage become easy with Audit Logs under the aiGatewaySkills services
• Developers can screen tools call and their results in Unity Gateway for jailbreak and prompt injection attempts.

You can use ug to configure Claude Desktop and Chatgpt Desktop work agents with governed models, MCP and skills.

• Get the spend status to track the budget against Unity Gateway budget limit and use ug usage to show up to 5 personal and shared spent limits.

Agent Bricks

• End-to-End Agent Scaffolding: A new developer CLI for building, testing, and deploying custom code agents.
• Integrated Agent Runtime: Scaffold projects from standard templates, test locally, and deploy to the Databricks Agent Runtime with managed memory, sessions, tools, and MLflow tracing pre-configured.
• Memory pipelines bridges raw session and long term memory. At each run an LLM analyzes stored chat histories to extract facts, user preferences and key decisions indexing them into a retrievable memory entries for future sessions. The pipeline can run as a scheduled Background job or executed on Demand

Key Takeaway: Full-lifecycle developer tooling, managed runtime, and automated long-term memory for Databricks agents.

AI functions enhancements

• AI functions Unity Catalog permissions are GA. Admin can setup the permissions on system.ai schema to controls which tasks can users uses.
• Standard AI_EXTRACT are faster while improving the overall quality without additional cost.
• AI_PARSE_DOCUMENT return detailed figures and for the charts, the description, series, legends and axes are structured as JSON.
• AI_ENRICH is beta can ground each row with a search tool on an external Model Context Protocol (MCP) server that you register in Unity Gateway, such as You.com or Exa.

Mlflow enhancements

• Select OpenJev (Qwen3.5 4B) as the evaluator model service of a custom LLM as judge service policy
• Mlflow logs tab for AI runtime has been redesigned. It does support searching and filtering logs by node and time range following a running workload as new output arrives and downloading logs.

AIBI, Genie and BI Migration

• Bulk import from Power BI and Tableau: Accelerate legacy BI modernization by batch converting up to 100 Power BI or Tableau workbooks to native AIBI Dashboards, Metric Views eliminating manual reconstruction across large datasets.
• Import Power BI .tmdl files: Authors can upload Power BI .tmdl files to create metric views with the /importBI Genie Code skill.

Genie Code for dashboard authoring:

• Genie Code can use images stored in your workspace as image widgets, not only uploaded files.
• The dashboard canvas stays visible while Genie Code builds datasets step by step, so you can watch widgets appear instead of seeing a blank screen.
• Genie Code can create dashboard materializations when you ask it to

Data Modeling and Datasets

• Materialize SQL datasets (Beta): Drastically reduce dashboard load times and save compute resources by precomputing results for raw SQL datasets alongside local metric views bypassing repetitive and heavy query execution.

• Lakehouse Real-Time warehouse support (Beta): Power Operational and streaming analytics with sub second responsiveness by connecting AIBI dashboard with Lakehouse RT
• Decimal precision : Ensure absolute fidelity across financial and scientific reporting as Generated visualizations respect defined decimal precision rules.
• Select multiple tables when adding a data source: You can now select multiple tables at the same time in the Add data source picker when building a dashboard data model.
• Dashboard relationships, which let you build complex multi-fact data models scoped to an AI/BI dashboard, are GA.

Visualization and Dashboard Design

• Custom theme scope: Custom themes no longer color the View queries dialog, widget context menus, or data reference popovers, matching the behavior of built-in themes.
• Funnel chart fields: Fields carry over when you change a chart to or from a funnel chart.
• Sankey label improvements: Labels on Sankey diagrams overlap less.
• Field picker tooltips: Hovering over a field or measure name in the field picke shows its full name in a tooltip.
• Layered combo charts: Authors can build combo charts from separate chart layers.
• Data labels and custom total label for waterfall charts: Authors can add data labels to waterfall charts and customize the total bar’s display name, color, and label.
• Pivot table drill-down: Viewers can drill down through the row and column hierarchy of a pivot table one level at a time.
• Theme fonts for rich text titles: Rich text widget titles and descriptions inherit the dashboard theme’s font styles.
• Faceted counters: You can facet a counter visualization to show one counter tile per dimension value.
• Tabbed editor popover height: Tabbed editor popovers keep their opening height instead of growing.

Inactivity and Filters

• Single-field filter default: Changing the field on a single-field filter clears the previous field’s default value. The filter falls back to All, or to None when All isn’t allowed.
• Widget-level filter refresh: The values in widget-level filter refresh when a dataset changes, without a browser refresh.
• Private bookmarks: Dashboard viewers can save filter combinations as private bookmarks to return to later.

Private bookmarks are visible only to the user who creates them.

• Legend cross-filtering: Selecting an item in a visualization’s legend applies cross filtering
• Global filter display in embedded dashboards: Embedded dashboards can show, collapse, or hide global filters by default.

Exporting, Delivery and Scheduled Tasks

• Granular PDF and Smarter email delivery: Get precise control over report sharing by compiling only select dashboard pages into a single PDF backed by more resilient scheduled distribution that delivers whichever attachments fit under email size thresholds.

• Dashboard task page selection and tabular attachments: Dashboard tasks in jobs can optionally include dashboard page selection and tabular attachments.
• Dashboard Insights (Beta): Viewers can have Genie analyze the charts on a published dashboard page and summarize notable findings, such as trends and outliers, as insight cards.

Key Takeaway: AIBI Dashboards are transitioning into a mature, full-scale replacement for Power BI and Tableau by pairing automated bulk migration with enterprise-grade data modeling and interactivity. Coupled with native Lakehouse performance, Genie expands beyond basic Q&A into an end-to-end co-pilot that actively builds dashboards, optimizes compute, and surfaces automated insights.

Agent Models and performance

• Flash (Beta): Prioritize rapid execution speed over analytical depth for routine everyday tasks.

• Agent mode model upgrade: Agent mode now uses upgraded models that better handle long context and respond faster.

Integrations and external tools

• Connect to external tools (Beta): A Genie Agent can search and take actions in external tools like Google Drive through MCP connectors.

Volumes and content search

• Faster file retrieval (Beta): Content search parses and indexes your files so the Genie Agent can reason over a larger set of files, draw on specific passages across many files, and answer questions with lower latency.
• Content search details hidden from viewers (Beta): The context panel no longer shows technical content search details, such as the number of files indexed, to users with CAN VIEW or CAN RUN permission on a Genie Agent.
• Automated Content Search and Volume Sync (Beta): When you attach a volume to a gent agent, Databricks automatically enables content search and keeps the index continuously updated as files are added or removed without inquiring any manual intervention.
• Volumes from restricted catalogs (Beta): You can attach a volume as a source for a Genie Agent even when the volume’s catalog isn’t accessible from the agent’s workspace by default

Authoring and Embedding

• Guided agent creation: A new guided experience walks you through creating an effective Genie Agent using Genie Code. It reads your data and suggests context that helps Genie answer questions accurately, such as descriptions of your tables and example queries.

• Deletion warning for sources and instructions: Deleting sources or instructions from a Genie Agent requires explicit permission.

• Volume file links in embedded agents: Links to volume files in an embedded Genie Agent open the file instead of redirecting users to the workspace.

Governance and budget

• Budgets used to manage Genie costs support access control lists (ACLs): Account admins and workspace admins can grant the Budget Manager or Budget Viewer role to specific users, groups, or Databricks service principals, delegating who can view and manage each budget.

Key takeway: This release significantly expands genie Agent autonomy and combining low latency flash execution and external MCP connectivity with zero touch volume indexing, streamlined embedding and granular budget controls

Chat and User experience

• Conversation outline: Navigate lengthy threads using the new left edge outline hover over outline markers to instantly preview a message and the start of its response.
• Reuse context from past investigations: Genie One can use the context from the earlier investigation to answer faster when you ask a question similar to one you asked before.
• Streaming responses: The answers streams progressively in real time rather than loading in a single batch
• Download a chat: You can export and download Genie One conversation transcripts in the side bar.
• Personalized sample questions: Tailored questions appear beneath the search bar on the Genie One homepage.

Skills and extensibility

• Upload skills (Public Preview): Directly upload existing skills files and folders to your skills folder in the workspace.
• Share skills with Unity Gateway (Beta): Publish and govern Genie One skills via Unity Catalog schema and share them with your team.

Data processing, files and executions

• Snippets from external tool queries (Public Preview): Genie Ontology can extract snippets from queries that external tools, such as dbt run on Databricks.
• Code execution in Genie One: Genie One can perform advanced analysis beyond standard SQL by executing code in a secure, isolated sandbox and operating entirely within your existing Unity Catalog data access control.

• Upload and analyze supported file types in natural language across Genie One and Agents Upload a file and Upload a file to a Genie Agent.

Collaboration, governance and Access

• Collaborative document editing: People you share a Genie One document with can now edit it if they have CAN EDIT permission.

• Group memberships in user info: When Genie One retrieves a user’s information, it also returns the user’s group memberships. You can use workspace instructions and table descriptions to tailor how Genie One responds to members of specific groups.
• Remembered sign-in email in the Genie mobile app (Public Preview): The Genie mobile app remembers the email address of the last user who signed in and prefills it at the next sign-in. See Use Genie on mobile.

The Takeaway: This release significantly matures the Genie One ecosystem by introducing governed skill sharing via Unity Catalog, unleashing secure code execution for advanced analytics and deploying sweeping quality of life UI upgrades to accelerate daily workflows.

https://www.youtube.com/watch?v=OPQODfOK_wM

TL;DR Governance Hub is an account-level interface that pulls data governance, AI usage, and cost monitoring into one place. It’s currently in Beta. Its AI page surfaces token usage, AI spend, what share of AI traffic is covered by guardrails, budget status, the most-active models and users over the last 30 days, and recommendations like flagging high-volume models missing inference tables. It’s a monitoring and insights layer, not a new permission system Unity Catalog is still where access, lineage, and auditing live. An account admin turns it on under Account console → Previews.

What is the Databricks Governance Hub?

If you’ve ever tried to answer “are our AI workloads actually governed, and what are they costing us?” by stitching together five screens, this is the fix.

Databricks Governance Hub is an account-level interface for monitoring data governance, AI usage, and cost in one place, and it is currently in Beta.

• Account-level — one view across your workspaces, not per-workspace guesswork.
• Three lenses — governance coverage, AI usage, and cost, together.
• Insights, not enforcement — it tells you where you’re exposed; it doesn’t change grants.

What does the AI page show?

The Governance Hub AI page reports token usage, AI spend, the percentage of AI traffic covered by guardrails, budget status, the most-active models and users over the last 30 days and recommendations such as high-volume models missing inference tables.

• Usage & spend : tokens consumed and the bill behind them.
• Guardrail coverage : the share of AI traffic actually governed.
• Budgets: current status, with links to manage them.

How does it relate to Unity Catalog?

“governance hub” sounds like it should replace Unity Catalog. It doesn’t it reads from it. Unity Catalog remains where access, lineage, and auditing are enforced; Governance Hub is the monitoring and insights layer on top, and it does not add permissions or manage access itself.

How do I turn it on?

An account admin enables Governance Hub from

Account console → Previews

after which its pages appear under Governance in the account-console sidebar.

FAQ

Is Governance Hub generally available?

No it’s in Beta as of this release.

Does Governance Hub manage permissions?

No. It’s a monitoring and insights layer. Access control, lineage, and auditing stay in Unity Catalog.

What does it tell me about AI?

Token usage, AI spend, guardrail coverage, budget status, most-active models and users (last 30 days),and recommendations.

How do I enable it?

An account admin turns it on under Account console → Previews; the pages then show up under Governance in the sidebar.

Related:

·What’s new in Unity Catalog — September 2026

· What’s new in Genie One — September 2026.

Sources:

·Governance Hub overview

· AI usage in Governance Hub

· Unity Catalog

· Databricks Answers: Unity Catalog

Databricks One-Stop Shop | Unity Catalog & Lakeflow | NextGenLakehouse

https://www.youssefmrini.tech

TL;DR. Unity Catalog secrets let you store credentials (API keys, tokens, passwords) as governed objects addressed as catalog.schema.secret with the same grant model as the rest of Unity Catalog. They’re different from workspace secret scopes, which are workspace-level objects addressed as scope/key. To create one you need CREATE SECRET (with USE CATALOG/USE SCHEMA) on the schema; to read one you need READ SECRET plus USE CATALOG and USE SCHEMA on the parents. Retrieve it in code with dbutils.secrets.get(catalog=, schema=, key=), which requires Databricks Runtime 17.3 LTS+ or serverless environment version 4+.

What are Unity Catalog secrets?

They’re a way to keep sensitive values out of your notebooks and jobs while governing them exactly like tables and models by catalog, schema, and grant.

Unity Catalog secrets are governed securable objects that store sensitive values (API keys, tokens, passwords) under a three-level `catalog.schema.secret` namespace, managed with standard Unity Catalog privileges.

• Addressed as catalog.schema.secret, so access follows your existing UC hierarchy.
• Managed with grants (CREATE SECRET, READ SECRET), not a separate ACL system.
• Keep credentials out of code, config files, and notebook output.

How are UC secrets different from secret scopes?

This is the most common point of confusion. Both store secrets, but they’re separate systems.

Unity Catalog secrets live at `catalog.schema.secret` and are governed by Unity Catalog, workspace secret scopes are workspace-level objects addressed as `scope/key. You do not create a secret scope inside Unity Catalog.

• UC secrets governed by catalog/schema grants; read with dbutils.secrets.get(catalog=, schema=, key=)
• Workspace secret scopes created and managed with the Databricks CLI; read with dbutils.secrets.get(scope=, key=)

How do I create and read a UC secret?

Creating a UC secret requires `CREATE SECRET` (with `USE CATALOG` and `USE SCHEMA`) on the schema; reading one requires `READ SECRET` plus `USE CATALOG` and `USE SCHEMA` on the parent catalog and schema.

The flow is: grant the privilege, create the secret, then grant read access.

• Grant creation rights on the schema: GRANT USE CATALOG ON CATALOG main TO my-group; then GRANT USE SCHEMA, CREATE SECRET ON SCHEMA main.default TO my-group;
• Create the secret in Catalog Explorer (catalog → schema → Create → Secret), or via POST /api/2.1/unity-catalog/secrets.
• Grant read access: GRANT READ SECRET ON SECRET main.default.api_key TO my-group;
• Read it in code: dbutils.secrets.get(catalog=”main”, schema=”default”, key=”api_key”).

FAQ

Are Unity Catalog secrets the same as secret scopes?

No. UC secrets are governed objects at catalog.schema.secret; secret scopes are workspace-level objects at scope/key. You manage them with different tools and APIs.

What privileges do I need to read a UC secret?

READ SECRET on the secret, plus USE CATALOG and USE SCHEMA on its parent catalog and schema.

Which runtime supports the UC secrets lookup?

dbutils.secrets.get for UC secrets requires Databricks Runtime 17.3 LTS or above, or serverless environment version 4 or above.

Can I create a UC secret without SQL?

Yes — use Catalog Explorer (Create → Secret) or the REST API (POST /api/2.1/unity-catalog/secrets)

Related

·What’s new in Unity Catalog — September 2026

· Stored Procedures in Databricks SQL.

Sources

·Secrets in Unity Catalog

· Workspace secret management

· Unity Catalog securable objects. ·

· Databricks Answers: Unity Catalog

TL;DR: Real-time data engineering on Databricks boils down to three pieces working together: Structured Streaming runs the logic, Auto Loader grabs the raw files, and Lakeflow Pipelines keep the whole machine alive in production.

The Three Core Layers

Structured Streaming (The Engine): Write standard batch DataFrame or SQL queries; Spark executes them incrementally as data lands. It guarantees fault-tolerant, exactly-once processing using:

• Checkpoints: Save stream state to recover seamlessly after failures.
• Output Modes: Determine how rows write out (append, update, or complete).
• Triggers: Set the processing rhythm, from continuous processing to micro-batches

Auto Loader (The Ingest Tool): Purpose-built to ingest files landing in cloud storage (cloudFiles).

• Ingests only new files without costly directory re-scans.
• Automatically detects schema evolution and dumps unparseable records into a _rescued_data column.
• Effortlessly scales to millions of files an hour without custom state tracking.

Spark Declarative Pipelines : Removes the operational glue work.

• Define streaming tables declaratively instead of manually wiring checkpoints and schema paths.
• Native integration with Auto Loader via Python (pyspark.pipelines) or SQL (read_files).
• Delivers automated lineage, cluster orchestration, error recovery, and retries.

Real-Time Mode (RTM) in Databricks is an evolution of Apache Spark Structured Streaming that delivers end-to-end latencies down to 5 to 50 milliseconds (up to 100x faster than traditional micro-batch mode).

Core Architectural Innovations

RTM replaces periodic batching with continuous execution using three main mechanics

• Continuous Data Flow: Processes each event immediately upon arrival instead of collecting them into discrete, periodic chunks.
• Simultaneous Stage Scheduling: Runs all query stages concurrently like a conveyor belt rather than blocking upstream tasks.
• Streaming Shuffle: Bypasses disk-based shuffles by passing data continuously and directly in-memory between tasks

FAQ

Is Structured Streaming different from batch code?

• No. It uses identical Spark SQL and DataFrame APIs — Spark just runs it continuously and tracks progress via checkpoints.

When should I use Auto Loader vs. standard stream reads?

• Use Auto Loader for files landing in cloud storage (S3, ADLS, GCS). Use direct Structured Streaming connectors for message buses like Kafka or Event Hubs.

Is Lakeflow Pipeline mandatory?

• No, but managing raw streaming jobs manually is fragile. Lakeflow automates state management, checkpointing, and retries.

How does Auto Loader handle unexpected schema changes?

• It infers new types on the fly and routes mismatched fields to the rescued-data column, preventing pipeline crashes.

Related:

·Getting Started with Python Unit Testing in Databricks

· What’s new in Unity Catalog — September 2026.

Sources:

·Structured Streaming concepts

· What is Auto Loader?

· Schema inference and evolution

· Lakeflow Declarative Pipelines

· Load data in pipelines

. Databricks Answers: Data Engineering

Databricks’ fully managed Postgres for operational apps — what it is, how it connects to the lakehouse, and what “LTAP” actually means.

TL;DR. Lakebase is Databricks fully managed, serverless PostgreSQL database for OLTP (transactional) workloads, built into the Data and AI Platform and governed by Unity Catalog. It autoscales (including scale to zero) and supports instant database branching.

What is Databricks Lakebase PostgreSQL?

Lakebase is an operational database you don’t have to run. Instead of provisioning a Postgres server, sizing it, and patching it, you get a serverless Postgres endpoint that scales up on demand and down to zero when idle and it lives inside the same platform and governance as your analytics.

Databricks Lakebase is a fully managed, serverless PostgreSQL database for transactional (OLTP) workloads, integrated with the lakehouse and governed by Unity Catalog.

• Standard Postgres — use your existing Postgres drivers, SQL, and tools.
• Autoscaling + scale-to-zero — capacity follows load; you don’t pay for idle compute.
• Branching — spin up an instant copy of a database for dev/test, like branching code.
• Governed — access and lineage flow through Unity Catalog, not a separate silo.

Why Postgres for OLTP in a lakehouse?

The lakehouse is excellent at analytics (OLAP) but was never meant to serve the fast, single-row reads and writes that applications need. Lakebase fills that gap with Postgres the most widely adopted open-source transactional database so your app layer and your analytics layer finally share one platform instead of a brittle pipeline between two vendors.

Lakebase brings OLTP to the lakehouse so operational apps and analytics run on one governed platform, removing the ETL hop between a separate app database and your analytical data.

How does Lakebase connect to the lakehouse?

Two distinct, one-directional paths it helps to keep them straight:

• Synced tables (Unity Catalog → Lakebase). Databricks keeps a managed Postgres copy of a Unity Catalog table so apps get low-latency reads. Treat synced tables as read-only; application writes belong in your own operational tables.
• Unity Catalog registration (Lakebase → Unity Catalog). Registering a Lakebase database exposes it as a read-only catalog in Unity Catalog, so analysts and Genie can query operational data under the same governance. Querying it requires serverless compute — Pro/classic SQL warehouses and classic compute aren’t supported.

Synced tables push governed lakehouse data into Postgres for fast app reads, Unity Catalog registration exposes a Lakebase database back to the lakehouse as a read-only, governed catalog.

What is LTAP, and where does Genie fit?

LTAP (Lake Transactional/Analytical Processing) is Databricks’ architecture for running transactional (OLTP) and analytical (OLAP) workloads on one shared storage and governance foundation, with Lakebase as the transactional layer.

FAQ

Is Lakebase generally available?

Yes. Lakebase launched in Public Preview in June 2025 and is generally available on AWS (January 2026) and on Azure (March 2026).

Is Lakebase just Postgres?

It’s standard PostgreSQL under the hood, so your drivers and SQL work — but delivered serverless, with autoscaling, scale-to-zero, branching, and Unity Catalog governance built in.

Can my application write to a synced table?

Treat synced tables as read-only copies of lakehouse data, keep application writes in separate operational tables in Lakebase.

What does LTAP stand for?

Lake Transactional/Analytical Processing an architecture, not a product name and not a synonym for Lakebase or Genie.

Do I manage the Postgres server?

No. Lakebase is fully managed and serverless, Databricks provisions, patches, and scales the Postgres instance for you, including scaling to zero when it’s idle.

Related:

·What’s new in Genie One — September 2026

· What’s new in Unity Catalog — September 2026.

Sources:

·Lakebase (OLTP) overview

· LTAP overview

· Serve data with synced tables

· Register a Lakebase database in Unity Catalog

· Lakebase is now GA (blog).

· Watch: What Is LTAP? Lakebase + Genie (Databricks CTO)

· Databricks Answers: Platform

TL;DR. Yes, Databricks SQL supports native stored procedures via CREATE PROCEDURE and CALL, and they’re generally available. Procedures are governed Unity Catalog objects written in LANGUAGE SQL, with IN/OUT/INOUT parameters and a BEGIN … END body.

They require Databricks Runtime 17.0 or above and Unity Catalog (not the Hive metastore). Don’t confuse them with SQL scripting (BEGIN … END, variables, loops) that runs from Runtime 16.3+, persisted procedures need 17.0+.

Does Databricks support stored procedures?

For years the answer was “not really — use notebooks or SQL scripting.” That changed: Databricks SQL now has first-class, persisted stored procedures.

Databricks SQL supports native stored procedures as governed Unity Catalog objects, created with CREATE PROCEDURE and invoked with CALL, written in LANGUAGE SQL with IN, OUT, and INOUT parameters.

• Governed by Unity Catalog : a procedure is a securable object with grants, not a loose script.
• Parameters :IN, OUT, and INOUT, with optional DEFAULT values for IN.
• Security context : SQL SECURITY INVOKER or DEFINER controls whose privileges apply at run time.

What are the requirements?

Stored procedures require Databricks Runtime 17.0 or above and a Unity Catalog catalog; they are not supported on the Hive metastore.

• Runtime 17.0+ for persisted CREATE PROCEDURE.
• Unity Catalog only — procedures live in a UC catalog and schema.
• LANGUAGE SQL and exactly one SQL SECURITY clause are required.

How do I write and call a stored procedure?

Create it in a real UC schema (not the Hive metastore — it’ll just error), then call it. The one thing that trips people up: an OUT/INOUT argument has to be a variable, never a literal.

CREATE OR REPLACE PROCEDURE add_numbers(

IN p_a INT, IN p_b INT, OUT p_result INT)

LANGUAGE SQL SQL SECURITY INVOKER

AS BEGIN SET p_result = p_a + p_b; END;

Then invoke it, capturing the result in a session variable:

DECLARE answer INT;

CALL add_numbers(2, 3, answer);

SELECT answer; — 5

FAQ

Are Databricks stored procedures generally available?

Yes — CREATE PROCEDURE and CALL are GA in Databricks SQL. The GA label landed gradually, so if you don’t see them, check your runtime version before you blame your permissions.

What runtime do I need?

Databricks Runtime 17.0 or above for persisted stored procedures. SQL scripting (without a persisted procedure) runs from Runtime 16.3+.

Do stored procedures work on the Hive metastore?

No. They require Unity Catalog.

What languages can I use?

LANGUAGE SQL. The body is a BEGIN … END compound statement that can use SQL scripting constructs.

Related:

·How to Start Databricks Fast: Express Setup vs Free Edition

· What’s new in Unity Catalog — September 2026.

Sources:

·Introducing SQL stored procedures (blog)

·CREATE PROCEDURE reference

· CALL reference

· SQL scripting.

· Watch: How to Use Temporary Tables in Databricks SQL

· Databricks Answers: Databricks SQL

Last updated October 2026

BI Migration and Genie Code AI Authoring

• Enhanced BI Import (/importBI): Importing Tableau and Power BI reports via Genie Code now supports ranked table calculations and expanded parameterized calculations.
• Governed Metric View Preference: Genie Code prefers governed Unity Catalog metric views over ad-hoc SQL datasets during dashboard generation, reusing existing metric views whenever possible.
• SQL Parameters & Local Metric Conversions: Genie Code understands SQL dataset parameters and can convert standard SQL datasets directly into local metric views.

Data Modeling, Relationships and Metric Views

• Cross-Dataset Filtering via Relationships (Public Preview): Filters, parameters, and selections applied to one dataset automatically cascade to widgets built on connected datasets.
• Relationship-Scoped Filters (Public Preview): Widgets respond strictly to filters on related datasets, preventing unrelated cross-filtering interference.
• Metric View Enhancements: Supports include-style Level of Detail (LOD) expressions in the editor, and exports respect defined field display names.

Advanced visualizations and Styling

• Theme Import/Export & Previews: Export dashboard themes to JSON to share across dashboards or inspect color palette previews directly in the preset theme picker.
• Named Color Mappings: Map specific colors to field values by name across color-by-field and color-by-series configurations for workspace-wide consistency.
• Line Shape Control: Toggle between linear, smooth, and stepped lines on area and line charts.
• Geospatial Path Maps: Path maps can construct trajectory lines (e.g., GPS tracks) from point sequences using separate latitude/longitude columns or point geometry fields.
• Pivot Table Enhancements: Configure default expansion levels for row/column hierarchies, format values/headers of totals independently, and apply color scales to cell text in addition to background fills.

Interactivity, Filtering and Ask Genie

• Embedded Dashboard Filter APIs: Programmatically set and retrieve filter states in embedded dashboard applications.
• Granular Filter Overrides: Configure specific widgets to ignore selected filters while remaining active across the rest of the dashboard page.
• Contextual Ask Genie: Ask Genie uses your currently selected chart as context and displays scoped starter questions tailored to the active dashboard page and widget.
• Rich Text & Dynamic @ References: Embed live, formatted aggregated metrics (like @sum or @avg) into text widgets, titles, and descriptions that dynamically update based on active filters.

Operations & Subscriptions

• Data Attachments in Alerts: Scheduled dashboard subscriptions delivered to Slack and Microsoft Teams can now include CSV, TSV, or Excel data attachments.
• Target Warehouse for Materialized Views: Choose the specific SQL warehouse used when exporting dashboard datasets as Unity Catalog materialized views.
• Page-Level Key Insights: Automatically generates key insights per dashboard page, complete with a live generation indicator and a New Insights badge.

Key Takeaway: Transforms Databricks AI/BI Dashboards into a mature BI platform with automated Tableau/Power BI migration tools, rich custom formatting, and cross-dataset governance through native relationships.