Rahul Sounder·May 21Interoperability & Usability— Lakehouse Federation, Lakeflow Spark Declarative Pipelines (SDP)…Project 1 — “In one of the enterprise banking projects, we implemented Lakehouse Federation to solve a major challenge where customer and…
Rahul Sounder·Feb 23Confluent Kafka Batch/Real-Time Spark Data Streaming into Databricks Delta TablesEither we can create a Topic and we can create a Connector which inturn creates a Topic
Rahul Sounder·Feb 18Databricks CLI to have Access at DBX Account & Workspace LevelDatabricks CLI allows you to connect and interact with Databricks platform (account console and workspaces) from local terminal.
Rahul Sounder·Feb 18Accessing Azure Storage from Databricks with Access Connector in AzureThe storage location can be defined at 3 levels
Rahul Sounder·Feb 11Databricks Performance Efficiency — One of 7 PillarsThis image shows the complete Spark query optimization lifecycle, starting from the moment a user submits a query until Spark executes it…
Rahul Sounder·Feb 11Catalyst Optimizer in Apache SparkThe Catalyst Optimizer is the core engine in Spark SQL that automatically transforms high-level queries (SQL or DataFrame API) into the…
Rahul Sounder·Feb 8Databricks — Securing Data in Unity CatalogHow to Mask a Column with Function in Databricks DLT
Rahul Sounder·Jan 28Databricks Cluster Access Modes: Unrestricted vs SharedUnrestricted clusters break every security guarantee Unity Catalog is designed to enforce.
Rahul Sounder·Jan 27GCP Databricks Google BigQuery Connection (15)For Google Service Account Key JSON, Lets Create a Service Account in GCP IAM