Shamen·Nov 1Experimenting with Declarative AI Governance: One of the Best Ways to Govern Databricks Unity AI…How we combined declarative Team YAMLs, native Terraform, Unity Catalog ABAC, and a REST API bridge to automate multi-team LLM routing…A response icon1A response icon1
Shamen·Oct 23Benchmarking Databricks Agent Memory Pipelines: -96.8%An empirical experiment comparing Cold Sessions, Raw History Stuffing, and Distilled Agent Memory Pipelines using databricks-agentbricks…
Shamen·Oct 10Databricks Sandboxes: The Secure Playground for AI AgentsAs autonomous AI agents shift from structured chat responses to dynamically writing and executing code, a critical infrastructure challenge…
Shamen·Oct 2Breaking the Limits of Data Ingestion: An Event-Driven Databricks Architecture on AWSData engineering teams often wrestle with unpredictable data arrival patterns. When files land irregularly, relying on scheduled batch jobs…
Shamen·Sep 25Stress-Testing Spark Structured Streaming Defaults vs. Optimizations on DatabricksWe all know the feeling. You deploy a Spark Structured Streaming job, it runs smoothly for a week, and then data volumes spike. Suddenly…
Shamen·Sep 12I Built an AI-Powered Data Pipeline Generator for Databricks — Here Is What HappenedHow I turned repetitive Medallion architecture boilerplate into a 5-minute interactive AI skill that works directly with Databricks Genie…
Shamen·Sep 4Databricks Domains and Tags: Paving the Way for Genie OntologyAs data platforms scale, making data discoverable and understandable to the business becomes a top priority. Databricks introduced Domains…
Shamen·Aug 29Why Databricks Lakeflow Designer is a Game-Changer for Data PipelinesBuilding data pipelines shouldn’t feel like untangling a giant knot of code. If you spend your days architecting solutions on the…
Shamen·Aug 20How Omnigent Saved My AI Agents from Going RogueLet’s be real — after almost nine years of building cloud data platforms and distributed systems, I’ve written my fair share of boilerplate…
Shamen·Aug 15Automating Data Governance in Databricks: Dynamic Table Tagging with Govern Tag AutomationIn modern data architecture, maintaining a robust governance strategy across your data platform is non-negotiable. With Databricks Unity…
