<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Colin Van Dyke on Medium]]></title>
        <description><![CDATA[Stories by Colin Van Dyke on Medium]]></description>
        <link>https://medium.com/@colinvandyke?source=rss-2bc73782c13b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*DP8oHpSIx5ejZmKiWxOrqg.png</url>
            <title>Stories by Colin Van Dyke on Medium</title>
            <link>https://medium.com/@colinvandyke?source=rss-2bc73782c13b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 11:41:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@colinvandyke/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Getting Started with Mosaic AI Gateway]]></title>
            <link>https://medium.com/blue-orange-digital/getting-started-with-mosaic-ai-gateway-c8d2e467b425?source=rss-2bc73782c13b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8d2e467b425</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[databricks]]></category>
            <category><![CDATA[governance]]></category>
            <dc:creator><![CDATA[Colin Van Dyke]]></dc:creator>
            <pubDate>Thu, 21 Nov 2024 15:55:18 GMT</pubDate>
            <atom:updated>2024-11-21T15:55:18.429Z</atom:updated>
            <content:encoded><![CDATA[<p>With the ever-increasing speed of change within the AI space, organizations are clamoring for ways to apply governance and control over their use of AI models at scale. Databricks has been leading the charge in rolling out features that help organizations get a handle on managing their data and AI assets in a unified way using Unity Catalog.</p><p>Databricks has its pace of rapid innovation in this area through a new product they released this fall called Mosaic AI Gateway. This product is aimed at helping organizations secure their data and Gen AI deployments through centralized governance. Mosaic AI Gateway is helping its users apply controls and rules to their use of models inside and outside of Databricks, as well as providing a new level of visibility into usage and cost across all AI models.</p><p>So what is it exactly? Mosaic AI Gateway is a product that utilizes the excellent model serving endpoints that Databricks users have taken advantage of for a long time. It adds on the critical capability to integrate external foundation model providers, enforce usage, and enable governance by basically acting as a firewall for your AI models. There are <a href="https://docs.databricks.com/en/machine-learning/model-serving/score-foundation-models.html">many ways</a> to use models via the model serving endpoints, but most critically, for many developers, you can use the Open AI client directly. Once your models are configured in the gateway, you can simply re-target your client to call the model serving endpoint instead of the foundation model provider. This simple change will allow you to start taking advantage of a whole host of benefits provided by the new Mosaic AI Gateway product, including:</p><ul><li><strong>Permission and rate limiting</strong> to control who has access and how much access.</li><li><strong>Payload logging</strong> to monitor and audit data sent to model APIs using <a href="https://docs.databricks.com/en/machine-learning/model-serving/inference-tables.html#what-is">inference tables</a>.</li><li><strong>Usage tracking</strong> to monitor operational usage on endpoints and associated costs using <a href="https://docs.databricks.com/en/admin/system-tables/index.html">system tables</a>.</li><li><strong>AI Guardrails</strong> are used to prevent unwanted and unsafe data in requests and responses.</li><li><strong>Traffic routing</strong> to minimize production outages during and after deployment.</li></ul><p>Another great aspect of this new AI Gateway feature is that you don’t need to use all of the Databricks platforms to start using it and get the benefits of these features!</p><p>This centralization also gives you more robust control over credentials for foundation models. It eases the ability for your organization to adopt models from different providers, keeping you nimble and on top of the latest developments.</p><p>So let’s see how to really get this thing working.</p><h3>AI Gateway Setup</h3><p>The first step is to configure an external model serving endpoint: See the docs for more in-dept details: <a href="https://docs.databricks.com/en/machine-learning/model-serving/create-foundation-model-endpoints.html#ext-model-endpoint">https://docs.databricks.com/en/machine-learning/model-serving/create-foundation-model-endpoints.html#ext-model-endpoint</a></p><p>There are 3 options for setting up an endpoint — Using the UI, Rest API, MLFlow SDK. I’ll show the MLFlow SDK and the UI below. The API follows the same pattern as the SDK.</p><p>MLFlow SDK</p><p>Start by installing and initializing the SDK. Then you can call the create_endpoint method with your desired configuration</p><pre>%pip install mlflow[genai]&gt;=2.9.0</pre><pre>client = mlflow.deployments.get_deploy_client(&quot;databricks&quot;)<br><br><br>client.create_endpoint(<br>    name=&quot;openai-completions-endpoint&quot;,<br>    config={<br>        &quot;served_entities&quot;: [{<br>            &quot;name&quot;: &quot;openai-completions&quot;,<br>            &quot;external_model&quot;: {<br>                &quot;name&quot;: &quot;gpt-4o-mini&quot;,<br>                &quot;provider&quot;: &quot;openai&quot;,<br>                &quot;task&quot;: &quot;llm/v1/completions&quot;,<br>                &quot;openai_config&quot;: {<br>                    &quot;openai_api_key&quot;: &quot;{{secrets/my_openai_secret_scope/openai_api_key}}&quot;<br>                }<br>            }<br>        }],<br>        &quot;rate_limits&quot;: [<br>            {<br>                &quot;key&quot;: &quot;user&quot;,<br>                &quot;renewal_period&quot;: &quot;minute&quot;,<br>                &quot;calls&quot;: 20<br>            }<br>        ],<br>    }<br>)</pre><p>*<em>Make sure you set your secret scope first for your openai_api_key</em></p><h3>Model Serving UI</h3><p>Alternatively, you can use the UI to set up a new served entity — you have new options here for external models and a whole new section to configure your AI Guardrails</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KYZvXN_3tCLPOkUrCchSYg.gif" /><figcaption>Setup your served entity — OpenAI External model</figcaption></figure><p>*<em>Make sure you set your secret scope first for your openai_api_key or it won’t save</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P_0_lAspzScsWbDhSbNCdQ.png" /><figcaption>Configure your AI Gateway Guardrails</figcaption></figure><p>Some of the features you can enable with AI Gateway:</p><ul><li>Safety Filtering — using <a href="https://huggingface.co/meta-llama/Meta-Llama-Guard-2-8B#harm-taxonomy-and-policy">https://huggingface.co/meta-llama/Meta-Llama-Guard-2-8B#harm-taxonomy-and-policy</a></li><li>PII Detection — AI Gateway uses <a href="https://microsoft.github.io/presidio/supported_entities/">Presidio</a> to detect the PII (Currently this is US focused and detects the following: credit card numbers, email addresses, phone numbers, bank account numbers, and social security numbers)</li><li>Topic Moderation: list or upload a set of allowed topics. Given a chat request, this guardrail flags the request if its topic is not in the allowed topics.</li><li>Keyword Filtering: specify different sets of invalid keywords for both the input and the output. One potential use case for keyword filtering is so the model does not talk about competitors or their products.</li><li>Allow you to AB test different models transparently to your users and monitor the performance via inference tables</li><li>Test new model version before deploying them to end users.</li><li>Rate Limiting: Put limits on a per user or application from a centralized place — control who has access, and how much access.</li><li>Inference Table logging: This features logs all requests and responses to the endpoint into a table in Unity. Make sure you setup this up, it will allow you to see how your endpoint is being used in great detail and create dashboards and reports easily.</li></ul><h3>Endpoint in Action</h3><p>I set up my endpoint to limit the input topics to food and beverages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K3DNjenYW0XYpEVK8F1kuQ.png" /></figure><p>Now that we have our model serving endpoint setup and guardrails configured, let’s see what it looks like to use this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zNSTZjLVSngPyyrDDWPVmw.png" /></figure><p>When you hit one of the guardrails, your request will raise an exception; the endpoint currently returns a 500 response code with no additional information. I hope this feature gets updated to provide more details in the future.</p><p>The gateway also enforces the guardrails with using it via SQL — here’s what that looks like when you go outside of the allowed topics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2MKvqrcft5o0nml2N3xwCw.png" /></figure><h3>Gateway Dashboard</h3><p>Like I mentioned above, if you setup an inference table for your endpoint, it will log all the request details to it and you can create really useful dashboards. Databricks has created an <a href="https://github.com/mingyu89/ai-gateway/blob/main/%5BExample%5D%20AI%20Gateway%20Dashboard.lvdash.json">Example Dashboard</a> you can import to your workspace to get started.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oLSmyjV9PBw2t-_oY8YBIg.png" /><figcaption>AI Gateway Dashboard</figcaption></figure><p>Out of the box we get token counts for all the different endpoints, request amounts, response times, response codes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L0TXQzFJkfiWS7_UWWWqbw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A9M5Kg_rueqNxqqujFbpRw.png" /></figure><p>When using these model serving endpoints, you now have the ability to send along more <a href="https://learn.microsoft.com/en-us/azure/databricks/ai-gateway/configure-ai-gateway-endpoints#further-define-usage-with-usage_context">usage context with each request</a>. Use this feature to include extra details, like project, application, or other identifiers that could make splitting out this data more useful to your organization. These customizations would make it easy to fit a chargeback model to the usage of all AI models that the AI Gateway centrally governs. This is something that a lot of organizations would like to implement with the ever-growing demand for AI model usage!</p><h3>Conclusion</h3><p>Mosaic AI is a great addition to the Databricks line of products. It solves a real problem that organizations are seeing today of having a centralized way to govern the use of AI models at scale at a time when they are getting more and more pressure to do so.</p><p>If you’re ready to see how Databricks can elevate your data strategy, <a href="https://blueorange.digital/contact-us/">reach out to our team</a>. Blue Orange specializes in helping organizations like yours maximize the power of Databricks and take advantage of all the next-generation capabilities they are delivering!</p><p>Further Reading:</p><ul><li><a href="https://learn.microsoft.com/en-us/azure/databricks/ai-gateway/">AI Gateway Docs</a></li><li><a href="https://www.databricks.com/product/ai-gateway">Mosaic AI Gateway Product Page</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8d2e467b425" width="1" height="1" alt=""><hr><p><a href="https://medium.com/blue-orange-digital/getting-started-with-mosaic-ai-gateway-c8d2e467b425">Getting Started with Mosaic AI Gateway</a> was originally published in <a href="https://medium.com/blue-orange-digital">Blue Orange Digital</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Databricks + Fabric: Better Together]]></title>
            <link>https://medium.com/blue-orange-digital/databricks-fabric-better-together-5d386495acd0?source=rss-2bc73782c13b------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d386495acd0</guid>
            <category><![CDATA[microsoft-azure]]></category>
            <category><![CDATA[databricks]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-architecture]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <dc:creator><![CDATA[Colin Van Dyke]]></dc:creator>
            <pubDate>Wed, 29 May 2024 21:08:06 GMT</pubDate>
            <atom:updated>2024-09-12T13:40:28.763Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Databricks + Fabric — Better Together" src="https://cdn-images-1.medium.com/max/960/1*YB36Sj5EJZM25JKmp5IKQQ.png" /></figure><p>Organizations continuously seek robust, flexible, and scalable solutions to handle their diverse data needs in today&#39;s rapidly evolving data landscape. Databricks and Microsoft Fabric have emerged as leading platforms, each with strengths and target personas. While they share some similar technical features, their unique approaches to data management make them highly complementary services. This article explores how Databricks and Microsoft Fabric can be used together to create a powerful, integrated data ecosystem, highlighting the benefits and synergies of leveraging both platforms.</p><h4><strong>Overview of Databricks</strong></h4><p>Databricks is the inventor of the Lakehouse architecture, an open, unified foundation for all your data. Databricks provides a Data Intelligence Platform that unifies data, AI, and governance in one platform and architecture, enabling organizations to execute their data engineering, data analytics, and AI initiatives on a single powerful, open, secure platform.</p><p>The core components of the Databricks platform are seen below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yH79InALvrNgjvERnsuDvA.png" /></figure><p>Databricks is built on top of Open-Source technology and has been a pioneer in the data processing, machine learning, and open data format space since its inception. Spark, the core distributed data processing technology, has been the de facto standard for almost a decade. This has powered the platform&#39;s core and allowed data engineers and data scientists to easily scale their workloads and do amazing things with data.</p><h4><strong>Overview of Microsoft Fabric</strong></h4><p>Microsoft Fabric is an end-to-end analytics and data platform designed for enterprises that require a unified solution. It encompasses data movement, processing, ingestion, transformation, real-time event routing, and report building. It offers a comprehensive suite of services, including Data Engineering, Data Factory, Data Science, Real-Time Analytics, Data Warehouse, and Databases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zf-BOHsc3MQ65zOXzqnZhw.png" /></figure><p>The Fabric solution is built to be user-friendly, which enables greater accessibility to these powerful data technologies for less technical users. It enables organizations with citizen data practitioners to build data products using the same underlying technologies that companies with more advanced data teams have taken advantage of for a while. Built on top of the Lakehouse patterns in the form of Azure OneLake in the open Delta Lake format, it stitches together different experiences tailored for data engineers, analysts, data scientists, and others to provide them with a paired-down set of tools and options to allow them to achieve their respective goals.</p><h4>Databricks + Fabric are aligned and complementary services</h4><p>On the surface, it seems like there are a lot of overlapping capabilities between Fabric and Databricks, and that would not be wrong. However, these individual capabilities are not necessarily unique to these two services. Many platforms offer similar suites of services and capabilities, including others within the Microsoft ecosystem (see Synapse). Its more important to look at how the entire architecture and integration is converging in a way that serves the ultimate goal of Databricks to build a truly open and interoperable platform.</p><p>Unlike other cloud Databricks deployments, <strong>Azure Databricks</strong> is a native, first-party interoperable Azure service. The partnership between Microsoft and Databricks allows them to build deeper and tighter integration into their services and roadmaps. You can see this in the choices Microsoft has made when designing the core components of Fabric. A foundational component of the Fabric ecosystem is the Lakehouse. They’ve made it dead simple to create a Lakehouse with a few clicks. This aligns with Databricks and their central argument that a Lakehouse is the best Data Warehouse and the best way to build a future-proof data platform.</p><p>One important choice from Microsoft is that they chose to standardize their Lakehouse to use Delta Lake, the storage framework &amp; format pioneered and open-sourced by Databricks. There are a few popular competing options in this space today, between Hudi, Iceberg, and Delta, and there are a lot of benefits in some healthy competition in this space — like pushing each other to improve performance and add more features to get parity with traditional cloud data warehouses. I believe that, ultimately, there will not be a winner in this race and that open interoperability between these formats and frameworks will be the way of the future. That being said, I don’t think a format other than Delta Lake would make sense for Microsoft Fabric, given their long and public partnership with Databricks.</p><p>Another key integration point is that Databricks integrates with Power BI through native connectors. The seamlessness of the integration can be seen in how easily Azure Databricks allows you to publish and start using data in Power BI and the Fabric experience directly from the data explorer — <a href="https://youtu.be/ZpL-ews6b04?si=HURS6Fyn_N9e5Qal&amp;t=229">As demonstrated by Databricks CEO Ali Ghodsi here</a>.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FZpL-ews6b04%3Fstart%3D228%26feature%3Doembed%26start%3D228&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DZpL-ews6b04&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FZpL-ews6b04%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/6c9ae006126b82788ee166006b2f07aa/href">https://medium.com/media/6c9ae006126b82788ee166006b2f07aa/href</a></iframe><p>Since Azure Databricks is a first-party service co-developed by Databricks and Microsoft, it already natively integrates with many core services within Azure, many of which are already key components of the Fabric experience:</p><ul><li>Azure Active Directory</li><li>ADLS</li><li>Azure Data Factory</li><li>Azure Synapse Analytics</li><li>Power BI</li><li>Azure DevOps</li><li>Azure Virtual Network</li><li>Azure Event Hubs</li><li>Azure Key Vault</li><li>Azure Confidential Computing</li></ul><p>By integrating these two, you can take advantage of Databricks best in breed ML and AI capabilities into the Fabric ecosystem. MLFlow and Mosaic ML are critical components of the AI lifecycle that are very mature in the Databricks ecosystem but nascent in the Fabric one. Its not an area that is highlighted much, So I think there will be plenty of development between how these products will be used together. But as of today, using Databricks’ ML and AI Capabilities is currently the best way to build, deploy, and manage production scale workloads within this ecosystem.</p><p>Today you can connect data from OneLake into Databricks and vice versa in a few different ways, but it&#39;s not as seamless as it could be and not what I would want from a production-grade integration. They currently skirt the best practices for governance through Unity Catalog, so I wouldn’t recommend them if you are reliant on Unity.</p><p>These decisions and integrations signal that we will see more native capabilities coming quickly. I think a native Databricks Unity Catalog integration with OneLake, Fabrics&#39; data foundation, would be a critically important addition, enabling more robust use of Unity Catalog’s great governance features outside the native Databricks tooling and experience. Keep a close eye on these two services as they evolve together!</p><h4><strong>Benefits of Using Both Together</strong></h4><p>Building a foundation on Lakehouse allows your data estate to be well-managed by integrating these platforms&#39; fine-grained RBAC and ABAC capabilities. It also allows your solid foundation to enable you to take on AI and more advanced analytics use cases as your organization matures its data capabilities. Given their complementary nature, using these products together can give you the following benefits:</p><ul><li>Improved data governance and security</li><li>Increased productivity and collaboration</li><li>Cost efficiency and optimized resource utilization</li><li>Secure Data intelligence — same fine-grained RBAC &amp; ABAC</li><li>An open &amp; extensible platform</li></ul><p>Enhanced flexibility and scalability — Fabric and Databricks are targeted at different parts of the data market — if you carefully design a solution to integrate both, you can enable your less technical data practitioners to start building on top of a Lakehouse foundation through Fabric at the same time as your more technical data teams are executing against the robust platform Databricks provides with more fine-grained control over scaling workloads and using DevOps best practices. This can increase productivity and collaboration across these sets of users by allowing each group to use the tools that best suit their needs.</p><p>Many data leaders think about products when considering their data platforms; instead, they should rethink their strategy to be centered around an open Lakehouse architecture that enables all use cases and is durable for the long run!</p><h4><strong>Learn More</strong></h4><p>If you want to discuss Databricks, Fabric, or any other data-related topic, please contact me on <a href="https://www.linkedin.com/in/colinvandyke/">LinkedIn</a>. To learn more about Blue Orange Digital, You Strategic Data Partner — <a href="https://blueorange.digital/">check our website</a> and <a href="https://blueorange.digital/blog/databricks-fabric-better-together/">more details</a>. I would love to discuss your Data Strategy and how these ideas can help evolve it!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d386495acd0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/blue-orange-digital/databricks-fabric-better-together-5d386495acd0">Databricks + Fabric: Better Together</a> was originally published in <a href="https://medium.com/blue-orange-digital">Blue Orange Digital</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>