<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Alipouw on Medium]]></title>
        <description><![CDATA[Stories by Alipouw on Medium]]></description>
        <link>https://medium.com/@alipouw?source=rss-96f4fa37ea38------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*3RNueaVgWztaCZtg1Qefqw.jpeg</url>
            <title>Stories by Alipouw on Medium</title>
            <link>https://medium.com/@alipouw?source=rss-96f4fa37ea38------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 14:03:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@alipouw/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Security through agentic development]]></title>
            <link>https://medium.com/@alipouw/security-through-agentic-development-8e7654f1fbef?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e7654f1fbef</guid>
            <category><![CDATA[microsoft]]></category>
            <category><![CDATA[agentic-development]]></category>
            <category><![CDATA[ai-security]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Fri, 18 Sep 2026 20:14:28 GMT</pubDate>
            <atom:updated>2026-09-18T20:14:28.223Z</atom:updated>
            <content:encoded><![CDATA[<p>A common conversation these days is around security and governance of agents. How do I ensure trust with my security teams as agents proliferate across my organization? How can I control what these agents have access to, who has access to the agents? How do I ensure my agents are operating within the bounds defined for them to operate in?</p><p>This article is a continuation of my <a href="https://medium.com/@alipouw/from-copilot-to-capability-turning-ai-adoption-into-innovation-37d529c7b6cb">last article</a> going over driving value from the AI tools at your fingertips. This article comes at a critical time too with Jacob Coxon, former Anthropic researcher, who <a href="https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-out-of-control-ai-fears-707b7628">quit over safety concerns</a> in the race to the best self-improving systems. It further emphasizes the importance of taking a step back to slow innovation before it gets out of control.</p><h3>Security, reliability, governance, and trust</h3><p>There are multiple angles to look at security throughout and surrounding the SDLC process. In my previous article I reference the Stanford <a href="https://hai.stanford.edu/assets/files/ai_index_report_2026.pdf">AI index report</a>. The report states both a 26% gain in completed pull requests for developers using GitHub Copilot and a 55% rise of documented AI incidents. The productivity gain is build-time. The incidents are a run-time. Most organizations have invested in the former and are just now scrambling to mitigate the latter.</p><p>To take a wholistic view on security across the stack, I break it up in the following ways:</p><ol><li><strong>Design-time:</strong> design a threat model and use the Microsoft Cloud Adoption AI Framework (<a href="https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai/secure">CAF</a>) as the reference guide throughout build and runtime processes.</li><li><strong>Build-time: </strong>these are the security control throughout the SDLC — think code or secret scanning and evaluation gates.</li><li><strong>Run-time: </strong>any threat defined while AI is answering/retrieving on behalf of a user — think App gateways, keyless authentication, moderation, injection detection, token limits, circuit breakers, continuous evaluations.</li><li><strong>The agent harness: </strong>this is basically the scaffolding around your llm to bring more context and tools to your model. This is owned by you, as the enterprise, not the model provider.</li><li><strong>Identity, access &amp; monitoring: </strong>this is all about how you are monitoring and restricting agent access. Think: Agent identity, least privilege access, runtime monitoring, private networking, and telemetry.</li><li><strong>Data security &amp; governance: </strong>this speaks directly to agentic governance, which involves visibility and policy application to agent access. Information protection, DLP, agent/model governance, data governance.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9YoSSnCB9wlEyVlq6uWn2A.png" /><figcaption>Security in AI</figcaption></figure><p>The first three items above are phases as they happen in sequence. The last 3 are layers that cut across all 3 phases. Usually, it’s in the layers where organizations have the most security gaps.</p><h3>Design time</h3><p>Start with the threat model you already have and add to it. The Microsoft CAF AI guidance helps get your organization ready for AI adoption. Consider that AI not only adds to your threat models but significantly moves and expands the trust boundaries. You must consider how a model, now interacting with existing documents or tools, changes this once trusted input, to untrusted. Looking at the security section specifically, we want to ensure the following:</p><ul><li>Start with an established framework such as STRIDE, then reference AI-specific sources (like <a href="https://atlas.mitre.org/">MITRE ATLAS</a> and <a href="https://genai.owasp.org/">OWASP Generative AI risk</a>) to confirm coverage</li><li>Prioritize and classify by consequence such as blast radius or data sensitivity, rather than category of risk, like operational</li><li>Map agent identities, tool calls, memory systems and reteival soures — anything the agents and LLMs interact with</li><li>Define human-in-the-loop decision points to carry throughout your design and build process, to ensure nothing gets intentionally dropped after a successful demo sans human intervention</li><li>With increase AI workflows, this builds a case for additional insider risk and data exfiltration exposure — assess risk exposure to help define your investment in risk management tools such as Purview insider Risk Management</li></ul><h3>Build time</h3><p>At build, look to focus on evaluations, secret scanning and approval gates. We saw the need to this with the <a href="https://openai.com/index/hugging-face-incident-and-the-road-ahead/">Hugging Face incident</a>: when Open AIs internal research prototype broke out of its isolated evaluation sandbox environment and reached Hugging Face’s infrastructure. Following this Anthropic then review many of its own evaluations and found 3 cases of Claude reaching real system’s from a test environment and <a href="https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals">gained unauthorized access to other organization’s systems</a>.</p><p>In both cases above, the model or agent isn’t evil, but it’s relentless in the pursuit of its objective and its environment allowed it to escape and achieve its goal beyond what was intended. So, how do you lock down your environment to prevent these types of things:</p><ul><li>Review your threat model and ensure alignment throughout your build</li><li>Restrict outbound network access from agent build enviroments — GitHub supports a configurable forewall allowlist as an example</li><li>Give agents least-privilege access with short-living credentials</li><li>Leverage evaluations like code vulnerability, groundedness, hatefullness and unfairness, or agent task completion</li><li>Ensure secret scanning, push protection and branch protection on all PRs</li><li>Add automated fixes, such as <a href="https://docs.github.com/en/code-security/responsible-use/security-and-quality-ai-features">Copilot autofix</a>, and add security test gates</li></ul><h3>Run time</h3><p>Evaluations, such as red-team, blue-team and green-team agents (I know this sounds like a video game, but it’s a real multi-agent defense model), monitoring, keyless authentication, and API gateways are key here. Equally as important as build time, if not more, as this is in a production scenario with real users.</p><p>Keep the following in mind at runtime:</p><ul><li>Centralize model and tool access trough API gateways — this facilitates policy enforcement to improve control, implement token limits, ensure broad monitoring and implement access restriction</li><li>As Microsoft states in our <a href="https://learn.microsoft.com/en-us/security/agentic-security/agentic-security-agent-categories">documentation</a>: “Red team agents find and expose gaps. Blue team agents detect, triage, and investigate. Green team agents remediate and harden your posture.”</li><li>Use key-less authentication to remove stored secrets. Leverage managed identity with Entra authentication and defined role groups</li><li>Add circuit breakers and load balancing to improve responsiveness and reduce blast radius</li><li>Leverage continuous evals to detect harmful activity and groundedness of each prompt and responses</li><li>Use prompt-shields to help detect prompt injections or jailbreak attempts</li></ul><h3>The agent harness</h3><p>As mentioned above, think of the agent harness as all the bells and whistles surronding the llm that tun the model into an agent. It decides what the models is allowed to do with every decision. What’s important to decide on with the harness is as follows:</p><ul><li><strong>Tool and MCP server allowlists:</strong> which tools are allowed and which version? How do we govern MCP enpoints to avoin non-visible tool sprawl?</li><li><strong>Per-tool credentials: </strong>ensure all tools have authentication gates and require agent credentials for full access</li><li><strong>Deterministic gates:</strong> code checks to ensire policies are properly enforced</li><li><strong>Memory and session isolation: </strong>with persistent memory, this opens up addtional poisoning threats, which need to be managed and controleld</li><li><strong>The agent kill-switch: </strong>the ability to disable an agent or particular tool without killing the entire application</li></ul><h3>Identity, access &amp; monitoring</h3><p>Core to ensuring agents act in the premission bounds defined, is the agent identy. With this you can assign access and monitor access patterns of the agent to control it’s blast radius and prevent unauthorized access. Adding to this, the agent identity gives you informaiton on who’s acting on which task, but not if they’re allowed to do that task or not. This brings in the <em>authorization </em>component. Look to distinguish between 3 modes of auth: the agent acting on its own behalf, on the behalf of the user and as a workload. If its acting on a user’s behlf, make sure its access never exceed that of the user.</p><p>Layer these with Information Protection and DLP features and you help further restrict data access to a defined set of task and resources. I’ll go into this more in the next section on data security and governance.</p><p>Monitoring tools such as an API Gateway or Agent registry privide visibility into the agent (and human) interactions across your estate. Microsoft provides tools such as APIM and Agent 365 to help accomplish this.</p><h3>Data security &amp; governance</h3><p>Data security isn’t only encryption at rest and in-transit, but also the access model for your data. Controling access to your data plane allows you to control who has access to which data. You want to lead with principle of least privilege and think about your data in layers. At the smallest level you have Row-level security, and column-level security. This can be expanded to object-level security and then further to folder/database-level security. Think about what data your agents need access to and start at the most granular and work your way out. This helps to reduce overprovisioning of access.</p><p>Ontop of the data access, we want to prevent against data loss and protect sensitive data even further. This is where Information protection, Insider risk management and data loss prevention services shine. Ensure you classify/label all of your data and leverage tools such as Microsft Purview to monitor and enforce access restrictions.</p><p>These are two different layers doing two different jobs. Your data platform needs to enforce who can read which objects, rows and columns. Purview classifies the data, applies sensitivity labels, and prevents it from leaking once an agent has legitimately retrieved it. You need both.</p><h3>Wrap-up</h3><p>As we see from the AI adoption report, adoption is no longer a differentiator in the market. Adoption is 88% but the gap is between adoption and deployment which remains in the single digits. This is where security and trust come in.</p><p>So…</p><p><strong>How do you build trust with security teams?</strong> Show controls throughout the build and run process — across then entire lifeceycle. Include AI in your threat model, run continuous evaluations, account for data and network security, implement code and secret scanning.</p><p><strong>How do I control what my agents reach? </strong>Follow least privilege access, action-level authorization, and ensure each agent has its own identify.</p><p><strong>How do I keep them in bounds? </strong>Define deterministic gates agents cannot reason past, and define a bounded tool allowlist.</p><p>As a reference, I put together a more wholistic diagram showcasing how the Microsoft stack covers security, reliability and governance across the entire SDLC and application lifecycle. This helps when designing your threat model, shows you how to leverage and secure AI at each part of the cycle, and serves as a guide to create trust in the agentic applications that you build.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RBIJ9eXxAufo24LRf_EvSw.png" /><figcaption>Security throughout and surrounding the end-to-end SDLC</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e7654f1fbef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Copilot to capability: turning AI adoption into innovation]]></title>
            <link>https://medium.com/@alipouw/from-copilot-to-capability-turning-ai-adoption-into-innovation-37d529c7b6cb?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/37d529c7b6cb</guid>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[microsoft-ai]]></category>
            <category><![CDATA[github-copilot]]></category>
            <category><![CDATA[personal-assistant]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Tue, 08 Sep 2026 14:13:29 GMT</pubDate>
            <atom:updated>2026-09-08T14:13:29.338Z</atom:updated>
            <content:encoded><![CDATA[<p>Stanford recently came out with their <a href="https://hai.stanford.edu/assets/files/ai_index_report_2026.pdf">2026 AI index report</a> showcasing the vast opportunity still remaining in the market to capitalize on, despite AI scaling “faster than the systems around it can adapt.” Five core signals stood out to me in this year’s report:</p><ol><li>AI has reached an adoption threshold, not a value threshold</li><li>Productivity gains are now real, measurable and predictable</li><li>Competition shifts from model access to execution quality</li><li>Organizational transformation requires intentional skilling, not just tool distribution</li><li>Capability is surpassing security, governance, reliability &amp; trust</li></ol><p>This aligns to what I see in the industry every day. Change is constant. Now how do you get ahead before you get behind?</p><p>In this article, I’ll touch on signals 1 to 4 using a practical example: how I built a team of agents to do my own job more efficiently and better, what it cost, and what I’d suggest doing differently. Hopefully this will provide some insight and skilling on how to best leverage the AI tools at our disposal to drive value internally and for your customers, improve execution quality, and drive real productivity gains.</p><p>Signal 5 deserves more than just a section. In the next article I’ll going to go over some security best practices to consider across the Data &amp; AI stack, specifically when you’re creating agents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/957/1*1KlpG0wt-YrEp-7BtoJJPw.png" /><figcaption>An example E2E workflow of the agents I execute on a daily basis</figcaption></figure><p>Here is the <a href="https://github.com/alipouw13/sales-agent-foundation/tree/main">repo</a> for the practical use case that I will be going over in this article.</p><h3>GitHub</h3><p>For those who don&#39;t know, GitHub is the largest and most well-known open-source platform on the market, and AI coding and security are core to it. Roughly <a href="https://www.ainvest.com/news/github-copilot-90-fortune-100-adoption-proves-software-infrastructure-product-2605/">90% of Fortune 100 companies</a> use GitHub Copilot (GHCP), which gives users access to almost all frontier models from multiple providers.</p><p>The honest productivity gains vary enormously by task and how the tool is used. As an example, I’ve had some customers of mine see a 30–70-minute productivity gain/day. This variance points directly to signal 4: transformation requires proper skilling, not just the proper tools. The organizations who see the best productivity gains are those who have taken the time to properly skill their workforce.</p><p>Handing someone a copilot license is much different than handing them an end-to-end working system. The rest of the article will go into the latter — how to leverage GHCP as a core tool in your everyday workflow to drive value and productivity gains.</p><h3>The use case</h3><p>Everything starts with a use case: what problem am I trying to solve and what will determine success?</p><p>I am a Solution Engineer at Microsoft. I am responsible for pre-sales technical conversations across 10+ different Microsoft products, and I’m expected to have a deep technical understanding of each of the products I sell. I build demos to showcase our products, understand the broader Microsoft ecosystem, drive sales strategy in partnership with the account team, and ultimately get the technical win. So how can I do this all at once, in a world where the tech keeps changing and the job description keeps expanding?</p><p>So, the question I set out to answer was:</p><p><em>As a Solution engineer or seller, how can I build a scalable team of agents, skills, and tools to measurably increase my productivity and throughput, and shorten sales cycles?</em></p><h3>What am I actually measuring</h3><p>This matters when it comes to proving to leadership what the AI ROI actually is. A productivity claim with no substance behind it is just a vibe check. Per signal 2, we need the gains to be measurable.</p><ol><li><strong>Time to produce an account brief:</strong> 4 hours<strong> </strong>-&gt; 30 minutes (8x time save)</li><li><strong>Time to draft a customer deck: </strong>4 hours -&gt; 1 hour (4x time save)</li><li><strong>Time to build a Fabric demo: </strong>40 hours -&gt; 4 hours (10x time saved)</li><li><strong>Time to create an architecture diagram: </strong>4 hours -&gt; 30 minutes (8x time saved)</li><li><strong>Time to create an opportunity: </strong>30 minutes -&gt; 5 minutes (6x time saved)</li><li><strong>Prep time before customer meetings: </strong>2 hours -&gt; 15 minutes (8x time saved)</li><li><strong>Rework rate: </strong>1 hour -&gt; 10 minutes (6x time saved)</li></ol><p>The last item above, rework rate, is super important. Even if an agent creates a draft in 10 minutes but it costs you another 50 minutes to iterate upon, then did it really save you time? Without agents there is always rework after you go over your ops/decks/architectures with others — does the agent rework with you as the review increase or decrease the amount of time spent on task?</p><h3>Prerequisites</h3><p>In order to create my minions, I needed the following setup. I’ve identified items required for anyone building their team of personal assistants and additional tools I used for my specific role.</p><p><strong>Required for anyone</strong></p><ol><li>VS Code: to run GitHub Copilot</li><li>GitHub Copilot: what develops and runs the agents</li><li>Connection to a CRM: information about your customers and your book comes from here</li><li>Access to your role description or comp plan: to understand what you do and how your measured</li></ol><p><strong>Required for my stack — yours may differ</strong></p><ol><li><strong>Work IQ:</strong> core to understanding your day-to-day interactions in the M365 suite</li><li><strong>Microsoft Fabric MCP: </strong>to understand the ins-and-outs of and interacts with Microsoft Fabric</li><li><strong>Microsoft Learn MCP server: </strong>to understand all things Microsoft related and best practices</li></ol><p>In the repo I shared, you are able to configure your <em>logical </em>sources<em> </em>like <em>crm, workplace, notes, web and decks</em> in the config file. This removes the need for agents to name a specific vendor tool (like Dynamics for the Microsoft <em>crm</em>), just like calling a variable set. Anything you define as unavailable the agent will report as missing instead of hallucinating and inventing around it.</p><h3>The implementation</h3><p><strong>Step 1: The core 3 building blocks</strong></p><p>Before prompting, I wanted to go over the 3 core building blocks in your repository <em>.github </em>folder that make up your agent workforce.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/1*fTlIBhc9zj1yr1tZZY1E1A.png" /><figcaption>Agent workforce components</figcaption></figure><p>People sometimes skip skills but these are critical in ensuring agents produce trustworthy responses. Anything two agents might need to agree upon is in a skill. Anything one agent needs, stays in that agent — just like how functions work in a python script.</p><p><strong>Step 2: Ask GHCP for a prompt and spec</strong></p><p>After ensuring I had my prereqs nailed, I started by asking Copilot what it thought I needed based on what it could already see about my work. I sent the following prompt to an empty repo:</p><p><em>Based on my role guide and what you understand I do on a day-to-day basis, give me a prompt to give to you to create a spec to build out an agent workforce with relevant skills to assist me with reoccurring tasks, keeping security top of mind.</em></p><p>If you take one thing from the article it should be the prompt above and it’s structure. You are not a prompt engineer and don’t have to be with the tools these days. With the proper tools GHCP has more context on what you do day-to-day than what you can easily put in a context window. Asking it to draft a spec first surfaces activities and workflow you could have forgotten and provides you to opportunity to add to the spec before building the solution.</p><p>Once this runs, GHCP will provide you with a prompt to run to generate a spec to build your agents.</p><p><strong>Step 3: Generate the spec with guardrails</strong></p><p>After the spec is generated, I review and add to it as needed and then put my Copilot on autopilot to build out the workforce. The two constraints I add before letting the build go:</p><ul><li><strong>Generate the file structure before generating the agents: </strong>Every agent has the same six sections in the same order. Every prompt names its agent. Every skill name matches its directory.</li><li><strong>Prompt Copilot</strong> to ‘<em>Span multiple specialized agents in parallel, and to run validation loops to ensure accuracy, security, and consistency in the build process</em>.’</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x1dXp6tiIgVWPWAd4U5piA.png" /><figcaption>The verification loop when creating your agents</figcaption></figure><p>Because I created the repo for multiple personas, I had a few more roles beyond my own (6 total) that are covered by the agents and skills below:</p><ul><li><strong>25 agents</strong>: 5 SDLC persons (spec, plan, build, validate, review) and 20 revenue workflows</li><li><strong>20 prompts</strong>: reusable prompts for different repeatable workflows</li><li><strong>10 skills</strong>: shared ‘contacts’</li><li><strong>6 role playbooks</strong>: what good looks like for each persona</li></ul><h3>The economics</h3><p>Generating the 20 agents is cheap and is getting cheaper. Generating twenty <strong><em>verified </em></strong>agents that agree with one another and work together is not, and that is the cost conversation that I rarely have but should be having more and more. The expensive failure is not a simple wasted API call. It is an agent that confidently hands you a number it made up, which you then put in front of a customer.</p><p>In this sense, economics have shifted. The question is no longer “how fast can I create this?” Rather, it’s:</p><ul><li>How cheaply can I verify it?</li><li>How quickly can I tell that output is wrong?</li><li>What does it cost me when I cannot tell?</li></ul><p>Now based on this, bringing us back to implementing valuable agentic solutions, the three things design decisions I would always repeat are:</p><ol><li><strong>Make agents cite everything:</strong> this ensures that verifying a claim takes thirty seconds instead of thirty minutes. Verification cost is a large constraint, so drive it toward zero.</li><li><strong>Put shared tasks/procedures in skills:</strong> 2 agents that score the same thing differently produce output you can’t combine. You need to ensure consistency to use the output with confidence.</li><li><strong>Automate easy structural identifiers: </strong>as an example, the repo has a validator that runs in CI on every push and enforces sixteen structural checks like section order, agent and prompt parity, dead links, or common credential patterns. It has no dependencies and it runs in 9 seconds.</li></ol><p>The structural identifiers are only part of it. Running a green build tells you the structure is intact but doesn’t flag real customer names or phone numbers or revenue. This is where we need to bring in proper security guardrails across the stack, which is the bridge to signal 5 and my next article.</p><h3>Next steps for you</h3><p>Try this for your organization: after establishing out prereqs, start by cloning this <a href="https://github.com/alipouw13/sales-agent-foundation/tree/main">repo</a>.</p><ol><li>Open it in VS Code and ask copilot which agents it can see</li><li>Copy the config/profile.example.md to config/profile.md (from the root directory run <em>cd config &amp;&amp; cp profile.example.md profile.md</em>) and fill it in and connect your tools</li><li>Read the role (<em>docs/roles/) </em>that best maps to yours and review which agents it suggests starting with</li><li>Run an agent against an account you know best to determine if it works</li><li>Read its <em>Gaps</em> line to see where there are gaps and try to fill them</li></ol><p>Let me know if this was helpful or you have any questions. Look out for the security article coming out soon!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37d529c7b6cb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building enterprise-grade production applications using Fabric Rayfin]]></title>
            <link>https://medium.com/@alipouw/building-enterprise-grade-production-applications-using-fabric-rayfin-05731fdbb120?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/05731fdbb120</guid>
            <category><![CDATA[app-development]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <category><![CDATA[fabric-rayfin]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Mon, 27 Jul 2026 15:18:49 GMT</pubDate>
            <atom:updated>2026-07-27T15:18:49.776Z</atom:updated>
            <content:encoded><![CDATA[<p>After a long hiatus of Iceland travel (pictured below) and Microsoft end of fiscal year craziness, we are back with an article on one of Microsoft Fabric’s newest announcements in partnership with Replit, <a href="https://github.com/microsoft/rayfin">Rayfin</a>. Rayfin is a Backend-as-a-Service (BaaS) open-source SDK and CLI where developers (and coding agents) define and deploy a complete application backend in code, then run it on Microsoft Fabric with enterprise governance, security, and authentication built in. By the end of this article you’ll know what Rayfin is, how to set up the prerequisites, and how I used it to build a Fabric report-optimization app from a couple of prompts. More details <a href="https://community.fabric.microsoft.com/t5/Fabric-Updates-Blog/Introducing-Rayfin-A-new-AI-first-way-to-build-deploy-and-govern/ba-p/5191676">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9RBVBOHZSxLxgwo9iOJNzg.jpeg" /><figcaption>Iceland</figcaption></figure><p>Your business users love Power BI so why not build applications backed by enterprise security right in their flow of work? Rayfin brings quick application development, enterprise security and architecture together within a familiar work environment for your end users. Developers can build in the familiar Replit environment and deploy straight into a securely managed Fabric tenant, with OneLake storage and platform governance inherited by default.</p><p>I tried Rayfin for the first time a few weeks back and it’s incredible how quickly you can deploy an enterprise application using natural language. I built on a colleague’s idea for actionable report optimization across your Fabric tenant and scaled it to include additional insights across all your fabric items and connections and connected to your real data (instead of sample data).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6abJhAefoN02uOmnXqDJXw.png" /><figcaption>Fabric Estate Oprimizer Rayfin app</figcaption></figure><p>The native Fabric Capacity Metrics app is a great start, but how do I actually use it to easily reduce over-usage, allocate capacity efficiently, and optimize my tenant, for example by contacting owners whose reports haven’t been opened in over 3 months? That’s where this Rayfin Fabric Estate Optimizer app comes in. I’ll walk through the build and prerequisites below.</p><h3>Prerequisites</h3><p>You can start with Rayfin in a few ways — as documented in <a href="https://learn.microsoft.com/en-us/fabric/apps/create-app">mslearn</a>, you could install the <a href="https://www.npmjs.com/package/@microsoft/rayfin-cli">rayfin cli</a> and then create an app directly from there. Get started with Rayfin by first enabling and/or downloading the following:</p><ul><li>Fabric capacity in one of the <a href="https://learn.microsoft.com/en-us/fabric/admin/region-availability">supported Fabric App regions</a></li><li>In Admin Settings in Fabric, toggle the Fabric Apps (preview) setting to Enabled</li><li>Admin or contributor rights to a workspace in Fabric where your app will be deployed</li></ul><p>If you’d rather not wire up the CLI yourself, the community built <a href="https://github.com/spatney/rayfin-fabricator">Rayfin Fabricator</a> app streamlines setup, makes the deployment process much more seamless for business users, and installs most of the dependencies below for you. Pick the CLI if you’re comfortable in a terminal; pick Fabricator for a more guided experience.</p><ul><li><a href="https://github.com/">GitHub</a> account with <a href="https://github.com/features/copilot">Github Copilot</a> credits</li><li><a href="https://portal.azure.com/">Azure</a> account</li><li><a href="https://nodejs.org/en">Node.js</a> (latest version)</li><li><a href="https://www.npmjs.com/">npm</a></li><li><a href="https://git-scm.com/">Git</a></li><li><a href="https://learn.microsoft.com/en-us/cli/azure/?view=azure-cli-latest">Azure cli</a></li><li><a href="https://cli.github.com/">Github CLI</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/985/1*jLk5yAq8WkQPfn4DVjoPrQ.png" /><figcaption>Rayfin Fabricator requirements</figcaption></figure><p>You should be able to download all of this directly using the <a href="https://github.com/spatney/rayfin-fabricator">Rayfin Fabricator app</a> and you’re off to the races.</p><h3>Rayfin Fabric Estate Optimizer App</h3><p>Most Fabric users leverage the <a href="https://learn.microsoft.com/en-us/fabric/enterprise/metrics-app">Fabric monitoring app</a> for workspace and capacity monitoring, however, however, there are sometimes concerns over usability and the depth of insight. Yes, there are the <a href="https://github.com/microsoft/fabric-toolbox/tree/main/monitoring/fabric-cost-analysis">Fabric Cost Analysis (FCA)</a> and <a href="https://github.com/microsoft/fabric-toolbox/tree/main/monitoring/fabric-platform-monitoring">fabric platform monitoring</a> repos that you can implement, but why not tailor a monitoring app specifically to how your organization spends? Let’s play around with Rayfin to see if we can build a fully integrated capacity monitoring application with a few prompts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iqjuggyKIRo7NBFNlZRLvw.png" /><figcaption>Fabric Estate Optimizer data model</figcaption></figure><p>I started with a Report optimizer app which will evolve to a broader capacity monitoring app. To do so, you first need to connect a repo and a workspace. Next you get to building. I used two prompts for the initial build, then a few small follow-ups to troubleshoot. The first prompt has Copilot draft a high-quality build spec (using prompting best practices); the second tells it to execute that spec.</p><ul><li><strong>Prompt 1:</strong> <em>“Act as a Sr. software engineer. Define a prompt for the following scenario using prompting best practices as defined in this repo: </em><a href="https://github.com/x1xhlol/system-prompts-and-models-of-ai-tools">https://github.com/x1xhlol/system-prompts-and-models-of-ai-tools</a><em>. Scenario: </em>Build a Rayfin application called “Report Optimizer.” This is a Fabric tenant report-governance and cost-optimization workspace. It inventories every Power BI / Fabric reports in a tenant, scores usage, finds waste and duplication, tracks capacity (CU) cost, and answers plain-English questions. Use the default Rayfin stack, Microsoft Fabric authentication, and a typed Rayfin data client.</li><li><strong>Prompt 2:</strong> <em>“Use this prompt to design the capacity application”</em></li></ul><p>After submitting the second prompt the tool will usually take 10–20 mins to deploy but once it&#39;s done the build then click deploy (or have copilot auto deploy in the prompt). You may need to screenshot an error or copy the error details and paste them into the chat and ask Copilot to act as a Sr. SWE and fix the error. Expect a few iterations, as this is normal for a preview tool. After creating the report optimizer app, I expanded the scope of the applicaiton to include all fabric items.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yib-WHyrV98fmHJnPgAr2w.png" /><figcaption>Deploy Rayfin app</figcaption></figure><p>Guaranteed, you and your team have hundreds of ideas to implement, but not enough time to do so. Putting tools like Rayfin in their hands lets them quickly build apps that optimize workflows, cut costs, and speed up decisions, while giving IT and regulators confidence in a secure, governed foundation.</p><h3>Next steps</h3><p>In a world that&#39;s moving so quickly, it&#39;s hard to stay on top of every new tool and innovation. Rayfin is definitely one that you should keep an eye on as it will help you reduce costs, minimize development overhead and reduce time to value significantly.</p><p>If you think your organization could benefit from Rayfin, feel free to reach out if you are a Financial Service customer, or have a try for yourself and follow the instructions above to get started on your own. You can even deploy the capacity monitoring app from my repo <a href="https://github.com/alipouw13/capacity-optimizer">here</a> and give it a go yourself. You may want to add or modify pieces but it&#39;s a great place to start.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=05731fdbb120" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OneLake Security Deep Dive]]></title>
            <link>https://medium.com/@alipouw/onelake-security-deep-dive-8cb2c0195524?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/8cb2c0195524</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[onelake]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Thu, 21 May 2026 18:42:16 GMT</pubDate>
            <atom:updated>2026-05-21T18:42:16.886Z</atom:updated>
            <content:encoded><![CDATA[<p>As promised in my <a href="https://medium.com/@alipouw/defining-a-secure-microsoft-fabric-architecture-bb525b0d83d7">Fabric security overview</a> article and timed with the recent official GA announcement of OneLake security, this article we are going into OneLake security and workspace/item-level security. We will touch on managed roles, the SQL analytics endpoint, table/folder-level security, RLS/CLS, current limitations, and (most importantly) how these primitives shape how you should architect and operate your Fabric data platform. Note: OneLake security currently applies to Lakehouses (Read/Write), Mirrored Databases (Read), and Databricks Mirrored Unity Catalog (Read).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a8K8C3yRCenLrYOVpR222Q.png" /><figcaption>OneLake Security</figcaption></figure><h3>Workspace roles</h3><p><a href="https://learn.microsoft.com/en-us/fabric/fundamentals/roles-workspaces">Workspaces roles</a> are the broadest security boundary within a Fabric capacity to enforce access control. Roles can be assigned to users or security groups within your tenant. The following roles are applied at the workspace level: Admin, Member, Contributor, Viewer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/931/1*B40T_vIGloLHSuLxL5s_Hw.png" /></figure><p>The most important thing to note when it comes to workspace roles and OneLake security is that workspace Admin, Member and Contributor roles automatically grant Write permissions and override OneLake security Read restrictions. This is important when we get to OneLake security, as OneLake security only meaningfully restricts Viewers and users granted ReadAll on an item — the higher roles still see eveything. This means the workspace itself is the real hard security boundary and OneLake security is the fine-grained surface on top of it. Apply the principle of least privilege at the workspace layer first, as they grant very broad access. See <a href="https://learn.microsoft.com/en-us/fabric/fundamentals/roles-workspaces#-workspace-roles">here</a> for details on the permissions grated at each role.</p><h3>Data access control model</h3><p>OneLake security uses RBAC to manage permissions. Each role has:</p><ul><li><strong><em>Type: </em></strong>GRANT, DENY (only GRANT is available today)</li><li><strong><em>Permission: </em></strong>actions allowed/disallowed — Read or Read/Write</li><li><strong><em>Scope: </em></strong>OneLake data affected — all data or selected tables, folders, or schemas within a supported item</li><li><strong><em>Members: </em></strong>who to apply the role to — Entra identities including users, security groups or service principals / managed identities</li></ul><p>For member assignment, you can manually define individuals and members or create <a href="https://learn.microsoft.com/en-us/fabric/onelake/security/create-manage-roles#assign-virtual-members">virtual memberships</a> with permission groups. Virtual membership dynamically adjusts based on the Fabric item permissions of the users — this is a better long-term solution when you’re thinking about scalability. You must have Fabric Write or Reshare permissions (usually workspace Admin or Member) to create managed roles.</p><h4>Workspace roles &amp; OneLake Security</h4><p>On top of control plane data access, workspace roles provide default access levels to data items by using OneLake security default roles. Each supported item is provisioned with default OneLake security roles that mirror traditional workspace-level data access. The most common is DefaultReader, which uses virtualized membership that grants Read to users who have ReadAll on the item. See below for default role types and their associated permissions by item type.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/1*1MxvVgpJnU8vYoeWDA63sA.png" /></figure><p>When you add a user to a more granular custom data access role, make sure you remove them from the DefaultReader role, so they don’t maintain full data access. Alternatively, delete or modify DefaultReader when you create your first restrictive role. <strong><em>Make this a standard step in your item-provisioning runbook.</em></strong></p><h4>Table and folder-level (object-level) security</h4><p>OneLake security roles can grant access to tables and folders in supported items. For a folder to be treated as a table:</p><ol><li>It must sit under Tables/ (and in a valid schema folder for schema-enabled items).</li><li>It must contain a _delta_log with valid JSON.</li><li>It must not include child shortcuts.</li></ol><p>If any of these fail, table-level security denies access entirely. Other behavior that must be considered and designed around:</p><ul><li><strong>Inheritance</strong>: Once permissions are granted on a folder, it inherits to every child, including subfolders that fall inside shortcuts. There is no way right now to break inheritance on a child.</li><li><strong>Listing shortcuts</strong>: Listing a directory returns all internal shortcuts regardless of the user’s access on the shortcut target — access is only checked at <strong><em>open</em></strong>.</li></ul><h4>Item-level security</h4><p>You can configure item-level permissions independently of workspace roles by sharing an item or via the manage permissions pane on the item. A user’s effective access is the combination of what they get from the workspace and any direct item shares. You can grant item-level access two ways: <strong><em>Share </em></strong>(quick path, defaults to Read plus optional Reshare/Build add-ons) or <strong><em>Manage permissions</em></strong> (full control, including removing inherited workspace access). The matrix below summarizes each permission and what it lets a user do.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*5Ol_xbxxMFLRCi80SLdn-g.png" /></figure><h3>OneLake Security on SQL analytics endpoint</h3><p>To use OneLake security with the SQL analytics endpoint you must switch the endpoint to User identity access mode (one-time per endpoint). There are two access modes with entirely different governance models:</p><ul><li><strong>User identity mode:</strong> manage security centrally with consistent Entra enforcement. Uses a passthrough Entra authentication where access is determined based on the security rules defined within the OneLake item, not by any SQL-level GRANTs.</li><li><strong>Delegated identity mode: </strong>all access control is defined and enforced at the SQL engine layer. OneLake security roles and access policies are not applied.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*nubTlbY6Ri6N9_NIpw7iHg.png" /></figure><p>When we think about open architectures, unifying data, and federated architectures, user identity mode takes the cake. Entra-based enforcement provides consistent governance across Power BI, Spark, Lakehouse, and SQL. If you have workload with SQL-native semantics or legacy T-SQL tooling, then you could use delegated identity. I would, however, explore ways to modernize this architecture to Entra-based auth.</p><p>One core security gotcha is with security sync in user identity mode: renaming/deleting a column referenced by an RLS/CLS rule puts the database into an error state until the policy is fixed. Spark/notebook code referencing the renamed column will also fail until updated. Build a change-management discipline around schema evolution.</p><h3>Shortcut security</h3><p>There are two authentication models for shortcuts to authorize data access:</p><ul><li><strong>Passthrough shortcuts:</strong> the caller’s identity is evaluated against the shortcut target.</li><li><strong>Delegated shortcuts: </strong>a stored credential is used to access the target; the caller is evaluated against OneLake security on the shortcut</li></ul><p>Internal shortcuts (OneLake to OneLake) are always passthrough. You don’t need to configure any permissions on the shortcut. Permissions cannot be defined on the shortcut itself. You must set permissions on the target folder in the producer item. This means the producer item is always the security authority and the consumer-side shortcuts cannot loosen or tighten that.</p><p><strong>Identity passthrough breaks for Power BI DirectLake over SQL and SQL analytics endpoint in delegated identity mode. </strong>For both, the item owner’s identity (not the caller’s) is used at the shortcut target. To preserve passthrough, ensure you use <strong>DirectLake over OneLake mode</strong> or <strong>T-SQL in user identity mode</strong>. Additionally, if the target item type doesn’t support OneLake security, fallback is a ReadAll permission check.</p><p>Delegated shortcuts are for external sources, such as ADLS Gen2, S3, Iceberg, Dataverse, etc. See a full list of external shortcuts <a href="https://learn.microsoft.com/en-us/fabric/onelake/onelake-shortcuts#external-onelake-shortcuts">here</a>. A user accessing an external shortcut requires Fabric Read permission on the target item. An internal shortcut pointing to an external shortcut still requires the user to follow to the original external shortcut’s access rules. See full documentation <a href="https://learn.microsoft.com/en-us/fabric/onelake/onelake-shortcut-security">here</a>. See <a href="https://learn.microsoft.com/en-us/fabric/onelake/create-onelake-shortcut">here</a> to create an internal shortcut and subpages on creating external shortcuts.</p><h3>RLS/CLS</h3><p>Row and column level security (<a href="https://learn.microsoft.com/en-us/fabric/onelake/security/row-level-security">RLS</a>/<a href="https://learn.microsoft.com/en-us/fabric/onelake/security/column-level-security">CLS</a>) can be applied to delta tables only and primarily affect Viewers*. This is where your end-to-end security architecture makes a difference. You must consider how to apply permissions with RLS/CLS, at the item/folder-level, at the workspace-level, how networking should be configured, as well as information protection and DLP with Purview. Reference <a href="https://medium.com/@alipouw/defining-a-secure-microsoft-fabric-architecture-bb525b0d83d7">this</a> article for a full overview of E2E security architecture in Fabric.</p><p>Some key constraints to consider for both RLS and CLS include:</p><ul><li><strong>RLS: </strong>Delta only; ≤1,000 chars per predicate; limited SQL surface. No joins, no functions, no subqueries.</li><li><strong>CLS: </strong>Delta only; at least one column must remain visible; engines disagree on SELECT * behavior (Spark filters silently, SQL endpoint / semantic models block the query). In the SQL endpoint, CLS resolves via intersection (deny-wins) while in Spark it’s a union — the same user can see different columns in different engines.</li><li>You <strong>cannot combine an RLS-only role and a CLS-only role</strong> for the same user. You must put both in a single role.</li><li><strong>Non-authorized engines</strong> (e.g., raw ADLS API): RLS/CLS tables are <strong>fully blocked</strong>, not filtered.</li></ul><p>*Although OneLake security is targeted to Viewers, there are some exceptions where higher-privilege roles are still affected:</p><ul><li><strong>RLS in user identity mode:</strong> When a downstream consumer accesses producer data via DirectLake-over-SQL or a SQL endpoint in Delegated identity mode, the item owner’s identity (not the caller’s) is evaluated against the producer’s roles. A downstream workspace Admin can therefore be filtered by an upstream RLS rule they aren’t subject to.</li><li><strong>CLS schema inspection:</strong> CLS does not protect against schema introspection meaning column names remain visible in metadata views (defined in the limitations above too).</li><li><strong>Security Sync Failure: </strong>if the security sync fails, all users of the affected roles may have their access restricted</li><li><strong>Workflow restriction in User’s identity mode: </strong>SQL GRANT/REVOKE on tables is ignored. Admins who try to manage permissions via T-SQL must instead author OneLake security roles.</li></ul><h3>Architectural implications</h3><p>Okay so, this is all fine and dandy, but why should you care, and how can you use this implement a secure and functional architecture at scale?</p><ol><li><strong>Adopt a producer/consumer (hub-and-spoke) workspace model: </strong>Admin/Member/Contributor always see all data. Sensitive data must live in workspaces where consumers are Viewers only. Producer workspace items hold the authoritative roles; consumer workspace items consume via shortcuts in User identity mode. <strong>Bias to item-level Read sharing over assigning Workspace Viewer broadly — it’s easier to revoke and audit per dataset.</strong></li><li><strong>DefaultReader </strong>is given to everyone with ReadAll: <strong>delete or restrict it</strong> as a standard provisioning step.</li><li><strong>Identity strategy must be security-group-only.</strong> Distribution lists don’t resolve; nested groups don’t traverse the producer to consumer boundary (cross-tenant/cross-workspace Entra group resolution); B2B needs tenant configuration.</li><li><strong>Engine selection is a security decision. </strong>DirectLake-over-SQL and SQL Delegated mode break identity passthrough. Standardize on DirectLake-over-OneLake and SQL User’s identity mode.</li><li><strong>Lakehouse layout must be schema-enabled and shortcut-aware. </strong>For OneLake security to work, keep shortcuts out of Tables/. Non-schema lakehouses break data preview for secured tables. This is critical for gold-layer tables and is core to <strong>medallion architecture design</strong>.</li><li><strong>Service principals must use OneLake ReadWrite roles: </strong>Scope service principals to OneLake ReadWrite roles or item-level permissions; avoid granting them workspace Admin/Member, which bypasses OneLake security.</li><li><strong>RLS/CLS rules</strong>: Combine both in one role per domain, and sensitivity-level.</li><li><strong>Gaps require additional security layers</strong>: Read audit gaps, no Data Share integration, no Dynamic Data Masking (DDM). Leverage Purview (sensitivity, DLP, Activity Explorer), Defender for Cloud Apps, and Fabric network security / Private Links.</li></ol><p>Take the above along with our <a href="https://medium.com/@alipouw/defining-a-secure-microsoft-fabric-architecture-bb525b0d83d7">E2E Security Overview</a>, and data strategy section of the <a href="https://medium.com/@alipouw/choose-the-right-microsoft-agent-builder-for-your-agentic-use-case-b57592e6cd03">agent builder article</a> to get started on building your security foundation in Microsoft Fabric. Hope this was helpful!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8cb2c0195524" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Snowflake & Microsoft Fabric integration: 3 ways to connect them]]></title>
            <link>https://medium.com/@alipouw/snowflake-microsoft-fabric-integration-3-ways-to-connect-them-2c71c13e572e?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/2c71c13e572e</guid>
            <category><![CDATA[data-virtualization]]></category>
            <category><![CDATA[unified-data-platform]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <category><![CDATA[database-mirroring]]></category>
            <category><![CDATA[snowflake]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Tue, 12 May 2026 16:05:29 GMT</pubDate>
            <atom:updated>2026-05-12T16:05:29.472Z</atom:updated>
            <content:encoded><![CDATA[<p>There’s been many announcements recently on the interoperability between Snowflake and Fabric. You can work with Snowflake data in Fabric through multiple integration patterns including mirroring data, leveraging shortcuts with Iceberg-Delta metadata virtualization, and Snowflake databases in Fabric with Iceberg interoperability. Now how do you know when to use which? In this article I go over the 3 core ways to work with Snowflake data in Fabric, scenarios when you would use each method, and cost considerations with links to tutorials to set up your environment for each feature.</p><p>At a high-level, use:</p><ul><li><strong>Mirroring</strong>: if Fabric is your analytics platform of choice and you will get benefit from Power BI Direct Lake mode.</li><li><strong>Shortcuts using metadata virtualization</strong>: if you want a single data copy while Snowflake remains the primary writer and Fabric is needed for analytics.</li><li><strong>Snowflake-managed Iceberg in OneLake:</strong> for the best long-term open, interoperable architecture when both platforms need write access to the same data.</li></ul><p>Realistically, most customers use a combination of all three, depending on the use case. Read more below for more insight into each.</p><h3>Mirroring</h3><p>Mirroring is one of Microsoft Fabric’s core workloads to move data into OneLake. Mirroring is a zero ETL solution to copy data from an ever-expanding <a href="https://learn.microsoft.com/en-us/fabric/mirroring/azure-sql-database">list</a> of databases or data warehouses to Fabric, including open mirroring Partner solutions. Depending on the data source, mirroring can either replicate the entire database or just the metadata in OneLake in Delta tables and keep the data in sync in near real time. In the case of Snowflake, all selected data is replicated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sFWX9H-_A7hyyo-moAkytA.png" /><figcaption>Mirrored Snowflake DB to Fabric</figcaption></figure><p>Use mirroring when you are using Fabric as your primary analytics platform and you want the best performance Fabric experience, including Direct Lake reporting. Go through this <a href="https://learn.microsoft.com/en-us/fabric/mirroring/sql-server-tutorial?tabs=sql2025">tutorial</a> to get mirroring set up between Snowflake and Fabric. To note, mirroring requires periodic reads from Snowflake and can introduce source system load depending on change volume — this should be minimal, but worth considering.</p><p>From a cost perspective, on the Snowflake side you will be charged for compute and cloud services for reading source tables and capturing changes. Fabric doesn’t charge for data ingress into OneLake. Mirrored database storage is included free up to your capacity’s mirroring entitlement (e.g., F64 includes 64 TB of mirrored storage). You also don’t pay for any ETL services for ingesting any data. See <a href="https://learn.microsoft.com/en-us/fabric/mirroring/snowflake#mirrored-snowflake-cost-considerations">here</a> for more details on mirroring cost considerations.</p><h3>Shortcuts on Snowflake-managed Iceberg tables</h3><p>When you have existing Iceberg tables stored in an external object store and want to read data in Fabric without data movement, you need to leverage OneLake shortcuts. Behind the scenes, Fabric leverages metadata virtualization, which allows Iceberg tables to be interpreted as Delta tables and vice versa from the shortcut’s perspective. Behind the scenes, <a href="https://xtable.apache.org/">Apache XTable</a> is used for metadata conversion between both formats.</p><p>You would create a shortcut from Snowflake Iceberg in an external store to Fabric Delta tables if you wanted to keep Snowflake as the primary writer but also combine Snowflake data with other sources and facilitate reporting in Power BI with Direct Lake mode. See a tutorial <a href="https://learn.microsoft.com/en-us/fabric/onelake/onelake-iceberg-tables">here</a> to get started on Iceberg to Fabric and vice versa. There are <a href="https://learn.microsoft.com/en-us/fabric/onelake/onelake-iceberg-tables#limitations-and-considerations">limitations</a> for the Iceberg to delta virtualization, so be sure to review these before implementing.</p><p>Similarly, if you have data in Fabric and need to bring it to Snowflake for read-only analytics or exploration, review the start of <a href="https://www.snowflake.com/en/developers/guides/getting-started-with-iceberg-in-onelake/#create-a-snowflake-catalog-linked-database">this tutorial</a> for implementation guidance. The Catalog-Linked Databases (CLD) setup allows Snowflake to read data in OneLake by interpreting Delta tables with Iceberg-compatible metadata. This is useful when you have data that is easy to get into Fabric (eg: any M365 Data, native Azure sources, sources you can easily shortcut into Fabric to reduce data copy) but you want to use Snowflake as your primary analytics platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nk6zS7cjlnx3XpUXp90hgw.png" /><figcaption>CLD in Snowflake: FABRIC_BRONZE_CLDB</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oSD5zvn4au9m3dRZiOXTag.png" /><figcaption>Bronze lakehouse in CLD</figcaption></figure><p>From a cost perspective, within Snowflake you pay for compute to write or maintain your iceberg tables, while in Fabric you only pay for the read operations on the virtualized data. You don’t pay for storage in Fabric, pipelines or any replication costs that we mentioned in the mirroring section.</p><h3>Snowflake-managed Iceberg in OneLake</h3><p>This is the best long-term solution when prioritizing open formats, scalability, and cross-platform interoperability. If you’re using Snowflake as your analytics platform of choice but want reporting or to leverage Fabric for select analytics, then the Snowflake database in Fabric allows unified storage in OneLake, optimized storage costs and open formats with the best performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fWWyT1AVSnmmruvT_0anTQ.png" /><figcaption>Snowflake Database in Fabric</figcaption></figure><p>Use this in combination with CLD in Snowflake to enable shared access patterns across both platforms, with OneLake as the single source of truth, read interoperability, and support for write patterns on both platforms. With this single copy, you can query the data from both platforms. You can also leverage mirroring for your gold tables into Fabric if you want to take advantage of Direct Lake mode for optimized reporting speed and cost.</p><p>From a cost perspective, you pay for Snowflake compute required to write or update/maintain the Iceberg tables and Fabric SKU required for any Fabric analytics you run on the data. You pay for storage once, no cost for replication or ETL between platforms. Review the end of the same <a href="https://www.snowflake.com/en/developers/guides/getting-started-with-iceberg-in-onelake/#create-a-snowflake-database-in-onelake">tutorial</a> we looked at for the CLD.</p><h3>Summary</h3><p>There are multiple paths to achieve the same outcome, and it ultimately depends on your organization’s platform strategy and operating model. A quick decision framework based on our conversation above:</p><ol><li><strong>Is Fabric your primary analytics platform and is Power BI reporting a <br>top priority? </strong>Start with mirroring. You’ll get the best Direct Lake performance with minimal setup.</li><li><strong>Do you already have Iceberg tables managed elsewhere (Snowflake or <br>another store) and want to read them in Fabric without moving data?</strong> Use <br>shortcuts with Iceberg–Delta metadata virtualization.</li><li><strong>Do both Snowflake and Fabric need read and write access to the same data, with open formats as a long-term strategy? </strong>Go with Snowflake-managed Iceberg in OneLake, paired with CLD in Snowflake for bidirectional access. Layer mirroring on top of your gold tables when you need Direct Lake.</li></ol><p>If you’re looking to save on cost, use Fabric for primary analytics engine. However, if you’re using Snowflake for most or all of your data processing and Fabric primarily for storage and reporting, you can land everything on OneLake and enable two-way read/write with CLD and Snowflake-managed Iceberg at each medallion layer. Keep your reporting layer in Fabric and mirror the gold layer to use Direct Lake mode without having to use metadata virtualizaiton. This way you minimize data copies, optimize for cost, and optimize for reporting to ensure the most effective use of both platforms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HtRooFhkwsjDpP7OmpfZrQ.png" /><figcaption>Snowflake x Fabric architecture for using Snowflake as primary analytics engine</figcaption></figure><p>The above architecture illustrates using Fabric for initial integration of some data sources with Snowflake integrating other source into the bronze layer. A CLD exposes OneLake data to Snowflake, allowing Snowflake to transform Fabric-ingested data alongside Snowflake-managed datasets into a silver layer. The physical data is stored in OneLake in Iceberg format at each layer, represented by the Snowflake DBs in Fabric.</p><p>Mirroring the gold database into Fabric enables Direct Lake reporting, while Iceberg tables enable shared access between Snowflake and Fabric. If you were to do some transformations in Fabric, you would add silver and gold lakehouses in Fabric and expose them to Snowflake via CLD. At the top, Purview can be used across all three patterns to provide unified governance, lineage, security, and data discovery across Snowflake and Fabric assets.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c71c13e572e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Choose the right Microsoft agent builder for your agentic use case]]></title>
            <link>https://medium.com/@alipouw/choose-the-right-microsoft-agent-builder-for-your-agentic-use-case-b57592e6cd03?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/b57592e6cd03</guid>
            <category><![CDATA[co-pilot]]></category>
            <category><![CDATA[agent-builder]]></category>
            <category><![CDATA[agentic-applications]]></category>
            <category><![CDATA[microsoft-foundry]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Mon, 04 May 2026 13:57:49 GMT</pubDate>
            <atom:updated>2026-05-04T13:57:49.996Z</atom:updated>
            <content:encoded><![CDATA[<p>Microsoft is a behemoth — this is both good and bad. Good because you have access to so many tools to help your business thrive. Bad because with all these resources comes confusion on when you use which tool and what additional considerations you need to implement a given solution or application. I’ve heard this story time and time again with so many customers (and internal to Microsoft). Even when it comes to learning, we are constantly bombarded with new services and resources to help us succeed but it’s a task in itself to sift through all the noise and get to what’s really important.</p><p>Now when it comes to agentic applications, Microsoft has provided a broad selection of options, depending on your use case and persona, but how do I actually know what’s right for me without digging excessively into the documentation? First, our America’s CTO put together a helpful <a href="https://www.youtube.com/watch?v=WUgujz0y1K4">video</a> on this so I would recommend watching it. Second, I went about developing a quick app to go through setting up your business for success — this is what I’ll walk through today. By the end of this article you’ll know how to assess your AI landing zone readiness, how to choose between M365 Copilot / Copilot Studio / Foundry / Fabric agents, and what a multi-agent advisor architecture actually looks like in practice.</p><p>Please note, this is an internal MSFT application right now, however, if this is of interest, you can play around with my <a href="https://github.com/alipouw13/agentic-advisor">repo</a> and deploy for yourself, or if you’re a customer, please reach out and we can go through a demo. The core Azure services include Microsoft Foundry, Cosmos DB, Entra ID SSO, and MCP-based grounding.</p><h3>Agentic advisor: application overview</h3><p>There are 4 areas covered in the app to help the user understand the foundational principles of building agentic applications before using the advisor feature which takes into account these foundational principles in providing a recommendation for your use cases.</p><ol><li><strong>Landing zone: </strong>covering Microsoft CAF best-practice architecture.</li><li><strong>Data strategy: </strong>Covering designing your unified data layer in Microsoft Fabric. This includes the architecture, security foundations and a CI/CD strategy for promoting changes across environments.</li><li><strong>Agent adoption: </strong>Aligned to MSFT CAF framework for adopting, deploying and scaling agents</li><li><strong>Agent types: </strong>An overview of the 4 core types of agents in the MSFT ecosystem (M365 Copilot, Copilot studio, Microsoft Foundry, Fabric or Fabric through Foundry/Copilot Studio/M365 Copilot) and a quick initial decisioning framework</li><li><strong>Agent advisor: </strong>The main part of this app — an advisor to provide a baseline framework to build scalable agentic applications for your use cases.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pl3muJzQraXjuMBfiLgCvA.png" /></figure><p>We’ll start with the foundations (landing zones and building your unified data layer) then move to adoption, agent types, and finally the advisor itself.</p><h3>Landing zones</h3><p>Landing zones are an essential part in designing a successful cloud architecture. Core components include identity and access management through Entra ID, network isolation via a hub-spoke topology with private endpoints for AI services, governance guardrails through Azure Policy and management groups, and observability using Azure Monitor and App Insights. An AI landing zone extends this foundation with components specific to AI workloads locked down behind private endpoints — think Foundry projects, AI Search, Key Vault for secrets, and Cosmos DB. See below for specifics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*0jP81G3xw-3JdL517U0Iow.png" /><figcaption>Required landing zone controls</figcaption></figure><p>The app provides a readiness checklist of 10 items across foundation, networking, identity, AI services, observability, compliance, governance, and security so you can assess where you stand before building anything.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wzil54vUqrUSoDtzxbHEEw.png" /></figure><p>See <a href="https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/landing-zone/?tabs=hubspoke">CAF landing zone architectures</a> and <a href="https://github.com/Azure/AI-Landing-Zones">AI landing zones</a> for full implementation insights. My colleague Matt Felton is an all-star and also has some fantastic <a href="https://journeyofthegeek.com/author/mattfeltonma/">blogs</a> on properly configuring landing zones and all things Foundry.</p><h3>Data strategy</h3><p>I’ve talked a lot about data strategy in my previous blog posts, so I won’t beat a dead horse, but as a high-level, you need unified data to get the best insights and ROI from your applications and agents. Without a solid data foundation you end up with contradictory copies of the same data and, predictably, frequent agent hallucinations. Below we have the CAF data architecture providing a framework for designing your data platform to be ready for AI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*puIjGo9SWgRWOvY1RLAfqw.png" /><figcaption>CAF unify data architecture</figcaption></figure><p>Fabric consolidates your data silos into a OneLake to serve one governed copy of data in Delta format to use for analytics, reporting, and AI development. The app walks through the 5 core storage options (lakehouse, warehouse, SQL database in Fabric, eventhouse, and mirrored databases), capacity tier sizing from F2 sandbox to F512 and mission-critical capacities, and the medallion architecture (bronze, silver, gold) for building trusted, and certified data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*TwSgy-fquyhmh-dXjB0TZQ.png" /><figcaption>Fabric architecture</figcaption></figure><p>Capacity sizing rule of thumb is listed below. The medallion architecture is just a 3-tiered approach to organize your lakehouse data with each layer increasing in data quality. It ensures your consumers are pulling from clean and report-ready data instead of messy, raw data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_TvmTBmCuHsz3RfNmhe3wQ.png" /><figcaption>Capacity sizing guidance</figcaption></figure><p>On top of that we layer the unified semantic intelligence layer: Fabric IQ with ontologies, knowledge graph, and semantic models, which is where your agents sit to deliver grounded, governed results. This is where your agents will sit to deliver the best results, with security, governance and compliance acting as guardrails to define boundaries on which data your agents can access. Thinking from the Fabric lens, you must consider our 4 levels of security, explained in my previous <a href="https://medium.com/@alipouw/defining-a-secure-microsoft-fabric-architecture-bb525b0d83d7">article</a>:</p><ul><li>Purview</li><li>Platform/network-level security (inbound and outbound)</li><li>Workspace and item-level security</li><li>OneLake security</li></ul><h3>Agent adoption</h3><p>Once you have your landing zones established and a defined data strategy, you can start building for scalable AI applications. The app follows the CAF AI Agent Adoption framework organized into 4 phases:</p><ol><li><strong>Plan: </strong>identify use cases, define success metrics, select your platform, build AI skills</li><li><strong>Govern &amp; Secure: </strong>implement Responsible AI policies, define governance standards, prepare your environment</li><li><strong>Build: </strong>decide single vs. multi-agent, build RAG pipelines, establish evaluation and quality gates</li><li><strong>Manage: </strong>integrate into business workflows, monitor KPIs, scale to a governed agent ecosystem</li></ol><p>Each phase includes an interactive checklist with links to the corresponding MS Learn documentation, so you can track your readiness and know exactly where to go next. See more detail <a href="https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai/">here</a> and see below for the checklist provided in the app.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*HwfA4btnP7RWOE8qlxaD-A.png" /><figcaption>Agent adoption checklist</figcaption></figure><h3>Agent types</h3><p>Agent types goes over a high-level explanation of each agent type, use cases and a quick decision matrix on when to use each. The app includes an interactive decision matrix that:</p><ol><li>Asks 5 questions: orchestration complexity, primary user persona, data location, customization level, and time to value.</li><li>Recommends the right platform for your scenario based on best practices.</li><li>Recommends when agents are not the right answer: if your task is structured and predictable, or you just need simple Q&amp;A, you probably don’t need an agent at all (use power automate or an LLM).</li></ol><p>The 4 platforms map to clear personas: M365 Copilot Agent Builder for business users who need something in minutes, Copilot Studio for citizen developers who need connectors and workflow automation, Azure AI Foundry for pro-code developers building complex multi-agent systems, and Fabric Data Agents for data teams who want conversational analytics over their lakehouse and warehouse data. The cost profile varies across across the four options: from per-seat (<a href="https://www.microsoft.com/en-us/microsoft-365-copilot/pricing/enterprise">M365 Copilot</a>) to pre-paid message packs plus pay-go pricing (<a href="https://learn.microsoft.com/en-us/microsoft-copilot-studio/billing-licensing#get-started-in-copilot-studio">Copilot Studio</a>) to token-based (Foundry <a href="https://azure.microsoft.com/en-us/pricing/details/azure-openai/">Open AI</a> and <a href="https://azure.microsoft.com/en-us/pricing/details/ai-foundry-models/microsoft/">other models</a>) to capacity-based (<a href="https://azure.microsoft.com/en-us/pricing/details/microsoft-fabric/">Fabric</a> Data Agents).</p><p>In short, leverage the table below to help in your decisioning:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gu2kNbXWJssoD-Zn43K_aw.png" /><figcaption>Microsoft agent decision guide</figcaption></figure><h3>Advisor</h3><p>The core functionality of this application is in the name: <em>the agentic advisor. </em>This advisor provides the user an architectural recommendation and implementation plan for their specific use case. As an example, it will define if you need a single-agent or multi-agent system built in M365 Copilot, Copilot studio, Microsoft Foundry, Fabric or Fabric through Foundry/Copilot Studio/M365 Copilot. It will provide reasoning, data integration patterns, tools and services required and a high-level mermaid diagram architecture and implementation guidance. From this it would provide an implementation guide for making your use case real in your environment. See below for a sample flow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ooyZ9Y1pAoUCcCTfudigOg.png" /><figcaption>Sample E2E agentic advisor flow</figcaption></figure><p>The recommendations come from the collaboration efforts of agents the following agents, executed by a central orchestrator agent. The agents are grounded on real-time documentation instead of stale training data with 3 using MS Learn MCP and one using Fabric MCP. Real-time MS Learn grounding means recommendations stay correct as Microsoft renames/retires services. After a suggestion is produced the user is able to iterate, provide feedback and ensure the implementation plan output meets their needs — this provides a human-in-the-loop and iterative experience.</p><ul><li><strong>M365 knowledge agent: </strong>leverages work-iq to understand internal assets that could influence the architectural recommendation.</li><li><strong>MS learn agent: </strong>leverage <a href="https://learn.microsoft.com/en-us/training/support/mcp">MS learn mcp</a> to research relevant information relevant to the use case and recommendation.</li><li><strong>Fabric intelligence agent:</strong> uses Fabric MCP to get access to user workspaces to inventory current items to provide environment-aware recommendations.</li><li><strong>WAF architecture agent:</strong> Provides <a href="https://learn.microsoft.com/en-us/azure/well-architected/pillars">Azure Well-Architected Framework (WAF)</a> (5 pillars: Reliability, Security, Cost Optimization, Operational Excellence, Performance Efficiency) and <a href="https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/overview">Cloud Adoption Framework (CAF)</a> (7 methodologies: Strategy → Plan → Ready → Adopt → Govern → Secure → Manage). Assesses AI landing zone readiness using CAF AI Ready guidance leveraging MS learn mcp searching WAF and CAF content.</li><li><strong>Platform selection agent: </strong>Leverages MS learn MCP and static knowledge base: agent-platform-guide.md, decision-matrix.json, platform-comparison.json, and session transcript to provide the best recommendation for the use case.</li><li><strong>Governance and scaling agent:</strong> leverage MS learn MCP and static knowledge base: governance-scaling-guide.md, .github/agents/ definitions (CAF adoption, SaaS architect, API architect, governance reviewer) to assess how to agent(s) can scale in an enterprise architecture.</li><li><strong>Orchestrator agent:</strong> uses keyword-based routing, parallel execution, synthesis, approval flow, and implementation plan generation to orchestrate sub-agents and user responses.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5tvICHEobS1vNv6ihe0LoQ.png" /><figcaption>Agent architecture</figcaption></figure><p>The Platform Selection and Governance agents run on every request since these are non-negotiable foundations and must be considered for every implementation. The M365, Fabric, and WAF agents activate when your use case mentions relevant signals (e.g., “SharePoint” triggers the M365 agent, “lakehouse” triggers Fabric). Agents run in parallel to minimize latency, and each grounds its response using the provided MCP if applicable. As mentioned, the MS Learn, WAF, Platform Selection, and Governance agents all use MS Learn MCP to search current Microsoft documentation in real time, while the Fabric agent uses Fabric MCP to query your actual workspace. The orchestrator then synthesizes all agent outputs through a CAF-aligned decision framework to propose an optimized solution design.</p><h3>Summary</h3><p>Microsoft is a very innovative place, sometimes to its detriment. We’ve done a poor job at guiding customers on leveraging the right products for the right use case. Our agent ecosystem is broad and it’s breadth enables users across your organization once you have the proper framework to navigate it. I hope this article provided some guidance and a practical framework for choosing and scaling the right agent architecture. If you want to try it yourself, the repo is available <a href="https://github.com/alipouw13/agentic-advisor">here</a> and if you’re a Microsoft customer interested in a guided demo, reach out and let’s make your agentic use case real.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b57592e6cd03" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Defining a secure Microsoft Fabric architecture]]></title>
            <link>https://medium.com/@alipouw/defining-a-secure-microsoft-fabric-architecture-bb525b0d83d7?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb525b0d83d7</guid>
            <category><![CDATA[regulatory-compliance]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[network-security]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Fri, 17 Apr 2026 20:54:14 GMT</pubDate>
            <atom:updated>2026-04-17T20:54:14.554Z</atom:updated>
            <content:encoded><![CDATA[<p>We’ve covered Fabric as a u<em>nified data platform </em>and how to leverage this unified data for AI and Agentic applications. In highly regulated industries, we need to also consider security and networking, along with a baseline architecture that will scale with your workloads. In this blog post, we will walk through a high-level Fabric architecture and tough on everything security and networking with Purview and Fabric.</p><h3>Fabric Architecture</h3><p>There are three levels of architecture elements to consider: capacities, domains, and workspaces. Capacities are used for compute of all operations in Fabric. These are the highest level of segmentation. They also act as the primary boundary for performance isolation, noisy‑neighbor protection, and in some cases compliance controls. I’ve worked with many customer who separate capacities for data residency purposes (ie: EU, North America, etc), as well as between prod and non-prod environments. Aside from data residency requirements, I suggest having 3-4 capacities: (optional) Mission critical, Prod, Non-prod, and Sandbox. This ensures critical operations will not be throttled by non-critical workloads.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VCT3vk1ZP2UDcX3aUAst0g.png" /></figure><p><a href="https://learn.microsoft.com/en-us/fabric/governance/domains">Domains</a> are a feature that enable you to build a data mesh architecture. Organize by business function or LOB (Risk, Customer Analytics, Finance, Trading). Each domain owns its data products and workspace lifecycle. These domains should align to your governance domains in Microsoft Purview (or other governance tool). You can use OneLake shortcuts to share data products across domains without duplication. Purview classifications, sensitivity labels and governance context remain associated with the data as it’s share across domains.</p><p>Think of <a href="https://learn.microsoft.com/en-us/fabric/fundamentals/workspaces">workspaces</a> as a container, isolating workloads and permissions from one another. Workspaces can be assigned to one capacity and one domain. Workspaces in the same domain can be assigned to different capacities. Typically, you will have at least 3 workspaces per LOB or LOB application, for dev, test, and prod. Within analytical workspaces you’ll aim to implement bronze, silver and gold layers, following a medallion architecture. Leverage <a href="https://learn.microsoft.com/en-us/fabric/cicd/git-integration/intro-to-git-integration?tabs=azure-devops">git-integration</a> cicd pipelines with variable groups for promoting content from one environment to the next.</p><h3>Fabric security layers</h3><p>Think of the security in layers: Purview (<a href="https://learn.microsoft.com/en-us/fabric/governance/microsoft-purview-fabric">in Fabric</a> and beyond Fabric for <a href="https://learn.microsoft.com/en-us/purview/data-governance-overview">governance</a>, <a href="https://learn.microsoft.com/en-us/purview/purview-security">security</a> and <a href="https://learn.microsoft.com/en-us/purview/purview-compliance">compliance</a>), platform and network security (<a href="https://learn.microsoft.com/en-us/fabric/security/security-inbound-overview">inbound</a> and <a href="https://learn.microsoft.com/en-us/fabric/security/security-managed-vnets-fabric-overview">outbound</a>), <a href="https://learn.microsoft.com/en-us/fabric/data-warehouse/security">workspace and item-level</a> access, and <a href="https://learn.microsoft.com/en-us/fabric/onelake/security/get-started-security#onelake-security-preview">OneLake security</a>.</p><p><strong>Purview</strong></p><p>Purview covers the top-level policies applied to your most critical data assets. Specific to AI and Agents, Purview enables users to secure interactions with CoPilot in Fabric, detect risky interactions and unethical behavior in data agents and other AI apps with <a href="https://learn.microsoft.com/en-us/purview/dspm-for-ai?tabs=m365">DSPM for AI</a>. You can then define <a href="https://learn.microsoft.com/en-us/purview/create-retention-policies?tabs=teams-retention">retention policies</a> for your copilot experiences with data lifecycle management, define <a href="https://learn.microsoft.com/en-us/purview/collection-policies-solution-overview">collection policies</a> for copilot integrations, and run eDiscovery cases to identify information that can be used as evidence in legal cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*2XZ49FdULpeoNsvMDFhnnA.png" /><figcaption>Supported solutions for Copilot in Fabric</figcaption></figure><p>On the data security side, you can leverage Information Protection to apply sensitivity labels to your entire Fabric estate, and corresponding <a href="https://learn.microsoft.com/en-us/purview/dlp-powerbi-get-started">DLP</a> policies. You can also configure policy indicators for lakehouses and Power BI workloads — these are indicators that correspond to specific triggers and risk activities with <a href="https://learn.microsoft.com/en-us/purview/insider-risk-management-settings-policy-indicators#microsoft-fabric-indicators">Insider Risk Management</a>. Finally, you can run a <a href="https://techcommunity.microsoft.com/blog/microsoft-security-blog/microsoft-purview-data-risk-assessments-m365-vs-fabric/4487222">data risk assessment</a> on your Fabric estate in DSPM.</p><p><strong>Fabric platform and networking security</strong></p><p>At the platform and networking level, we have inbound and outbound network security. This enables organizations to restrict inbound public access and also manage connections between Fabric and its data sources in a secure and compliant manner. We will cover this in more detail in a section below, zooming in on each part of the diagram below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kn42mpsSzreodJwn_xBGqA.png" /><figcaption>Fabric networking</figcaption></figure><p><strong>Workspace and item-level security</strong></p><p>Next, we have workspace and item level access. Admins can configure workspace roles to limit the scope of access given to different user groups. <strong>Per-item permissions g</strong>rant access to individual lakehouses, warehouses, or reports, etc. without full workspace access.</p><ul><li><strong>Item sharing: </strong>Share specific items with users via link or direct assignment</li><li><strong>Read / ReadAll permissions: </strong>Fine-grained read access at item level</li><li><strong>Compute permissions: </strong>Permissions scoped to specific query engines (SQL endpoint, semantic model)</li><li><strong>Sensitivity label inheritance</strong>: Labels applied to source items flow to downstream consumers</li></ul><p><strong>OneLake security</strong></p><p>The last piece is OneLake security, which acts as the <strong>data-plane</strong> security model. It provides role-based access control enforced consistently across all Fabric compute engines and authorized third-party engines. This is where you apply the most granular access control. OneLake security provides folder-level, table-level, CLS, and RLS,. Ccompute engines security features add OLS and DDM. We will cover each in other blog posts — today we are focusing on a holistic security view of the platform.</p><h3>Fabric networking — inbound</h3><p>Fabric workspaces can be organized into public and private workspaces. This speaks to the inbound traffic — for <em>public workspaces, </em>inbound is allowed and for <em>private workspaces, </em>inbound traffic is not allowed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1DEO1eplFciEjYQEFNNuMA.png" /><figcaption>Inbound Fabric networking</figcaption></figure><p>For public workspaces, the best way to access the Fabric tenant is via <a href="http://app.fabric.microsoft.com/">app.fabric.microsoft.com</a>. This allows inbound traffic from any IP by default, but you can also configure an IP allowlist through public internet.</p><p>For private workspaces, we can do this at the <a href="https://learn.microsoft.com/en-us/fabric/security/security-workspace-level-private-links-set-up?tabs=fabric-portal">workspace-level</a> or the <a href="https://learn.microsoft.com/en-us/fabric/security/security-private-links-use">tenant-level</a>. In both scenarios when you expose the private endpoint, create a linked service either at the tenant level of workspace-level, and then you need to set up different dns private link zones to properly resolve the private IPs. In order to connect to fabric private workspaces, the user must be on a VM in Azure or accessing Fabric through private connectivity on-prem. Conditional formatters must then be done for the zones on prem dns to send to dns private resolver which resolved the private IPs. See below for the private dns zones that must be created depending on the resources being accessed, and if its at the tenant or workspace-level.</p><ol><li><strong>privatelink.analysis.windows.net</strong> — The core Analysis Services / Fabric API plane (inherited from Power BI and Azure Analysis Services)</li><li><strong>privatelink.pbidedicated.windows.net </strong>— The dedicated capacity compute plane (inherited from Power BI Premium)</li><li><strong>privatelink.prod.powerquery.microsoft.com</strong> — The Power Query mashup engine (separate Microsoft service)</li><li><strong>privatelink.fabric.microsoft.com </strong>— When using workspace-level private link</li></ol><p>I generally recommend using workspace-level PEs to be more granular and avoid issues. This way you can target specific prod workspaces for example. <strong><em>Make sure items in your workspace are supported before configuring workspace-level PEs. Also note that you cannot currently enable both inbound (workspace-level PE) and outbound access protection via the portal UI — it requires the REST API. See limitations and supported scenarios </em></strong><a href="https://learn.microsoft.com/en-us/fabric/security/security-workspace-level-private-links-support?tabs=fabric-portal-1%2Cfabric-portal-2%2Cfabric-portal-15%2Cfabric-portal-3%2Cfabric-portal-4%2Cfabric-portal-5%2Cfabric-portal-6%2Cfabric-portal-7%2Cfabric-portal-8%2Cfabric-portal-9%2Cfabric-portal-10%2Cfabric-portal-11%2Cfabric-portal-12%2Cfabric-portal-13%2Cfabric-portal-14"><strong><em>here</em></strong></a><strong><em>.</em></strong></p><h3>Fabric networking — outbound</h3><p>When it comes to outbound traffic, in each workspace type, we can have one of the following: Public, Fabric managed VNet, VNet data gateway, on-prem data gateway, and trusted workspace access.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v0ci5eKW2HQ4FMaO4IXxyA.png" /><figcaption>Outbound Fabric networking</figcaption></figure><ul><li><strong>Public: </strong>Fabric workloads connect to data sources over the public internet but can be restricted using firewall rules on the sources themselves (allowlist Fabric service tags or specific IPs or Azure sources).</li><li><a href="https://learn.microsoft.com/en-us/fabric/security/security-managed-vnets-fabric-overview"><strong>Fabric Managed VNet</strong></a><strong>: </strong>Use when connection to <a href="https://learn.microsoft.com/en-us/fabric/security/security-managed-private-endpoints-create">Azure PaaS resources</a> locked behind private endpoints or <a href="https://learn.microsoft.com/en-us/fabric/security/connect-to-on-premise-sources-using-managed-private-endpoints">on-prem sources</a>. All traffic flows over the Microsoft backbone. You need to configure outbound network security and Managed private endpoints to Azure resource which have to be approved by the target source. This is primarily for <em>Spark-based workloads (Notebooks, Spark Job Definitions, Lakehouses, Environments) and Eventstream (Event Hubs/IoT Hub only)</em>. Other workloads like Dataflow Gen2, Pipelines, and Mirroring use VNet DG instead.</li><li><a href="https://learn.microsoft.com/en-us/data-integration/vnet/overview"><strong>VNet Data Gateway</strong></a><strong>: </strong>Leverages power platform VNet service under the hood. Microsoft spins up a containerized gateway inside a delegated Power Platform subnet in your VNet. Ensure you have the Power Platform resource provider. Create a private endpoint for your data sources in another subnet in the same VNet. Configure VNet DG pointed to the delegated subnet.</li><li><a href="https://learn.microsoft.com/en-us/data-integration/gateway/service-gateway-install#download-and-install-a-standard-gateway"><strong>On-prem Data Gateway</strong></a><strong>: </strong>Not all services can connect using VNet data gateway so many customers using Oracle on VM for example or any on-prem sources need to install an agent this to connect to Fabric.</li><li><a href="https://learn.microsoft.com/en-us/fabric/security/security-trusted-workspace-access"><strong>Trusted workspace access</strong></a><strong>:</strong> Trusted workspace access allows you to access ADLS Gen2 accounts behind a firewall using trusted workspace identity.</li></ul><h3>Outbound network security use cases</h3><p>Let&#39;s go through 5 use cases which you can configure to each of the 5 outbound traffic methods (Public, Fabric managed VNet, VNet data gateway, on-prem data gateway, trusted workspace access).</p><ul><li><strong>Batch processing with Public outbound bringing in data from Azure SQL database:</strong> initiating a connection to Azure SQL (allow select networks and azure services/resource) from a Notebook or pipeline and writing to a Fabric lakehouse (connected to the notebook).</li><li><strong>Notebook connection to Azure SQL DB, writing to lakehouse, using Fabric managed VNet:</strong> Create a <a href="https://learn.microsoft.com/en-us/fabric/security/security-managed-private-endpoints-create">managed private endpoint</a> in your fabric workspace. Approve the managed private endpoint created from fabric in the networking settings of the Azure SQL DB. Connect to Azure SQL DB from the notebook and write to your lakehouse.</li><li><strong>SQL connection using dataflow Gen2 using VNet data gateway:</strong> Create a new gateway in <em>Manage connections and gateways</em> and create a new gateway to connect to your delegated subnet. Create a new dataflow Gen2, create a new connection and use the VNet Data gateway.</li><li><strong>Copy job in pipelines from Oracle using on-prem data gateway:</strong> create a <a href="https://learn.microsoft.com/en-us/fabric/data-factory/how-to-access-on-premises-data#create-an-on-premises-data-gateway">data gateway</a>, create a pipeline and a copy activity, <a href="https://learn.microsoft.com/en-us/fabric/data-factory/connector-oracle-database#set-up-your-connection-in-a-pipeline">create a new connection</a> to your oracle database through the on-prem data gateway. Ensure you copy the data raw as it is more performant to copy the raw data and then transform the data after it lands in the lakehouse.</li><li><strong>Shortcut from ADLS Gen2 using trusted workspace access</strong>: create a workspace identity, ensure you workspace identity has <em>Blob Data Contributor, Storage Blob Data owner, or Storage Blob Data Reader role </em>on your storage account, configure a resource instance rule so that specific Fabric workspaces access your storage account based on their workspace identity. Now you can create a new shortcut from the files section of your lakehouse. You can do this to a folder or individual file. See <a href="https://learn.microsoft.com/en-us/fabric/security/security-trusted-workspace-access#configure-trusted-workspace-access-in-adls-gen2">here</a> for a walkthrough.</li></ul><h3>Summary</h3><p>There are a lot of important pieces to absorb in the article so let me attempt to summarize some key take-aways. Microsoft Fabric is you unified data solution where you can enforce multiple layers of security and governance to ensure your most critical data assets are being access by defined users and groups. Leverage the following layers:</p><ol><li>Governance, security and compliance for data security and AI/Agent prompt monitoring with Microsoft Purview</li><li>Capacity boundaries to enforce compliance, governance and performant solutions</li><li>Domain boundaries to implement federated architecture</li><li>Workspaces and item-level security</li><li>OneLake security at the sub-item, table and folder-level</li><li>Network and platform-level security — both inbound and outbound access</li></ol><p>This article can help serve as a baseline for when you’re designing and deploying your secured and unified data estate. In subsequent articles we can dive deeper into OneLake security and workspace/item-level security.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb525b0d83d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your agents are only as secure and accurate as your data]]></title>
            <link>https://medium.com/@alipouw/your-agents-are-only-as-secure-and-accurate-as-your-data-c060b51a435a?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/c060b51a435a</guid>
            <category><![CDATA[ai-agent-governance]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[cloud-security]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <category><![CDATA[microsoft-purview]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Fri, 03 Apr 2026 19:40:39 GMT</pubDate>
            <atom:updated>2026-04-03T19:40:39.669Z</atom:updated>
            <content:encoded><![CDATA[<p>After another brief hiatus, we are back. Between FabCon (key announcements <a href="https://blog.fabric.microsoft.com/en-us/blog/fabric-march-2026-feature-summary?ft=All">here</a>), marathon training (pray for me on April 20th) and never-ending releases in the AI space (truly never ending), it’s been hard to keep up quite honestly. I’ve been wanting to write something like this for a little while now, as it builds on some of my previous articles on Data and AI governance. Today I wanted to share how I’ve been working with customers to build a<strong><em> secure and governed data foundations for AI</em></strong>, as well as some <strong><em>new developments in the security space with Microsoft Purview and Microsoft Fabric</em></strong>.</p><p>When we think of AI applications, many of us think of agents nowadays. Usually, we’re building agents for our personal use, or maybe even for yourself and your coworkers. When it comes to taking your agents to the enterprise level and making them available across your entire organization or even to your customers, this requires an additional level of planning, especially when it comes to regulated industries. We need to know how to scale these agents in a secured and governed way so something like, oh I don’t know, the<a href="https://www.cnbc.com/2026/03/31/anthropic-leak-claude-code-internal-source.html"> entire Claude codebase being leaked overnight</a> doesn’t happen to your organization.</p><h3>Common implementation challenges</h3><p>Many customers are excited about building agents and the productivity gains they will get from agents, but when it comes to org-wide implementation, two of the most common issues are:</p><ol><li>Solutions are not designed to connect to data sources at scale</li><li>Most data is consolidated in disparate sources which creates additional silos</li></ol><p>So first, I want to build agents, but from there how do I scale it? The anti-pattern that we always tell customers is <strong><em>don’t build agents directly to these different data sources</em></strong>. There is a lot of disjointed data to be used to build agents, applications, BI reports, or any analytic solutions, and if you point your agents to these, you’re never going to see economies of scale. This is one point where (almost) everyone is failing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jidKmqss9W93EpB3b8t3Uw.png" /><figcaption>Point solutions</figcaption></figure><p>This is where the common semantic layer or business ontology comes in. If we’re thinking from a Microsoft lens, this is Fabric IQ (context on business entities, systems of record and actions), Foundry IQ (context on policies, authoritative documents, and knowledge bases) and Work IQ (context on people, collaboration and day-to-day workflows). We will focus on Fabric IQ for the continuation of this article as it helps bring intelligence to your operational data.</p><p>The second area where organizations struggle is with vast amounts of siloed data, often with (at least) duplicate copies. And I mean this makes sense for long standing companies — they’ve had decades to collect data in an evolving data landscape and now we throw AI at it and the legacy systems can’t quite keep up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zuFEqFQsehJ5sIJqIVkLKg.png" /><figcaption>Data sprawl</figcaption></figure><p>Now recognizing that the data landscape is incredibly complex and it&#39;s hard to modernize at the flick of a switch. This doesn’t get around the problem that it’s really hard to get good insights when data is siloed and when we’re not sure when it&#39;s a complete picture of the business. So how do we get around this?</p><p>To unlock real value from the growing volume of data being collected, your data and analytics capabilities need to keep pace with real-time demands. That means moving away from a fragmented technology stack that slows down complex data processing and depends on specialized skills and tooling. Instead, data needs to be unified in a single, integrated platform with built‑in capabilities that make insights accessible to employees at every skill level. This is the first step to making your data estate AI‑ready: cleaning, unifying, and modernizing data to support scalable, responsible AI innovation.</p><h3>Implement a durable data foundation</h3><p>Now let’s get into the details and how I typically approach this with customers. We’ve all heard this one before, but AI is only as good as the data it sits on. There are four areas that every organization needs when implementing AI at scale:</p><ol><li>Data governance, security and compliance</li><li>Unified data</li><li>Efficient data management</li><li>Data intelligence</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oGqLVbOHXOVXQJR_JqCIIw.png" /></figure><p>Multiple systems feed into the unified data layer which is the data you need to build your solutions. The layer in between, the data management layer, needs to be automated to gain the most efficiency and insight. How do you get data to the ecosystem? How do you get data to the different business units? Then from there, how do we add a common intelligence layer and make sure that governance security compliance will basically, tie all of these layers together?</p><p>Tools such as Microsoft Fabric help in unifying your vast data estate with OneLake. Developers are able to leverage shortcuts, mirroring, pipelines, etc. to bring data together into this unified foundation. Once the data is unified then developers can leverage tools such as notebooks, pipelines, Copilot, GitHub Copilot, etc to get that data into a consumable format that’s ready for AI. At this point your business SMEs can help build intelligence into the data using ontologies, we can digitize your real-world business operations using digital twin, and we can leverage Graph to visualize billions of relationships. From there extract the intelligence with agents, no matter where they are built — whether that’s data agents or operations agents inside of Fabric, or agents in Foundry, Copilot Studio or even M365 Copilot.</p><p>See <a href="https://github.com/alipouw13/insurance-FabricIQ">here</a> for an example you can deploy comparing data agents&#39; responses to the same set of questions. One agent is sitting on raw eventhouse and lakehouse data; one is sitting on a semantic model; and one is sitting on a business ontology. Spoiler, the ontology agent is able to respond most appropriately to 90% of the questions posed.</p><p>This is all done within the guardrails defined by your centralized IT team. These teams can leverage tools such as Microsoft Purview to implement guardrails around who can access which data. Core features for your structured data include:</p><ol><li><strong>Information Protection (IP):</strong> Set up default and mandatory sensitivity labelling policies for all your assets inside of Microsoft Fabric. This allows you to apply DLP policies on this data to ensure sensitive data isn’t leaked.</li><li><strong>Data Loss Prevention (DLP):</strong> apply policies to automatically protect sensitive information once it is detected for all structured assets, including lakehouses, semantic models, mirrored databases, warehouses (preview), KQL (preview) and SQL databases (preview).</li><li><strong>Insider Risk management (IRM):</strong> identify places where users may have, intentionally or inadvertently, shared sensitive data or bypassed security policies. Leverage the new <em>quick policy creation of the Data Theft rule for Fabric</em><strong> </strong>to quickly set up protection against data exfiltration scenarios.</li><li><strong>DSPM for AI:</strong> now monitoring interactions of Copilots in Fabric and data agents (preview — should be out next week). Ability to run data risk assessments to identify potential oversharing risks in your organization (preview).</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1aX9Hw8NdWjxWG1BExxnQg.png" /><figcaption>DSPM for AI — data risk assessment (preview)</figcaption></figure><h3>Putting it all together</h3><p>Your agents will be useless unless they have the right data and business context. Agents not only need access to clean data but also need guardrails on the data in which they access. If we think of the foundations that we need to build scalable, and useful agents, we need: unified data; proper data management; and data intelligence to give business context to the data. With all of this wrapped with proper security, compliance and governance. Microsoft Fabric and Microsoft Purview build the unified and secured foundation that these agents need in order to integrate into your existing business processes and drive material value at scale.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c060b51a435a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Microsoft Fabric: From unified data platform to enterprise intelligence for AI]]></title>
            <link>https://medium.com/@alipouw/microsoft-fabric-from-unified-data-platform-to-enterprise-intelligence-for-ai-23f534d4a5c4?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/23f534d4a5c4</guid>
            <category><![CDATA[ontology]]></category>
            <category><![CDATA[intelligence-layer]]></category>
            <category><![CDATA[unified-data-platform]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Fri, 06 Mar 2026 18:26:15 GMT</pubDate>
            <atom:updated>2026-03-06T18:26:15.665Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s come to my attention that I haven’t yet done a deep dive on core Microsoft Fabric functionalities that help support enterprise AI applications. Most of what you hear is that Fabric is Microsoft’s SaaS solution for unifying your entire data estate and making it ready for your AI applications. This is true, but incomplete. Fabric isn’t just about unifying data. It’s about turning unified data into governed, business-aligned intelligence that can power enterprise AI systems securely and at scale.</p><p>This article goes deeper into how Fabric moves from data platform to intelligence platform, and how its semantic layer (often referred to as Fabric IQ capabilities) enables AI-ready data grounded in business meaning.</p><h3>Data without intelligence isn’t AI-ready</h3><p>Many customers that I talk to have a similar story:</p><ul><li>Data exists everywhere: across multi-cloud, on-prem warehouses, lakehouses, streaming platforms, Power BI models</li><li>Business definitions are inconsistent: teams create their own definitions of “customer” or “revenue” and none of them match</li><li>AI models and LLM applications lack grounding and consistent context</li><li>Governance is an afterthought rather than built-in</li></ul><p>Being AI-ready means your data is unified and has context, relationships, governance by design, and secure data access patterns. This is the gap Fabric is designed to close.</p><h3>From Data Platform to Intelligence Platform</h3><p>At its foundation, Microsoft Fabric is built on OneLake: your logical data lake (think OneDrive for data), providing a unified and secure data foundation for the intelligence layer. All workloads store and access data through OneLake with zero-copy access patterns; Copilot is embedded across workloads; and governance is supported through Microsoft Purview, both natively within Fabric and via the stand-alone Purview experiences for broader data estate visibility.</p><p>In my previous article on <a href="https://medium.com/@alipouw/data-mesh-in-the-age-of-ai-a97235f8bf32">Data Mesh</a>, I covered how OneLake acts as a “data mesh as a service.” Once your bring your data to OneLake, whether that&#39;s mirroring from Databricks Unity Catalog, shortcutting Iceberg tables from Snowflake, or connecting to S3 buckets — domains enable decentralized ownership and governance, while OneLake maintains centralized storage.</p><p>Getting the unified foundation is step one. Unified storage doesn’t create intelligence. The real value creation happens when the unified data is expressed in the language of your business.</p><h3>Fabric IQ: The Semantic Foundation</h3><p>As we’ve touched on for previous articles, Fabric IQ is a workload that brings together multiple semantic and intelligence capabilities to organize data across OneLake according to the language of your business. This consolidation of data with the appropriate business context is then exposed to analytics, AI agents, and applications with consistent semantic meaning. Fabric introduces a semantic-first approach to organizing your enterprise data.</p><p>IQ brings together six items that can be used individually or that, together, make up and enhance the semantic layer of your organization:</p><ul><li><strong>Semantic models:</strong> create an Ontology directly on semantic models power your most critical business reports</li><li><strong>Ontology:</strong> the enterprise vocabulary, including entity types, properties, relationships, and constraints, bound to real data</li><li><strong>Digital Twin Builder:</strong> The ability to create a digital representation of your business and operational assets through Ontology</li><li><strong>Graph:</strong> graph storage and compute for relationship traversals and graph algorithms using the GraphQL</li><li><strong>Data Agent:</strong> Conversational Q&amp;A systems using GenAI, grounded in your business concepts and data</li><li><strong>Operations Agent:</strong> AI agent that monitors real-time data and recommends or takes business actions</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b4_OwpO5AOZRmcRt21Yocg.png" /><figcaption>Microsoft Fabric: Unified Data and Intelligence for AI</figcaption></figure><p><strong>Ontology (preview): defining the language of your business</strong></p><p>The Ontology is the heart of IQ. It’s a shared, machine-understandable vocabulary: entity types (<em>Customer, Order, Shipment</em>), properties (a Customer’s name, email), relationships (<em>Customer </em>places <em>Order</em>), and data bindings that link concepts to actual OneLake data sources.</p><p>Define <em>Customer </em>once in the ontology, and every downstream tool uses the same definition. No more conflicting column-level definitions across teams. The ontology also includes a natural language-to-ontology query layer where you’re able to ask business questions in plain English and the system routes to the appropriate query engine based on available bindings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0bxPXPgZeaRKd5HJ3PnTCw.png" /><figcaption>Ontology item in Microsoft Fabric</figcaption></figure><p><strong>Semantic Models: from BI logic to ontology</strong></p><p>Most people are familiar with Semantic Models in Microsoft Fabric. They already define business entities, measures, hierarchies, and relationships across your data estate. So rather than rebuilding business meaning from scratch, organizations can leverage existing semantic models as a foundation to generate and extend their ontology. This allows entity types, relationships, and business logic defined in Power BI to inform broader ontology structures used by Graph, Digital Twin Builder, and Agents. This results in better continuity, where your trusted BI definitions become a reusable enterprise knowledge layer to power AI systems, operational intelligence, and digital twins. Think of Fabric IQ as the basis to elevate your existing BI knowledge into an ontology-driven intelligence framework.</p><p><strong>Digital Twin Builder (preview): digitize your environment to optimize operations</strong></p><p><a href="https://learn.microsoft.com/en-us/fabric/real-time-intelligence/digital-twin-builder/overview">Digital Twin Builder</a> in Microsoft Fabric allows organizations to create a governed, ontology-driven digital representation of their physical and operational environments. By modeling assets, processes, and business entities as connected concepts, then binding them to live data in OneLake , teams gain a structured view of how their operations actually function. This enables deeper analysis, performance monitoring, and early risk detection across these complex systems. Instead of reacting to operational issues, organizations can proactively optimize performance, reduce costs, and scale with more confidence.</p><p><strong>Graph (preview): model &amp; analyze your Ontology</strong></p><p>Enterprises are complex networks that require complex modelling to extract as much insight from as possible. <a href="https://learn.microsoft.com/en-us/fabric/graph/overview">Graph </a>in Microsoft Fabric runs directly on OneLake data without the need for duplication or ETL. Graph allows you to model, visualize, and analyze complex relationships across this data by using <a href="https://learn.microsoft.com/en-us/fabric/graph/gql-language-guide">ISO GQL standard (ISO/IEC 39075)</a> graph query language, alongside GraphQL APIs for external application access.</p><p>It is built to scale, handling large workloads of billions of relationships, and turns disconnected data into AI-powered insights. The ontology declares which things connect and why. While Graph provides an efficient way to view and query these relationships.</p><p>Add <a href="https://learn.microsoft.com/en-us/fabric/data-engineering/api-graphql-overview">Fabric API for GraphQL</a>, which auto-generates typed API endpoints over lakehouses, warehouses, and mirrored databases in minutes, and your application developers get a clean, programmable endpoint without learning Fabric internal plumbing.</p><p><strong>Data Agent (preview): surface insights from your data</strong></p><p>The Data Agent sits on your ontology (or other data) and lets anyone with access ask plain-English questions and get governed, context-aware answers in return. It identifies the right data source (lakehouse, warehouse, Power BI, KQL database, or ontology), generates the appropriate query (SQL, DAX, KQL, or ontology query), validates, executes, and returns human-readable results.</p><p>The key is to connect your Data Agents to your ontology as a source. Agents understand your business concepts, not just your tables and schemas. You can further extend their impact to Microsoft Foundry Agent Service and make Fabric the knowledge backbone for your agentic applications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qUSs7caJ10RO18zLyqPLSw.png" /><figcaption>Data Agent sitting on top of Loan Ontology which combines lakehouse and eventhouse data</figcaption></figure><p><strong>Operations Agent (preview): policies to proactive action</strong></p><p>The <a href="https://learn.microsoft.com/en-us/fabric/real-time-intelligence/operations-agent">Operations Agent</a> in Microsoft Fabric helps turn realtime data into intelligent through the following cycle: automate, observe, analyze, make decisions and act. Configure business goals, instructions, knowledge sources (real-time KQL data), and actions (connected to Power Automate flows). When conditions match, the agent sends a Teams message with context and a recommended action. You approve or reject, and the agent takes actions vis a governed workflow.</p><p>Think policy enforcement at lightning-fast speed: your critical operations are all continuously monitored with proactive recommendations delivered where your team works.</p><h3>Bringing it all together</h3><p>Here’s how I think about Fabric’s architecture as the foundation for an intelligent AI data platform:</p><ul><li><strong>Unified Storage (OneLake):</strong> Get all your data in one place. Apply data mesh principles: organize into domains, delegate governance, build data products.</li><li><strong>Unified Semantics (Fabric IQ):</strong> Model your data in business language. Define entity types and relationships once. Every tool speaks the same language.</li><li><strong>Unified Intelligence (Agents): </strong>Data Agents decipher insights. Operations Agents take policies and make proactive mitigations and decisions.</li><li><strong>Unified Applications (GraphQL + Foundry):</strong> Expose data through GraphQL for developers. Connect to Foundry where you build and deploy AI applications at scale.</li></ul><p>Fabric is all about data intelligence, bringing together data and analytics, and getting intelligence. Combine with Foundry where you orchestrate and build AI applications. Bring it together and you have a complete pipeline from raw data to business action.</p><h3>Closing</h3><p>Fabric is evolving beyond a data platform and becoming an intelligence platform. Fabric IQ is the semantic foundation that makes this shift possible. If you’ve followed my data mesh article, think of IQ as the natural next step: data mesh defines your domain ownership and data products. Fabric IQ gives those data products a shared vocabulary, relationship-awareness, and agent-readiness that turns organized data into enterprise intelligence.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23f534d4a5c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Databases built for AI and agentic applications]]></title>
            <link>https://medium.com/@alipouw/databases-built-for-ai-and-agentic-applications-cca8d873be8a?source=rss-96f4fa37ea38------2</link>
            <guid isPermaLink="false">https://medium.com/p/cca8d873be8a</guid>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[ai-database]]></category>
            <category><![CDATA[microsoft-azure]]></category>
            <dc:creator><![CDATA[Alipouw]]></dc:creator>
            <pubDate>Fri, 20 Feb 2026 15:19:05 GMT</pubDate>
            <atom:updated>2026-02-20T15:19:05.839Z</atom:updated>
            <content:encoded><![CDATA[<p>Last week we highlighted <a href="https://medium.com/@alipouw/horizondb-private-preview-cloud-native-postgresql-for-ai-applications-ad26189d6023">HorizonDB </a>— Microsoft&#39;s latest PostgreSQL database, ideal for building AI applications at scale. This week we’re going to dive a bit deeper into <strong>how to optimize your databases for AI</strong> and <strong>how to choose the right database for each AI workload.</strong></p><p>Data is at the core of getting the most benefit from your AI applications, yet only 19% of companies have actually harnessed the true value of their data (<a href="https://www.gartner.com/en/articles/how-your-ceo-is-thinking-about-ai">Gartner, 2025</a>). At the same time, the data landscape keeps on growing faster than ever before. Many of our customers face some critical considerations when determining how to best leverage their data for AI:</p><ul><li>Existing infrastructure and type of data</li><li>Upskilling requirements and operational familiarity</li><li>Cost and scaling</li><li>Performance requirements: latency, throughput and accuracy</li></ul><p>Given this new reality, in the world of Microsoft, where do we start?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-vdsrZTXD_ZSGNgIpmxRdw.png" /><figcaption>Building AI applications</figcaption></figure><p><strong>Lets start simple </strong>— as we shift applications from deterministic (“doing”) systems to probabilistic (“understanding”) systems, our databases need to support a new set of requirements:</p><ul><li>The capacity to handle data and AI at scale</li><li>The ability to ingest, process and serve data to applications and models in real-time</li><li>Native support for vector storage and retrieval</li><li>Tight integrations across analytics workloads and AI tooling for enhanced insight</li></ul><p>Before getting into the weeds, remember that when it comes to AI applications you almost always want to build your vector index where your data already lives. Keeping vectors and data together:</p><ul><li>Simplifies application architectures</li><li>Simplifies data management and reduces data movement</li><li>Reduces latency and improves the efficiency of vector-based operations</li><li>Improves and simplifies governance and security</li><li>Lowers operational overhead</li></ul><p>If your data lives in SQL, lets leverage SQL 2025 vector capabilities. If it lives in Cosmos , let’s build the vectors there. You have a bunch of blobs? Let&#39;s use AI Search as your central vector store. No matter where the data lives, there is an optimal solutions — and that&#39;s to keep the data where it is, and modernize from there.</p><p>That said, let’s walk through the core AI-ready databases in the Microsoft ecosystem to provide context on each, when to use each and when to modernize vs build new. We will cover: Azure AI Search, SQL Engine DBs, CosmosDB, DocumentDB, and PostgreSQL (including HorizonDB).</p><h3>Azure AI Search</h3><p>Azure AI Search is Microsoft’s fully managed enterprise vector and retrieval engine. It is well suited when your primary challenge is information discovery, not transactional persistence. Some core scenarios include:</p><ul><li>Documents stored in <strong>SharePoint or OneDrive</strong></li><li>Documents from external or third-party sources (eg: News articles, web content, other clouds)</li><li>Scenarios where s<strong>ecurity boundaries or compliance requirements</strong> mandate a dedicated search index rather than embedding vectors inside an operational database</li></ul><p>AI Search is the best engine for hybris retrieval (keyword &amp; vector retrieval), semantic ranking, and query-time grounding for agents.</p><p><strong>Modernize vs. Build New</strong></p><ul><li><strong>Modernize </strong>when you already have large document estates and need to make them AI‑accessible without restructuring core systems</li><li><strong>Build new</strong> when search is the application (knowledge assistants, research copilots, enterprise Q&amp;A)</li></ul><h3>SQL Engine (Azure SQL DB, SQL MI, SQL Server, SQL 2025, and Microsoft Fabric SQL)</h3><p>The SQL engine is one of the most widely used databases in the world and as such, is one of the most pragmatic vector databases available since your data is already there. AI capabilities for the SQL engine now include:</p><ul><li>Built-in vector types, embeddings, and semantic and hybrid search</li><li>Built-in chunking and embedding</li><li>Agentic framework integration (via <a href="https://learn.microsoft.com/en-us/azure/data-api-builder/mcp/overview">SQL MCP server</a>)</li><li>Model management with SQL managed identity authentication</li><li>Strong security and governance: identity, auditing, ledger, and access control</li><li>Deep integration with Fabric, Foundry, and analytics tooling</li></ul><p>Given SQL is the system of record for many enterprises, when AI needs to reason over your structured, governed and transactional data, SQL is often the right answer. Keep your vectors close to your data to reduce latency, create a more simplistic design; and leverage the skills your developers already have to overcome complexity. If the following is true, look to SQL as your AI database:</p><ul><li>Your data already lives in SQL</li><li>You want to minimize architectural complexity</li><li>You need ACID guarantees, auditing, and deterministic behavior alongside AI</li><li>You want agents reasoning over trusted, structured data</li></ul><p><strong>Modernize vs. Build New</strong></p><ul><li><strong>Modernize </strong>existing SQL workloads with vector search and RAG instead of exporting data</li><li><strong>Build new</strong> AI‑native apps when structured reasoning, compliance, and explainability matter</li></ul><h3>Azure CosmosDB</h3><p>When I think of an AI-ready databases, the first thing that comes to ming is CosmosDB for NoSQL. Open AI leverages Cosmos for ChatGPT for high-scalability, low‑latency conversational and operational workloads. Cosmos is unique in combining:</p><ul><li>Global distribution with infinite scale</li><li>&lt;10ms latency</li><li>99.999% HA read and write SLA</li><li>Schema-flexible documents and native vector storage</li></ul><p>With dynamic scaling it becomes more cost efficient for any workload. When it comes to storing vectors, your schema-free data is stored alongside multi-modal high-dimensional vectors just as are other properties of the documents.</p><p>When we think of AI use cases for Cosmos, think of the following:</p><ul><li><strong>Operational and vector database:</strong> keep operational and vector data in one place to save on cost and complexity. Think shardable and write-heavy workloads</li><li><strong>AI agent long-term memory: </strong>optimize agentic application responses, manage threads, enable auditing and ensure scale for multi-agent solutions</li><li><strong>RAG for data living in Cosmos:</strong> you have data already in Cosmos? Create more accurate and contextual responses on your data by leveraging vector indexing</li><li><strong>Multi-agent coordination at a global scale:</strong> ensure your agentic applications can scale globally with quick response times</li></ul><p><strong>Modernize vs. Build New</strong></p><ul><li><strong>Modernize</strong> when Cosmos already powers your app: add vectors and embeddings rather than additional systems</li><li><strong>Build new</strong> for globally distributed, real‑time, agent‑driven experiences where latency and availability are the priority</li></ul><h3>Azure DocumentDB</h3><p>Azure DocumentDB (formerly Cosmos for Mongo vCore) is a Mongo-DB compatible and open-source database, optimized for AI workloads. DocumentDB has:</p><ul><li>Up to 50% lower TCO than Mongo Atlas</li><li>Familiar open-source MongoDB APIs and tools</li><li>Built-in DiskANN vector search</li><li>99.995% HA SLA</li><li>Cost savings with independent compute and storage scaling, free backups and automatic scaling.</li></ul><p>If you’re looking to reduce TCO and want a Mongo compatibility without Mongo lock-in with AI-powered document workloads, then I suggest testing out Azure DocumentDB.</p><p><strong>Modernize vs. Build New</strong></p><ul><li><strong>Modernize </strong>existing MongoDB apps with native vector search</li><li><strong>Build new </strong>when you want document‑centric AI apps with flexible schema and predictable cost</li></ul><h3>PostgreSQL and HorizonDB</h3><p>Azure database for PostgreSQL goes beyond a managed open-source database and leverages manay AI capabilities, including:</p><ul><li>GenAI powered reasoning</li><li>Native vector search (pgvector &amp; DiskANN)</li><li>Native OpenAI integrations</li><li>Data mirroring to Microsoft Fabric to drive deeper AI insights</li></ul><p>When we think of PostgreSQL, think read‑heavy, relational workloads that benefit from extensibility, strong SQL semantics, and the open‑source community. Open AI recently published an <a href="https://openai.com/index/scaling-postgresql/">article</a> highlighting how they use Postgres to support massive read-heavy workloads to for 800 million ChatGPT users. This reinforces the strength of PostgreSQL in the era of AI.</p><p>As we covered last week, <a href="https://medium.com/@alipouw/horizondb-private-preview-cloud-native-postgresql-for-ai-applications-ad26189d6023">HorizonDB (private preview)</a> brings us everything you love about PostgreSQL and supercharges it with scale, efficient storage, and native advanced vector capabilities with DiskANN indexing. Over the next year as this database nears GA, I suggest testing out private preview and later public preview if you have any existing PostgreSQL workloads — the scale, efficiency and vectorization capabilities of HorizonDB are promising and will serve to modernize your existing PostgreSQL supported applications.</p><p><strong>Modernize vs. Build New</strong></p><ul><li><strong>Modernize </strong>existing PostgreSQL apps with vector and semantic capabilities — consider HorizonDB if your application will benefit from scale and efficiency</li><li><strong>Build new</strong> when you want a managed, open-source, cloud-native database, that benefits from low cost with cloud‑scale performance for AI‑native workloads</li></ul><h3>Summary</h3><p>Hopefully this provided some peace of mind that, yes, there are so many AI database options out there (and many that I didn’t cover), but deciding on your AI database become simple with the following questions:</p><ol><li>Where does your data live?</li><li>Do I need to modernize an existing application or building net new?</li><li>What am I optimizing for? Cost, latency, accuracy, scale, developer familiarity?</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*snfhkFBskRGFe6Lxi165zA.png" /><figcaption>Database strategy</figcaption></figure><p>If you know the answer to the above and review the databases described in this article, then the database selection becomes clearer. Your AI applications are powered by your enterprise data sitting in your many databases. Start with identifying a simple application to modernize — explore the database’s native capabilities; get your developers familiar with the new database tools and features; evaluate the results; and iterate and expand.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cca8d873be8a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>