<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pipelines Down Under on Medium]]></title>
        <description><![CDATA[Stories by Pipelines Down Under on Medium]]></description>
        <link>https://medium.com/@barbaramalafaia?source=rss-16343300048a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*iBFm-_cM9TrEm8lc1E5hWw.jpeg</url>
            <title>Stories by Pipelines Down Under on Medium</title>
            <link>https://medium.com/@barbaramalafaia?source=rss-16343300048a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 07:27:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@barbaramalafaia/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Trigger a Power BI Semantic Model Refresh from a Synapse Pipeline]]></title>
            <link>https://medium.com/@barbaramalafaia/how-to-trigger-a-power-bi-semantic-model-refresh-from-a-synapse-pipeline-75e78fc2db39?source=rss-16343300048a------2</link>
            <guid isPermaLink="false">https://medium.com/p/75e78fc2db39</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[azure-synapse-analytics]]></category>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[azure-data-factory]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <dc:creator><![CDATA[Pipelines Down Under]]></dc:creator>
            <pubDate>Mon, 05 May 2025 10:25:05 GMT</pubDate>
            <atom:updated>2025-05-05T10:25:05.146Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Refreshing a Power BI dataset (semantic model) from Azure Synapse Analytics is a common need in data engineering workflows, especially when transformations and data loads are orchestrated within Synapse pipelines. However, it’s not as straightforward as just calling an API. You need to correctly configure permissions, service principals, and make the right REST API calls securely from your Synapse workspace.</p><p>This post walks you through the entire process — step-by-step — from configuring Azure AD permissions to calling the Power BI REST API from within a Synapse pipeline. By the end, you’ll have a reliable, secure, and automated method for refreshing your datasets as part of your end-to-end data pipeline.</p><h3>Step 1 — Create a Cloud-Based Security Group</h3><p>Power BI requires the identity triggering the refresh to be a recognized service principal with the right permissions. Since Synapse Managed Identity is considered a service principal, it must be added to a <strong>cloud-based</strong> Azure security group.</p><blockquote>On-premises groups <strong>do not</strong> work — the Managed Identity can only be added to Azure AD (cloud) groups.</blockquote><p><strong>Group created</strong>: SG-CLOUD</p><h3>Step 2 — Add the Synapse Managed Identity to the Security Group</h3><h4>Identifying the Managed Identity</h4><p>There are two ways to find your Synapse Managed Identity Object ID:</p><p><strong>Option 1:</strong></p><ul><li>Go to Synapse Workspace &gt; Overview</li><li>Copy the <strong>Managed Identity Object ID</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*2nWgKGBioGFa4R5TKB1I2Q.png" /><figcaption>Identifying the Managed Identity (overview page)</figcaption></figure><blockquote>💡 <em>Note: The Synapse Managed Identity is also a Service Principal and can be found under</em><br> <strong>Microsoft Entra ID</strong> &gt; <strong>Enterprise Applications</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4Hs_dtX7h64qYu4cylE5Vw.png" /></figure><p><strong>Option 2:</strong></p><p>Open Synapse Studio &gt; Manage &gt; Credentials</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*qCJSg2UFZ7dqu7A7nlFTEw.png" /></figure><p>Open the listed <strong>Managed Identity</strong>, copy the <strong>Managed Identity Object ID </strong>and the <strong>Managed Identity Name</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kR74qhQxE5mKrKmNkBjd1A.png" /></figure><h4>Add the Managed Identity to the Security Group via PowerShell</h4><pre>$spn = Get-AzureADServicePrincipal -SearchString &quot;Managed Identity Name&quot;<br>$group = Get-AzureADGroup -SearchString &quot;SG-CLOUD&quot;<br>Add-AzureADGroupMember -ObjectId $($group.ObjectId) -RefObjectId $($spn.ObjectId)</pre><p><em>Replace “Managed Identity Name” and “SG-CLOUD” according to what you have in your environment.</em></p><p>⚠️ If you get an error “<em>Unable to update the specified properties for on-premises mastered Directory Sync objects or objects currently undergoing migration</em>”, it’s likely because the group is on-premises. Ensure you’re adding the Managed Identity to a <strong>cloud-based</strong> group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2_zBKFI49FiFUrOn2qQ8SQ.png" /></figure><h3>Step 3 — Grant Power BI API Permissions to the Group</h3><p>The Synapse Managed Identity is a <strong>service principal</strong>, so it must be explicitly allowed to interact with the Power BI APIs.</p><h4>In the Power BI Admin Portal:</h4><ol><li>Enable API access for the group:</li></ol><ul><li>Go to <strong>Tenant Settings</strong></li><li>Under <strong>Developer settings</strong>, find <strong>“Service principals can use Fabric APIs”</strong></li><li>Enable it and Apply to &gt; Specific groups &gt; Add the group: SG-CLOUD (replace it according to the group you created in your environment)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AHX4yDoaHJ1McPxH1AH6fQ.png" /></figure><p>2. Enable read-only admin APIs:</p><ul><li>Go to <strong>Tenant Settings</strong></li><li>Under <strong>Admin API settings</strong>, enable <strong>“Service principals can access read-only admin APIs”</strong></li><li>Enable it and Apply to &gt; Specific groups &gt; Add the group: SG-CLOUD (replace it with the security group you created)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sYssBDYpiNw1UXHWRF_mIA.png" /></figure><h3>Step 4 — Add the Managed Identity to the Power BI Workspace</h3><p>To trigger a dataset refresh, the Managed Identity must be <strong>at least a member</strong> of the Power BI workspace. Adding it via UI is not supported, so use PowerShell:</p><pre># Sign in to Power BI<br>Login-PowerBI<br><br># Set the Object ID of the Managed Identity<br>$SPObjectID = &quot;86XXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX70&quot;<br><br># Get the workspace by name<br>$pbiWorkspace = Get-PowerBIWorkspace -Filter &quot;name eq &#39;Workspace Name&#39;&quot;<br><br># Add the Managed Identity as a member<br>Add-PowerBIWorkspaceUser -Id $($pbiWorkspace.Id) -AccessRight Member -PrincipalType App -Identifier $($SPObjectID)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*el8eE9uPPR8scZzDhfOrpg.png" /><figcaption>Service Principal (Synapse Managed Identity) added to Power BI workspace after running the Power Shell script</figcaption></figure><p>❌ Note: Adding the security group instead of the service principal <strong>does not</strong> work for dataset refresh via API. Always add the <strong>Managed Identity (Service Principal)</strong> directly.</p><p>Now that permissions are correctly configured, you’re ready to integrate the refresh process into your Synapse pipeline using a Web activity to call the Power BI REST API.</p><h3>Step 5 — Trigger the Power BI Dataset Refresh via Synapse Pipeline</h3><p>Now that your Synapse Managed Identity has the correct permissions, you can trigger the Power BI dataset refresh by using a <strong>Web activity</strong> in your Synapse pipeline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*juaSa53fJTG7RD5HOFpt6g.png" /></figure><p>Here’s how:</p><h4>1. Create a Web activity in your pipeline</h4><p>Drag a <strong>Web</strong> activity onto your pipeline canvas.</p><h4>2. Configure the Web activity</h4><p>URL:</p><pre>@concat(&#39;https://api.powerbi.com/v1.0/myorg/groups/&#39;,pipeline().parameters.WorkspaceGuidId,&#39;/datasets/&#39;,pipeline().parameters.DatasetGuidId,&#39;/refreshes&#39;)</pre><p>Remember to add{WorkspaceGuidId} and {DatasetGuidId} in the Pipeline parameters.</p><p><strong>Method:</strong> POST<br><strong>Authentication:</strong><br>Type: Managed Identity<br>Resource: <a href="https://analysis.windows.net/powerbi/api">https://analysis.windows.net/powerbi/api</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/516/1*Z8K0qmnRgStVCd8ufdhe_w.png" /></figure><p>🧠 Tip: You can get WorkspaceGuidId and DatasetGuidId from the Power BI service URL when you&#39;re viewing the dataset, or use the Power BI REST API to list them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*enFy3WNwxEi2Bpbllu94ag.png" /><figcaption>WorkspaceId and DatasetId in the Power BI service URL</figcaption></figure><h4>3. (Optional) Add a Loop to Monitor Dataset Refresh Status</h4><p>The Power BI <strong>refresh endpoint is asynchronous</strong>, meaning it triggers the refresh but doesn’t wait for it to complete. To ensure your Synapse pipeline only finishes <strong>after the dataset refresh is done</strong>, you can configure a loop that <strong>polls the refresh status</strong>.</p><p>Here’s how:</p><ul><li><strong>Create a variable</strong> named RefreshStatus with a default value of &quot;Unknown&quot;.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*1A7LpelsqyVG-W_agu-svQ.png" /></figure><ul><li>After triggering the refresh using the <strong>Web activity</strong>, add an <strong>Until activity</strong> that calls the Power BI API to check the refresh status:</li></ul><pre>@concat(&#39;https://api.powerbi.com/v1.0/myorg/groups/&#39;,pipeline().parameters.WorkspaceGuidId,&#39;/datasets/&#39;,pipeline().parameters.DatasetGuidId,&#39;/refreshes&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*FtZHCI2MqRiEMDTNz_Echw.png" /></figure><ul><li><strong>Parse the response</strong> and assign the latest refresh status to the RefreshStatus variable.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HWom0FhtqkD7cDfrMbVHSQ.png" /></figure><ul><li>Use a <strong>Switch activity</strong> to handle different statuses:</li></ul><p>Unknown: Wait for 10 seconds and recheck. (this is returned when the dataset refresh is InProgress)</p><p>Completed: Proceed to the next step.</p><p>Failed: Fail the pipeline with an appropriate error message.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*E0zoqpXjpwayX0UtgVSmSA.png" /></figure><p>This approach ensures your pipeline reacts dynamically to the refresh result and handles failures gracefully.</p><h4>4. Test your pipeline</h4><p>When you run the pipeline, the Web activity should trigger the dataset refresh in Power BI.</p><h4>Test 1: Dataset refresh succeeded</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/966/1*3lsirrdIk_Uc5LQb5vzMcw.png" /><figcaption>Pipeline when a dataset refresh succeeds</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GiHow7SX9WKOTJfZgD8wRw.png" /><figcaption>Dataset refresh history</figcaption></figure><h4>Test 2: Dataset refresh failed</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/1*2g5Vq1QPDtguj0rhE3CUYA.png" /><figcaption>Pipeline when a dataset refresh fails</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sQJg4MFOmCL-p74TUCj-Qg.png" /><figcaption>Dataset refresh history</figcaption></figure><h3>✅ Conclusion</h3><p>With the proper configuration of Azure security groups, Power BI admin settings, and Synapse permissions, you can securely automate Power BI dataset refreshes from within your pipelines.</p><p>This method ensures end-to-end automation while keeping your authentication secure using Managed Identity — no secrets, no service principal passwords, just clean, cloud-native integration.</p><p>💡 <strong>Want to see the full pipeline in action?</strong><br> You can find the complete Synapse pipeline JSON, including the loop logic and Web activities, in this <a href="https://github.com/barbaramalafaia/PipelinesDownUnder/tree/main/Synapse-PowerBIRefreshPipeline">GitHub repository</a>.</p><blockquote><strong>If this post helped you and you want to support my content, you can </strong><a href="https://www.buymeacoffee.com/barbaramalafaia"><strong>Buy Me a Coffee</strong></a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75e78fc2db39" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>