<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shiran Rubin on Medium]]></title>
        <description><![CDATA[Stories by Shiran Rubin on Medium]]></description>
        <link>https://medium.com/@shiranrubin_25022?source=rss-e1425041fb6c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*unwsqkbvx_yjgEEihIcRgQ.jpeg</url>
            <title>Stories by Shiran Rubin on Medium</title>
            <link>https://medium.com/@shiranrubin_25022?source=rss-e1425041fb6c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 10:58:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shiranrubin_25022/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Anonymize Results in Azure Cognitive Search]]></title>
            <link>https://medium.com/microsoft/anonymize-results-in-azure-cognitive-search-fedee02ca19e?source=rss-e1425041fb6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/fedee02ca19e</guid>
            <category><![CDATA[azure-search]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[presidio]]></category>
            <dc:creator><![CDATA[Shiran Rubin]]></dc:creator>
            <pubDate>Wed, 03 Nov 2021 15:29:16 GMT</pubDate>
            <atom:updated>2021-11-03T15:29:16.380Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://docs.microsoft.com/en-us/azure/search/search-what-is-azure-search">Azure Cognitive Search</a> is a cloud search service that gives developers infrastructure, APIs, and tools for building a rich search experience. It is a basic need today to have the ability to search over your data and as time progress, more complicated needs arise from search mechanisms.</p><p>One of the issues with search mechanisms is the data itself, not all data can be retained, saved and searched over, data can easily contain Personally identifiable information (PIIs), which cause security issues or <a href="https://en.wikipedia.org/wiki/General_Data_Protection_Regulation">GDPR</a> violations.</p><p>So how can we avoid this and protect our data while still offering search options over it?</p><p>well, there is more than one option for this issue:</p><p><strong>Option 1:</strong></p><p>Process the data before we insert it as a search data, it means that where ever the data comes from, we will need to process it before it is being inserted into the search DB. This method will work but will result in another hop the data needs to go through, which means another hop it can fail in, another hop we need to worry about data consistency.</p><p>Disadvantages:</p><ol><li>This method can cause data loss.</li></ol><p>2. Requires another process maintenance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*WGdaw9k5mV693nN24oiVnA.png" /></figure><p><strong>Option 2:</strong></p><p>Use the indexing mechanism of the Cognitive Search. To have an effective search, a proper indexing process needs to happen over existing and new data, while the indexing process runs, we can manipulate the data.</p><p>The Azure Cognitive Search gives us two options to perform manipulation over the data while indexing.</p><p><a href="https://docs.microsoft.com/en-us/azure/search/cognitive-search-predefined-skills"><strong>Built in power skills:</strong></a></p><p>Use Azure Cognitive Search <a href="https://docs.microsoft.com/en-us/azure/search/cognitive-search-skill-pii-detection">PII detection</a> built in power skill. Azure Cognitive Search offers you the option to process the data while it is being indexed, which means you won’t need to have another hop or process to maintain the data and the search mechanism will take care of it on its own.</p><p>To have more flexibility and agility over the search, we can use Azure Customized Power skills.</p><p><a href="https://github.com/Azure-Samples/azure-search-power-skills"><strong>Customized power skills:</strong></a></p><p>Power Skills are a collection of useful functions to be deployed as custom skills for Azure Cognitive Search. You can find working code samples and details on how to build your own power skill in the <a href="https://github.com/Azure-Samples/azure-search-power-skills">azure-search-power-skills repository</a>.</p><p>We used the customized power skills repository to add <a href="https://github.com/microsoft/presidio">Presidio</a> as a new customized skill. <a href="https://github.com/microsoft/presidio">Presidio</a> is an open-source tool to recognize, analyze and anonymize personally identifiable information (PII). Using trained ML models, Presidio was built to ensure sensitive text is properly managed and governed.</p><p>Using the customized power skill will allow us to have leverage over the power skill itself, customize it, add more models or functionalities if we like. The sample repository uses either Azure functions or Docker containers to deploy a power skill.</p><p>We based <a href="https://github.com/Azure-Samples/azure-search-power-skills/tree/main/Text/PresidioPIIAnonymization">Presidio power skill</a> on the <a href="https://github.com/Azure-Samples/azure-search-power-skills/blob/main/Template/PythonFastAPI/README.md">Python Fast API</a> power skill and used docker container, app service and Terraform to deploy it. The azure-search-power-skills repository is actually <a href="https://github.com/Azure-Samples/azure-search-power-skills/blob/main/Text/PresidioPIIAnonymization/powerskill/app.py">using the Python package of Presidio and wraps it with FastAPI</a> and this is deployed as a docker container.</p><h4>So how does it work?</h4><p>After creating an Azure search resource, we have several configurations to define in order for the Presidio anonymizer to work on our data while indexing.</p><p>First things first, we will need to deploy Presidio as a Docker container to an app service. A full guide can be <a href="https://github.com/Azure-Samples/azure-search-power-skills/tree/main/Text/PresidioPIIAnonymization#how-to-implement">found here</a> on how to build, push and deploy Presidio as an Azure web app.</p><p>Once the application is up and running, your resource group should look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FO-7gtPEWfnZ5e0r1eYsPA.png" /><figcaption>Azure resources</figcaption></figure><p>To make this all work we have many components:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rrQaDavYDNCHqK257lOuyQ.png" /><figcaption>High level diagram</figcaption></figure><p>Nice drawing but what does it mean? We will explain it step by step.</p><ol><li>data source — where the search data and results will come from.</li><li>Search service which contains: skillset, index and indexer. All of which will help us to combine the power skill and index the search data and get the anonymized results.</li><li>App service with Presidio running on it.</li></ol><p>A full guide on how to define all of them can be <a href="https://github.com/Azure-Samples/azure-search-power-skills/tree/main/Text/PresidioPIIAnonymization#user-content-deploy-the-datasource-index-skillset-and-indexer">found in the readme</a> file.</p><p><strong>So how do they all play together?</strong></p><p>Lets start with the data source, where our search data comes from, by selecting one of the following options:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*ZB0ugLkBkJQ3kYR8377-yQ.png" /><figcaption>Data source input options</figcaption></figure><p>In this case we used Azure blob storage with text files:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uhw1qstd394PDiGvqLEq9g.png" /><figcaption>Data source example</figcaption></figure><p>One of the text examples contained the name Buzz Hargrove, which is a PII. Our goal is to prevent the name to be indexed and appear in the search:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0fW5_6haBSIi3SUhI6KGpQ.png" /><figcaption>PII text example for indexing</figcaption></figure><p>Cool, so we have the application, we have the data source connected and now we need to connect the search to the application. This is done by defining a skill set:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q4AWVd7CyY4PG4R4R7UpVQ.png" /><figcaption>Define search Skillset</figcaption></figure><p>We define the URI of the web app:</p><pre>POST /api/extraction<br>{<br>   &quot;text&quot;: TEXT_TO_ANONYMIZE<br>}</pre><p>This method will receive the text, <a href="https://github.com/Azure-Samples/azure-search-power-skills/blob/main/Text/PresidioPIIAnonymization/powerskill/powerskill/presidio.py">analyze, anonymize it</a> and return the anonymized text:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GrHUgkBr_XbRNqDMIo10JA.png" /><figcaption>Main logic for Presidio</figcaption></figure><p>We defined the skillset and the next step is to create the index. The index will make sure the specific field we want is going to be indexed. In our case it is ‘content’:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z5PKNYbSZG0wGxbgzTwuAA.png" /><figcaption>Define search index</figcaption></figure><p>For the last finishing touch, we will define the indexer. The indexer will connect the index to the skillset and will give us an option to invoke the process on specific time and even now:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U3HpixeX3-gdjuucFshnAQ.png" /><figcaption>Define search indexer</figcaption></figure><p>Once the indexer ran, the text was indexed and inserted after anonymization into the search engine. We can view the results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2lPp_S6Bcdc3Y3GDhho3PQ.png" /><figcaption>All search results</figcaption></figure><p>When we try to search for “Buzz Hargrove”, SUCCESS! nothing is found:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_HrqAwp4oMnPPvMKilvvSQ.png" /><figcaption>Empty result for Buzz Hargrove</figcaption></figure><p>Just to make sure the search does work, when we search for the word “Deputy” it is being returned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ed-06IUQZSC01U2v4awaNQ.png" /><figcaption>“Deputy” return search results</figcaption></figure><p>To sum up, the Azure Search Power Skill gives you a very powerful tool. It is very easy to create your own Power Skill or use an existing one to customize your search data including filtering, anonymizing and converting it to your own needs without having additional processes or overheads.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fedee02ca19e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/microsoft/anonymize-results-in-azure-cognitive-search-fedee02ca19e">Anonymize Results in Azure Cognitive Search</a> was originally published in <a href="https://medium.com/microsoft">Microsoft</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Investigate user behavior, using Azure application insights]]></title>
            <link>https://medium.com/microsoft/investigate-user-behavior-using-azure-application-insights-a7c19619ce1d?source=rss-e1425041fb6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/a7c19619ce1d</guid>
            <category><![CDATA[azure-monitoring]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[presidio]]></category>
            <category><![CDATA[user-behavior]]></category>
            <dc:creator><![CDATA[Shiran Rubin]]></dc:creator>
            <pubDate>Thu, 21 Oct 2021 12:07:04 GMT</pubDate>
            <atom:updated>2021-10-21T12:07:04.916Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://docs.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview">Azure Application insights</a> is a powerful monitoring tool, how powerful? Well, it helped us figuring out an anomaly in our application activity and almost find the user who is responsible for that weird behavior.</p><p>It all started on a nice spring evening, with an Azure Monitor email alert in our inboxes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/949/1*wbcWuroxUeSMOKrS7d5pnw.png" /></figure><p><a href="https://github.com/microsoft/presidio">Presidio</a> is an open-source tool to recognize, analyze and anonymize personally identifiable information (PII). Using trained ML models. In addition to an OSS library we also have a <a href="http://presidio-demo.azurewebsites.net/">demo site</a> for Presidio we maintain. This alert was one of several defined for the site, saying the number of requests the server received was greater than usual. First thing we did is to check the monitors, starting from several days before the alert until the moment the alert started, that’s where we found this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*QvZDAp6HCvIJpov1VBfXpg.png" /></figure><p>The first instinct will usually be to scale out, so the requests will be distributed to more servers. This is a good idea in case the machines CPU or memory is suffering due to the new traffic or we are getting to some limit on the amount of open connections on a single machine, in our case it wasn’t so, the server wasn’t even flinching:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GCihdV3PiKvXHocm8nWODQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*PMxmL-cj2wjNIX2cTdJ5qg.png" /></figure><p>The next thing we did was to ask ourselves why? Why suddenly there’s a spike? Did the site start to get that much attraction? Is this real user activity? If so, why didn’t the memory/CPU change or even, increased at all?</p><p>The graphs won’t be able to answer these questions for you, but the logs will. Viewing the requests logs between the increase timeline, we saw a weird user behavior:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YBFWJPrm2ybu8u7vqK9JYA.png" /></figure><p>Either someone really likes our favicon or something else is going on here. We suspected the latter.</p><p>This behavior didn’t last long, about 6 hours later the alert and traffic were gone, but don’t worry, they came back the next day and the day after, each time for 5–6 hours:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/913/1*8cf-K_sWhuYuL_UegeZRsw.png" /></figure><p>At this point, we were still curious about two things:</p><ol><li>Who is this person?</li><li>Why are they so fascinated with our favicon?</li></ol><p><strong>First question (who is it?):</strong></p><p>The right query will give us the right data of what we are looking for:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iDwGve2MJ2aadWCNobAg_A.png" /></figure><p>Great, we found the user! User HBu2e, simple!</p><p>Or not so much, as can be seen below, the client_ip was 0.0.0.0, we had the client city and state but not their IP so we couldn’t block them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KqNEvIaQqWnmYdQeDXwv-w.png" /></figure><p>We started investigating to why we can’t see the user IP and very quickly arrived to the <a href="https://docs.microsoft.com/en-us/azure/azure-monitor/app/ip-collection?tabs=net">Azure monitor documentation</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yWRz_Y_6Zq0oRp7AQQ1aBw.png" /></figure><p>Makes sense. By default Azure protects you from improper collection of user data and complies with world wide regulations such as GDPR. You can find more <a href="https://docs.microsoft.com/en-us/azure/azure-monitor/logs/personal-data-mgmt">guidelines about data stored in Log analytics and applications insights in here</a>.</p><p>If you do choose to enable this feature, you need to do it carefully by using two methods:</p><ol><li>Be aware it is on and make sure to turn it off.</li><li>Have a low log retention.</li></ol><p>We did not get to use the feature or block the user IP, but incase we wanted to, we would block an IP using <a href="https://docs.microsoft.com/en-us/azure/app-service/app-service-ip-restrictions">access restriction for Azure App Service</a>.</p><p>2. <strong>Second question (why favicon?):</strong></p><p>At first we thought it might be a crawler but then again, why would it only access the favicon, if it was a crawler bot, it did a really lousy job.</p><p>We tried to consult experts and ask our long time friend google, all we came up with is an <a href="https://arstechnica.com/information-technology/2021/02/new-browser-tracking-hack-works-even-when-you-flush-caches-or-go-incognito/">article on how websites can track incognito user using the favicon</a>, which was not relevant.</p><p>So this question will need to remain open for us and for you.</p><p>Thanks to Nava Vaisman Levy, Avishay Balter and Tamir Kamara.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a7c19619ce1d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/microsoft/investigate-user-behavior-using-azure-application-insights-a7c19619ce1d">Investigate user behavior, using Azure application insights</a> was originally published in <a href="https://medium.com/microsoft">Microsoft</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leverage Azure DevOps pipelines to build our Python packages.]]></title>
            <link>https://medium.com/microsoft/leverage-azure-devops-pipelines-to-build-our-python-packages-4a218ed256b7?source=rss-e1425041fb6c------2</link>
            <guid isPermaLink="false">https://medium.com/p/4a218ed256b7</guid>
            <dc:creator><![CDATA[Shiran Rubin]]></dc:creator>
            <pubDate>Tue, 06 Apr 2021 14:13:30 GMT</pubDate>
            <atom:updated>2021-04-06T14:13:30.275Z</atom:updated>
            <content:encoded><![CDATA[<h4>Transforming our build process from complicated and unmaintainable to an organized, reusable release process. We are going to tell you how we used code, Azure DevOps pipelines and DevOps design patterns to create our pipeline and build process.</h4><h4>Problem</h4><p>When developing a software one of the most important things is to have it easily built, deployed, extended and used. We can achieve that by having a clean and clear CI flow.</p><p>This is one of the obstacles we encountered while working on <a href="https://github.com/microsoft/presidio">Presidio</a>, moving it from V1 to V2. Presidio is an open-source tool to recognize, analyze and anonymize personally identifiable information (PII). Using trained ML models, Presidio was built to ensure sensitive text is properly managed and governed.</p><p>In V1, the entire build process was combined from a short YAML file for the azure DevOps pipelines and a huge messy makefile for everything else.</p><p>We needed something simpler, an easy process to maintain and build while supporting the functionality we wanted. This was one of the goals we aimed for in V2.</p><h4>Solution</h4><p><strong>First thing first, what do we need and want?</strong></p><p>We want to have our infrastructure and pipelines as code. We want it clean, clear, extendable and easy to maintain.</p><p><strong>requirements?</strong></p><p>We have three Python packages, each should be built and run with Python 3.6, 3.7 and 3.8.</p><p>We must be able to build and run packages locally, validate linting, security and compliance, run tests, create and publish to PyPi and Docker and deploy to development and production environments or our demo site.</p><p>Simple, right?</p><p><strong>How did we get there?</strong></p><p>Since there are many requirements, the pipeline writing is not straight forward. We started with a nice diagram to map pipeline triggers and their steps. There are three different triggers:</p><ol><li>Pull request pipeline — Triggered on each pull request created for Presidio. We run our automatic validations to make sure the pull request aligns with Presidio defined standards.</li><li>Continues integration pipeline — Before merging to main, we validate the standards, deploy the new version to development environment (demo site) and test it. We make sure there are no breaking changes for our demo site.</li><li>Release pipeline — Release a new version as a docker container to ACR (Azure container registry), PyPil, the production demo site and in Github (pages, version etc.).</li></ol><p>Each trigger works over three packages: Analyzer, Anonymizer and Image redactor. Between pipelines, many stages repeat themselves: unit testing, code linting, building containers/Python packages and running E2E tests are shared between the PR and CI pipelines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*pGo-ji_-qwUsQg63Iuumog.png" /></figure><p>In order to build this beauty, we used our previous knowledge of <a href="https://medium.com/microsoftazure/devops-patterns-sharing-reusable-components-d351a9b4fdf9">DevOps patterns and sharing reusable components</a>:</p><ul><li><a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops">Templates</a> — For reuse of code (using parameters) and separation of logic. We will be using <a href="https://medium.com/microsoftazure/devops-patterns-sharing-reusable-components-d351a9b4fdf9#e81b">templates as our building blocks</a>.</li><li><a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&amp;tabs=schema%2Cparameter-schema">YAML schema</a> — For declaring the workflow of each pipeline and template.</li></ul><p>Starting from creating the main templates, the ones who trigger the pipelines. We created a YAML file for each type of trigger.</p><p>From here on, we are creating <a href="https://medium.com/microsoftazure/devops-patterns-sharing-reusable-components-d351a9b4fdf9#a538">Template Hierarchy</a>. In some cases, we use different <a href="https://medium.com/microsoftazure/devops-patterns-sharing-reusable-components-d351a9b4fdf9#f799">Template parameterization patterns</a>. <a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&amp;tabs=schema%2Cparameter-schema#parameters">Parameters</a> helps us to customize the template to a specific flow.</p><p>Let’s see some code examples of reusable code:</p><p>In <a href="https://github.com/microsoft/presidio/blob/main/.pipelines/templates/release.yml">release.yml</a> file, template <a href="https://github.com/microsoft/presidio/blob/main/.pipelines/templates/build-and-push-containers.yml">build-and-push-containers.yml</a> is being used twice:</p><ol><li>Build and push the container with a version.</li><li>Build and push the container as the latest.</li></ol><pre>- job: BuildAndPushContainers<br>   displayName: Build and Push Containers<br>   dependsOn: version<br>   pool:<br>     vmImage: &#39;ubuntu-16.04&#39;<br>   variables:<br>     version: $[ dependencies.Version.outputs[&#39;setVer.version&#39;] ]<br>     REGISTRY_NAME: &#39;$(ACR_REGISTRY_NAME).azurecr.io/&#39;<br>     IMAGE_PREFIX: &quot;public/&quot;<br>   steps:<br>     - template: ./build-and-push-containers.yml<br>       parameters:<br>         REGISTRY_NAME: $(REGISTRY_NAME)<br>         IMAGE_PREFIX: $(IMAGE_PREFIX)<br>         TAG: &quot;:$(version)&quot;<br>         AZURE_SUBSCRIPTION: $(ACR_AZURE_SUBSCRIPTION)<br>         DOCKER_COMPOSE_FILE: &#39;docker-compose-text.yml&#39;<br>     - template: ./build-and-push-containers.yml<br>       parameters:<br>         REGISTRY_NAME: $(REGISTRY_NAME)<br>         IMAGE_PREFIX: $(IMAGE_PREFIX)<br>         TAG: &quot;:latest&quot;<br>         AZURE_SUBSCRIPTION: $(ACR_AZURE_SUBSCRIPTION)<br>         DOCKER_COMPOSE_FILE: &#39;docker-compose-text.yml&#39;</pre><p>Another cool feature which came to our rescue, is the <a href="https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&amp;tabs=schema%2Cparameter-schema#job">job matrix strategy</a>. We used it to run the Python code over different Python versions.</p><p>When we wanted to run the Analyzer package build and tests with different versions of Python, we used the matrix to declare the different versions. The <a href="https://github.com/microsoft/presidio/blob/main/.pipelines/templates/build-analyzer.yml">build-analyzer.yml</a> template will do the required steps each time with a different version:</p><pre>- job: TestAnalyzer<br>   displayName: Test Analyzer<br>   dependsOn:<br>       - &#39;Inclusivelint&#39;<br>   pool:<br>       vmImage: &#39;ubuntu-16.04&#39;<br>   strategy:<br>       matrix:<br>           Python36:<br>               python.version: &#39;3.6&#39;<br>           Python37:<br>               python.version: &#39;3.7&#39;<br>           Python38:<br>               python.version: &#39;3.8&#39;<br> <br>   steps:<br>       - task: UsePythonVersion@0<br>         inputs:<br>             versionSpec: &#39;$(python.version)&#39;<br>         displayName: &#39;Use Python $(python.version)&#39;<br>       - template: ./build-analyzer.yml</pre><p>Eventually, our build diagram is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FmjUmEZ3HaeZzVkKjVXsjg.png" /></figure><p>As can be seen, we are reusing many templates. They represent full flows repeating themselves over the different piplines:</p><ul><li>E2E tests.</li><li>Build and publish containers.</li><li>Deployment stage.</li><li>Build python.</li><li>Lint, build and test.</li></ul><p>By reusing templates with parameters and using the job matrix strategy, we made the pipeline code more readable, easy to use and maintain while the flow is clean and clear.</p><h4>Conclusion</h4><p>Many times, we pay attention to clean code, design of code and service and we forget how the CI, pipelines and the entire infrastructure is built. The importance of having a clean, clear, reusable pipeline with the ability to easily extend it, can save a lot of time and headaches.</p><p>In this post I have shown how we leveraged those best practices for writing and managing pipeline code, in presidio V2.0 to ensure that adding another Python version / a new service / another publish method or deployment method to our pipeline will take almost no effort and time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4a218ed256b7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/microsoft/leverage-azure-devops-pipelines-to-build-our-python-packages-4a218ed256b7">Leverage Azure DevOps pipelines to build our Python packages.</a> was originally published in <a href="https://medium.com/microsoft">Microsoft</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>