<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nouer Uz Zaman on Medium]]></title>
        <description><![CDATA[Stories by Nouer Uz Zaman on Medium]]></description>
        <link>https://medium.com/@noueruzzaman?source=rss-340fea656be9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*1aLNX3EXAaYcAjgy</url>
            <title>Stories by Nouer Uz Zaman on Medium</title>
            <link>https://medium.com/@noueruzzaman?source=rss-340fea656be9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 17:55:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@noueruzzaman/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Tableau Log Viewer]]></title>
            <link>https://medium.com/@noueruzzaman/tableau-log-viewer-68bb23e878f1?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/68bb23e878f1</guid>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Thu, 23 Oct 2025 12:03:38 GMT</pubDate>
            <atom:updated>2025-10-23T12:03:38.977Z</atom:updated>
            <content:encoded><![CDATA[<p>Tableau Log Viewer is a cross-platform tool with a simple interface that has a single purpose of making it easy to quickly glance over Tableau log files.</p><p><a href="https://github.com/tableau/tableau-log-viewer?tab=readme-ov-file">GitHub - tableau/tableau-log-viewer: Tableau Log Viewer is a cross-platform tool for quickly glancing over Tableau log files</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*VBv32TM3yh4FLdOA7tpACA.png" /></figure><p>How to Insteall it?</p><p>Click on the following link and download “tlv-win1.3.zip” and “source code (zip)”.</p><blockquote>https://github.com/tableau/tableau-log-viewer/releases/tag/v1.3</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Pw2P0h_y8_DZd6XAVZfxg.png" /></figure><p>Then once downloaded go to Downloads folder in the desktop and search for “tlv” Application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*70Dmc1CZ8oUhglBCCSmw8g.png" /></figure><p>Once you click on it you will get a dialog box, click “Extract All” and select the folder where you want to save it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/1*D7u0zLdUOaENKIViyNKMGg.png" /></figure><p>Then go to the saved folder and search for tlv Application and open it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*-Z-JB2z37URCK1Mvs4fePg.png" /></figure><p>If you need more information on how to use it then please click on the following link to read further instructions: <a href="https://github.com/tableau/tableau-log-viewer/commit/6151d5862f8af84cb85a8142f6f15a41fdc04813">https://github.com/tableau/tableau-log-viewer/commit/6151d5862f8af84cb85a8142f6f15a41fdc04813</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68bb23e878f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to create a Table in Snowflake]]></title>
            <link>https://medium.com/@noueruzzaman/how-to-create-a-table-in-snowflake-377b9f6d7c43?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/377b9f6d7c43</guid>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Mon, 21 Jul 2025 11:12:32 GMT</pubDate>
            <atom:updated>2025-07-21T11:12:32.408Z</atom:updated>
            <content:encoded><![CDATA[<p>Snowflake is the most popular cloud data warehouse in the world with over 20% market share in the cloud market.</p><p>The biggest plus point of Snowflake is that it can be easily scaled up or down, can easily integrate with other cloud providers like AWS and Azure and is SIMPLE to use as you can directly write SQL queries within the tool.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6MYltCiRg5sIb1Q3YTUJsA.png" /></figure><p>If you are working as a Tableau/Power BI developer you are mostly working on the reporting side to create dashboards for the senior stakeholders. However, creating dashboards is only 30% part of the work and majurity of it involves creating and testing data models within the BI tool or creating data tables within SQL.</p><p>It is a common use case for Tableau/Power BI developer where they would need to create a table/view table/store procedure so it can be used to create the report.</p><p><strong>Table in Snowflake</strong></p><p>So, what are the steps? To create a table in Snowflake a warehouse would be created or use predefined warehouse.</p><p>You can use commands like use warehouse to use predefined warehouse.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*AI5usDBbvF2qYPxlyjZ-_Q.png" /></figure><p>Then you need to use the following command where you start with the key words “create or replace table” then name of the table followed by writing the select statement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8yZlKHWlxRiktpCeP5CQaQ.png" /></figure><p>Additional commands like “cluster by” or “limit” could also be used, as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*NjBQPLUwR8lKjD_-rDeOYA.png" /></figure><p>After writing the code simply click on the run button at the top right.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/671/1*RjUqYF26o1zdg1M2JYcODA.png" /></figure><p>It will then return a message if the create table command was successful.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/845/1*iSm8vPD5e4t8JHltr-1fUQ.png" /></figure><h4>Temporary tables</h4><p>A fact to note is that:</p><blockquote>all tables created in Snowflake are by default temporary tables</blockquote><p>We had created an orders table (as show previously).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*Dm__Dp-MP3etzQWuwxmKgw.png" /></figure><p>To create a permanent table you would create another table on top of this so you are copying data from created table into new table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RTmKBFRjy3Xb11cXyF8AMA.png" /></figure><h4>Transient table</h4><p>Then a permanent table can be converted into a “Transient table”.</p><blockquote><em>Transient tables in Snowflake are a powerful feature that offers a balance between Standard tables</em> (permanent, fully protected) <em>and Temporary tables </em>(as log as you have the screen open it will exist till then)<em>. The biggest benefit is that storage costs are much lower than standard tables because permenant tables have time travel features (it saves previous versions of the table) and it is mainly used to store intermediate data during ELT/ETL pipeline work processes.</em></blockquote><blockquote>In other words it is used as a <strong>staging area</strong> to store tables that is yet to transform further.</blockquote><p>To create it use the key words “create or replace transient table” followed by the select statement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jBG7KE9j6NBh1SeRE2_isw.png" /></figure><p>So this was the short description of how to quickly create tables within the Snowflake environment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=377b9f6d7c43" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is the difference between DAX Studio & Tabular Editor 2]]></title>
            <link>https://medium.com/@noueruzzaman/what-is-the-difference-between-dax-studio-tabular-editor-2-87e761c2315b?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/87e761c2315b</guid>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Fri, 16 May 2025 14:07:14 GMT</pubDate>
            <atom:updated>2025-05-16T14:07:14.381Z</atom:updated>
            <content:encoded><![CDATA[<p>I have been using Power BI Desktop and Service at work for some time now as I am very much familiar about how much data modelling we can do in these tools.</p><p>However, when I came across Tabular Editor 2 and DAX Studio I wasn’t really sure what was the main difference between the two.</p><h3>What is Tabular Editor 2</h3><p>First of all what is Tabular Editor 2? Tabular Editor 2 is an external open source tool (like Python) which you can download for free from sqlbi website.</p><p>Main Points</p><ol><li>Helps to edit, mange and enhance<strong> data model</strong> within Power BI reports.</li></ol><p>2. In addition to that you can use the tool to write a <strong>script to automate any repetitive work</strong>; pretty much like how VBA does.</p><p>3. It can also be used for <strong>data governance</strong> and <strong>data quality</strong>.</p><h3>How it works</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uBqEB6qzBxqgVb9X8DgmoQ.png" /></figure><p>Once you download it you can find it within External tools ribbon.</p><p>Tabular Editor 3 is a licences premium version which offers more functionality than Editor 2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*PzFbJuSzrZPthJK9CyeniQ.png" /></figure><p>Once you click on it, a new window opens up</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N9awgazirMknSJWQr-yrPA.png" /></figure><p>Within it you can see all the tables found within the Power BI report along with roles and relationships.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*cjMwjAXojK4CxoGdJFlnig.png" /></figure><p>You can write a script within the Expression Editor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*cGt8o4RPBA0am5-10oCWxg.png" /></figure><h3>What is DAX Studio</h3><p>DAX studio is used to <strong>debug</strong> any complex <strong>calculated measures and columns</strong>. If the performance is not fairly well then DAX Studio can help to <strong>reduce the size</strong> of the model.</p><p>DAX Studio can be downloaded online and once downloaded you can find it within External tools section.</p><p>Then it will open a new window. If you go to Advanced tab, you find <strong>View Metrics </strong>button which is used to analyze performance of column(s) and how much memory it occupies when running the query.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G1J2QSpup4LwONcNilxsZA.png" /></figure><p>Once you click on View Metrics a new window opens at the bottom and in the <strong>VertiPaq Analyzer</strong> you find all the statistical information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6j5KjrQczHBQMefEG4BaKQ.png" /></figure><p>If you click on any table it opens up the list of columns within the table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*0GOovMly8iCg4HIWeCQdCw.png" /></figure><h4>Main Difference</h4><p>Tabular Editor is used to perform advanced scripting with C# or python to automate task and manage large Power BI models. Wheras, DAX Studio is used to implement performance tuning of DAX queries (measures, columns). It is also used to assess and improve DAX formulas.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87e761c2315b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understand the concept of Request library within Python!]]></title>
            <link>https://medium.com/@noueruzzaman/understand-the-concept-of-request-library-within-python-88bf77402763?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/88bf77402763</guid>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Sat, 23 Nov 2024 23:51:47 GMT</pubDate>
            <atom:updated>2024-11-23T23:51:47.294Z</atom:updated>
            <content:encoded><![CDATA[<p>Whenever you are working with Python pretty much everything revolves around APIs. <strong>API</strong> stands for Application Programming Interface and it helps Python to <strong>communicate</strong> with other tools. Basically, it acts like a medium to help two tools to interact with each other.</p><p>The biggest advantage of using Python is that it has huge collection of API libraries which can pretty much connect to any software or website out there. Now a <strong>library</strong> is simply a <strong>collection of modules</strong>. A <strong>Module</strong> is a <strong>file containing python definitions and statements</strong>. E.g. if we are running Tableau Server client api librarythen it will have modules like TSC.Server, TSC.TableauAuth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*SzUJEm7kNpvRBnUUELiSlw.png" /></figure><h4>Requests library</h4><p>One of the most popular library used in Python is pandas. You will get plenty of information on pandas on YouTube, Google or other forums. However, here we will talk about 2nd most used library which is requests library. Requests helps Python to interact with other websites.</p><p>Let’s look at it via an example. In this example, I am using VS Code to run python codes.</p><p>The first thing we would do is import pandas and requests libraries.</p><p>Whenever, we work with Requests library the most important module to use would be requests.get(). Then we run text function to give us nice formatted version which we can look at.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/1*2GE0TBOQLRN_OUlhvTszOg.png" /></figure><p>Text function will return an output which may look like a dictionary made up of key value pairs. We would want to convert this output into a <strong>dataframe</strong>.</p><p>Before we convert it into a dataframe we would want to convert it to a <strong>python dictionary </strong>using <strong>json</strong> function. The first thing you will notice is that earlier it was wrapped around double quotation marks now it shows <strong>single quatation marks</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/973/1*jOtZg3udFqGN902mLZxdSg.png" /></figure><p>Now as it is an actual dictionary which we can work with we can use it to create a new variable and assign this output to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*EEdxv1ZN1xri_nl_HJqk9Q.png" /></figure><p><strong>2nd example converting to json dictionary</strong></p><p>There are many ways to convert to dictionary and another method is using json.loads() which will give the same output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/635/1*fssMZJvpC1jvQPZ5BSWGlQ.png" /></figure><p><strong>Dataframe</strong></p><p>Now we can convert it to a dataframe by simply typing <strong>pandas.DataFrame</strong> and within the parenthesis write the dictionary variable created earlier. Make sure this <strong>dictionary variable </strong>is wrapped around <strong>curly</strong> <strong>square brackets</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/1*-gcNJ3UXrNzm9m0vwMpeEA.png" /></figure><p>Then assign another variable this would be name of the datframe and when you call it out in another cell it simply returns in a table format.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=88bf77402763" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Conducting Business Analysis as a Power BI/Tableau Developer]]></title>
            <link>https://medium.com/@noueruzzaman/conducting-business-analysis-as-a-power-bi-tableau-developer-65f07a452c8e?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/65f07a452c8e</guid>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Fri, 22 Nov 2024 12:31:58 GMT</pubDate>
            <atom:updated>2024-11-22T12:59:30.483Z</atom:updated>
            <content:encoded><![CDATA[<p>When we think of Power BI or Tableau Developer we only think of them as visualization experts who’s main job is to create fancy charts but they do a lot more than that.</p><p>Think of an iceberg where the person can only see the part above the sea but rest of the 80% is hidden below the sea. Similarly, only <strong>20% of developer’s job actually involves creating a visualization</strong> which is seen by the end user. The rest of the <strong>80% </strong>is about conducting user requirements, data gathering, data transforming/cleaning, data analysis, data testing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*PT6SEWt9GDGJgPHANNSaKg.png" /></figure><h4>Project Lifecycle</h4><p>When we are involved in a project, we should be familiar with the project lifecycle. This helps us to understand where we are in the project delivery journey. It starts off with <strong>Project Initiation</strong> with stakeholder meetings and user requirements. Then <strong>Project Planning</strong> is implemented via documentation. Then <strong>Project Execution</strong> is done via creating the dashboard based on requirements. Then we would do <strong>Project Testing</strong> (Monitoring) by testing the dashboard. Finally, we aim to <strong>Close the Project</strong> via project sign-off.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*nDjYldmsYX9cR67cF73oBw.png" /><figcaption>Credits: Project-management.com</figcaption></figure><h4>1. User Requirements &amp; Target Audience | Project Initiation</h4><p>When Power BI or Tableau Developers are involved in projects the <strong>most crucial</strong> part of creating a visualization is getting the clear and concise<strong> user requirements</strong> from the stakeholders as part of Project Initiation. This is because we want the base to be strong.</p><p>Think of construction workers making a building. The would first make sure the ground floor is constructed well and then proceed to make the additional floors. If the base is not strong the building will collapse.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/1*Xy9OLnOYpvDmX-tdQKlV5Q.png" /></figure><p>Similarly, if right requirements has not been gathered or the developer just assumes the requirements then the project may need to be created all over again.</p><h4>Target Audience</h4><p>To get user requirements the first step would be to hold initial stakeholder meeting. This would give us a good chance to understand who is our <strong>target audience</strong>. As a best practise we should always understand who is our target audience. Is the end-user the senior stakeholder like CFO or Financial Controller or is it an operations manager. This is because type of visualizations we choose will differ based on the seniority.</p><p>In a CIMA management accounting course, it teaches its’ students about 3 pillars/levels. It basically summarises that in every company there are 3 main levels: Operational, Management and Strategic. Operational level comprises of supervisors or Team Leads. Management level is made up of Senior Managers and Strategic level comprises of executive board members to CEO.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*L4VAyJyiG--5QkJqTw1Jww.png" /></figure><p>So, when we get a requirement to create a visualizations from either of them from these 3 levels, the purpose and ask is different. If the requirement is coming from CFO it is likely to be about helping them to make a key strategic decision or give a summerised view of the company. If the requirement came from a Team Lead/supervisor then they might want to get more detailed information about products so they can target specific products which are underselling.</p><blockquote>So, the point we always want to bear is when we present dashboards to senior executives we should make the visuals as summarised as possible more like KPI driven dashboard.</blockquote><p>The meeting would also give us the chance to ask questions around the project and requirements. Here the most important question to ask is “Why” we are doing it. There is a reason why we are making this dashboard and once we understand the purpose then it can help developers to create more meaningful dashboards. So, in the stakeholder meeting we should be asking the questions what is the purpose of having this dashboard.</p><h4>Stakeholder Matrix</h4><p>We should also aim to create a Stakeholder Matrix which should give us a good idea about where each stakeholder sits in the project. It is created on x-axis and y-axis. On the x-axis we have <strong>shareholder’s influence</strong> and on the y-axis we have <strong>shareholder interest</strong>. The most important stakeholder is the at the <strong>top right</strong> which has high interest and influence. This would be your <strong>end-user</strong> and you want to manage them closely. Then <strong>2nd most</strong> important stakeholder would be those who have low influence but high interest. These are typically your <strong>line managers</strong> and<strong> team members </strong>working on the same project we should aim to inform them on a regular basis. The 3rd most important stakeholder are those which have high influence but low interest. These are usually your CFOs or head of departments and we should aim to keep them satisfied. The 4th type is usually ignored as they have minimal impact. These are usually office/ground/admin staff who might see your dashboard in a form of information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*R3KNHaEPr8mmNsstxcGezQ.png" /></figure><h4>2. Documentation | Project Planning</h4><p>Once you get the requirements and understand its purpose, we should aim to document this for version control. Version control will help us to document the original requirements and also document any changing requirements which might occur in the future.</p><p>When we implement documentation, we should have 3 different kinds of documents which BRD, FRD and User Stories. <strong>Business Requirements Document (BRD)</strong> outlines high level stakeholder requirements and answers why this project is being initiated.</p><p><strong>Functional Requirements Document (FRD)</strong> gives you much more detailed view of the requirements. It outlines what data or columns needs to be used in order to deliver the dashboard. If the data is not present in data warehouse like SQL Server then it will mentions what logic/formula to use in order to create those columns. It might also show what visuals and numbers the end user want to see (if specified). It might also have formula of numbers they want to see on the visual. Like, “if they want to see 3 circle venn diagram then in each inner circle I want to see a certain threshold number and we would get formula which would help us to get those numbers”. This is the main document which used be used when creating the dashboard.</p><p><strong>User Stories</strong> is a brief explanation of requirements from the perspective of the end-user. It is usually written in a pattern with Want/could/should/would like as the key words. Screenshot below gives a good overview of how it should be written.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*vm37heKUAaWoGuPxEaUl3g.png" /></figure><p>Here we also aim to conduct <strong>Project Scope</strong>. Project scope is bascially revolves around a circle. Everything inside the circle is In Scope and everything outside the circle is Out of Scope. So, if it is In Scope it is your responsibility to do the work which involves what the end user wants to see on the dashboard. If it is Out of Scope it really falls outside of your control when delivering a project. E.g. if additional requirements are given but that data needs to be created as does not currently exist within the data warehouse then it would be considered Out of Scope.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/317/1*LHFTeXVvPWeG6CNOKNrvTQ.png" /></figure><p>Typically, what happens when we are working on delivering a project is that whenever we present our work (to show progress) in a weekly meetings, the stakeholder will add on more requirements. They will be like it wasn’t there in the original requirement but I would like to see this as well. So, what it does is that it extends the boundaries of the Circle so you will end up with more In-Scope work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/1*BqGuwwV6ZAE-C78aXV_Dkg.png" /></figure><p>Here, the developer should set boundaries on how much we can extend the scope because if we carry on extending the boundries of project scope then we would never deliver the project. Instead, we can negotiate with the stakeholder to agree on what they want to see on as part of 1st phase project delivery and anything else they want to add we can have those in the 2nd phase. What we done here is instead of following <strong>waterfall </strong>approach we followed an <strong>Agile </strong>method.</p><p>Agile method can be more efficient as it to sets the ground running instead of waiting to complete all the requirements and then deliver the dashboard. In this way, we would be able to close the project as part of 1st phase and the user will practically start using it. Then, while the user is already working on the dashboard we can release the 2nd phase work.</p><h4>3. &amp; 4. Dashboarding | Project Execution &amp; Project Testing</h4><p>Here, we would eventually start working on the dashboard using BI tool like Power BI or Tableau based on requirements. Once we have the first draft we would present it back to the stakeholder. In this way we will have iterative review meetings with the stakeholder scheduled every week.</p><p>Now the 4th step which is Project Testing also goes hand in hand with Project Execution because we would aim to test the dashboard before delivering first draft. Then at the end of the process when the stakeholder is satisfied with the dashboard we would carry out <strong>final full BI tool test.</strong></p><h4>5. Project Closure.</h4><p>In these steps, we aim to get a sign-off. This step can be a bit time consuming. Some stakeholders might go quite because the end-user might need permission from their line managers to give -”go-ahead” to sign it off. So, this all process might take some time. But, our job is not done until we get the sign-off so it is important to be persistent.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=65f07a452c8e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[All about Popular DAX functions in Power BI | Top 89 functions]]></title>
            <link>https://medium.com/@noueruzzaman/all-about-popular-dax-functions-in-power-bi-top-89-functions-7bb7cfd2a897?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/7bb7cfd2a897</guid>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Sat, 02 Nov 2024 13:28:10 GMT</pubDate>
            <atom:updated>2024-11-02T13:28:10.169Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>DAX (Data Analysis Expressions)</strong> is a formula language used in Microsoft Power BI and Power Pivot for Excel. It enables users to create calculations and measures that provide insights from their data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*SQc98ZISlihzUWYr-x7ZwA.png" /></figure><p>DAX can be used for variety of tasks which includes creating new calculated columns or measures, filtering or sorting data, doing date &amp; time calculations to creating complex calculations involving multiple tables and calculations.</p><p>There are 12 main functions types and the screenshot below seperates the most used to least used functions. The top half shows the most used DAX categories and the bottom half shows the later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*XVmDO-oHio4Eq2epcadsMw.png" /></figure><ol><li><strong>Aggregation Functions</strong> are the most commonly used functions in Power BI. The following screenshot shows the top 18 Agg DAX functions.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*nFHtx-SXgOwIO6NdRkFR8g.png" /></figure><p>2. The next popular functions are <strong>Filter functions</strong>. The following screenshot shows the top 15 Filter DAX functions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/762/1*gsBdPteqan6hqYthBrNIkw.png" /></figure><p>3. 3rd most popular functions are <strong>Logical Functions</strong>. The following screenshot shows the top 8 Logical DAX functions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*SN9PtFoBzoRCRE9KOwO0wg.png" /></figure><p>4. Then the next popular functions are <strong>Text </strong>Functions. The following screenshot shows the top 13 Text DAX functions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*b2G0dVtfoYb6fidCrxNZOA.png" /></figure><p>5. Then the next popular functions are <strong>Table </strong>Functions. The following screenshot shows the top 8 Table DAX functions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*ykFz6WEiDPBj23WHDjKa3w.png" /></figure><p>6. Then the next popular functions are <strong>Date &amp; Time </strong>Functions. The following screenshot shows the top 13 Date &amp; Time DAX functions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*TuG3mHZQQ9OQ5HGmY8rqAg.png" /></figure><p>The following screenshot shows remaining 14 popular functions from other categories.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*j4dnMBdxfXZIrzbDGBK13w.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7bb7cfd2a897" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Openrowset & Polybase (External tabl| Azure Synapse]]></title>
            <link>https://medium.com/@noueruzzaman/openrowset-polybase-external-tabl-azure-synapse-37fa4e606beb?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/37fa4e606beb</guid>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Sat, 02 Nov 2024 10:42:41 GMT</pubDate>
            <atom:updated>2024-11-02T10:42:41.558Z</atom:updated>
            <content:encoded><![CDATA[<h3>Openrowset &amp; Polybase (External tables) in Azure Synapse</h3><p>The following shows 14 points about Openrowset functionfunctions</p><p><strong>OPENROWSET</strong> — It simples gets data from data lake.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DTWnSKCWaz19XiYxIyshug.png" /></figure><p>1. <strong>Bulk </strong>is the file path</p><p>2. <strong>Format </strong>as either csv or parquet</p><p>3. We can use <strong>** </strong>to collect All files within the folder or we can use <strong>*</strong> if we are using a where clause.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*94y2HigsOIDCXkXVZLkxhA.png" /></figure><p>4. Can use Where, in, group, having or order by clauses as normal after the from() statement.</p><p>5. <strong>Filepath</strong>()— it extracts the path component from URL file path. So, if we write filepath(0) it would return container. If we are write filepath(1) then it will return the sales folder.</p><p>6.<strong> Publish </strong>to save the query script</p><p>7. Sometimes when you publish the file, you get no headers. To reassign headers we will use <strong>With </strong>statement after closed parenthesis. You can define column names and data types explicitly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xhbfjw4OfE5nIeLpY4n7oQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/947/1*SC-yDbfpZJ6BfoGyBgIi2w.png" /></figure><p>8. <strong>Collate </strong>— it specifies the sorting order and comparison rules for character data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/1*d4yk8NrVm-ggriaq6X2ntQ.png" /></figure><p>9. For simple queries we will use OPENROWSET. But for more complex queries we would use <strong>PolyBase </strong>by querying external data source. We can create external data source by creating objects in a database with reference to external data location. It can be re-used across multiple queries. It is useful when integrating large datasets which is coming from the data lake without holding data in the database.</p><p>It is created with the key word <strong>Create external data source </strong>&lt;name&gt; <strong>with (location </strong>= &lt;path&gt;<strong>). </strong>Then<strong> run </strong>it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*LiP2VRJGKnh8txUDinrtaw.png" /></figure><p>It then creates a sql database in <strong>Workspace </strong>section in Data pane.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lN24Mq7VTXEdNEV-viQaBA.png" /></figure><p>Then create <strong>New </strong>(empty) <strong>SQL script </strong>then run a select statement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1fHtWO41dC2-HYnuk0VHlQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vKENamR1J26xKw3REj9mDg.png" /></figure><p>10. If we do not have column headers we can run the following code. Where we write <strong>Create external table </strong>and within parenthesis write column names and its associated data types then use <strong>With </strong>statement to say use it with the data <strong>source, location </strong>and <strong>file format</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/1*lvcnlN7Bhpo3AHzadYGqZQ.png" /></figure><p>There is 2nd way to create external tables that is we can use <strong>With </strong>command first (if don’t need to define headers and data types), and then use <strong>As </strong>command. Within the As command we can write Select statement. In the <strong>From </strong>clause use <strong>Openrowset(bulk, data source, format, parser_version, header_row)</strong>. Then we would name this from clause with table name. Then write <strong>End </strong>as the final command.</p><blockquote>Whenever, we use <strong>As </strong>command we would also use finish it with <strong>End </strong>command.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*Nwuseign9jbOLOw3gqr75Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/1*7F1VKiwz3E5oKotgXODe4w.png" /></figure><p>11. Before we use create external table. It is good practice to use <strong>Create external file format</strong> and <strong>With </strong>clause wright arguments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/913/1*ghk5w48TMnGsHWSghImqxQ.png" /></figure><p>12. When you Run the code you will see a table inside <strong>External tables</strong> section.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*03bZeZopfNfelk8iIJzXPw.png" /></figure><p>13. To create any new sql queries you can do it in Develop tab by simply clicking on plus sign.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*0YEOb6vHZjB5N0MibSmcvQ.png" /></figure><p>14. If we want to transform data frequently we can use <strong>stored procedure</strong> to encapsulate CETAS state. We can do that by <strong>AS, BEGIN, CREATE EXTERNAL TABLE, Execute </strong>(at the end) commands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/1*_LqvpR9DISYYMHrsm02r7Q.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37fa4e606beb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is API in Python in 2 min]]></title>
            <link>https://medium.com/@noueruzzaman/what-is-api-in-python-in-2-min-4d5326fbff84?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/4d5326fbff84</guid>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Wed, 16 Oct 2024 17:03:54 GMT</pubDate>
            <atom:updated>2024-10-16T17:03:54.966Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is API in 2 min</h3><p>API stands for Application Programming Interface, In most simplest terms, it provides interface to communication with world of computing. We can understand it more in depth with the following example.</p><p>E.g., you are Spanish traveller and you has came to France for tourism. While, taking a tour in the city of Paris you are lost and are struggling to find the way out. Luckily, you had downloaded a travel app which will help you to navigate through the city. However, there is one problem the app is in French and you can only read in Spanish.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*Kr-q61iOkYWb4spzINK6KQ.png" /></figure><p>Then it came to your mind to use Google Translation to translate words from French to Spanish by simply copying the phrase. This helped you to navigate to where you wanted to go. So, in this example Google Translation is like an API.</p><p>In a nutshell, this was a simplified example of an API.</p><ol><li>API makes your job easier and makes your job more efficient.</li><li>It translates the communication between different systems (you and the app).</li></ol><p>When you are working with API in real world it gives you two things, <strong>Protocol </strong>and <strong>Data</strong>. You just have to read the documentation to how use the API code appropriately and then use <strong>Request</strong> library to call this code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4d5326fbff84" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Clustered Columnstore Indexing in SQL]]></title>
            <link>https://medium.com/@noueruzzaman/columnstore-indexing-in-sql-975a12ddad8d?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/975a12ddad8d</guid>
            <category><![CDATA[clustered-index]]></category>
            <category><![CDATA[columnstore-index]]></category>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Thu, 26 Sep 2024 11:36:52 GMT</pubDate>
            <atom:updated>2024-09-26T14:29:13.197Z</atom:updated>
            <content:encoded><![CDATA[<p>In SQL, you could only use or create row based indexes but in 2012 Microsoft introduced column based indexes and this helped to increase performance as it increased compression ratio by 7x.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*qaQUQ6-Y1ZLhc5QXg5uZqA.png" /></figure><p>Columstore Index be of two types either Clustered or Non-Clustered. Clustered Columstore Index stores all the values in a column are stored together and then next column stores its related data. This way all the columns are stored next to each other on the disk and structure of the Index is used to store data on the disk. As it is column based it does not allow filtering functionality. Whereas, in Non-clustered also saves all the data in a column together on the disk but it unlike Clustered it does not become the physical storage structure of the table. When we say Non-Clustered does not become the physical storage structure we mean the index itself is not the physical representation of the table’s data.</p><p>The main point of using Clustered Index is to speed up performance and save space in disk. Lets look at the time it takes to run normal row-level index codes.</p><p>To check the time it takes to run a code we can <strong>Set Statistics io on and time on</strong> commands.</p><p>If we run everything in the table it took 31ms and had 239 logical reads. Logical reads is a request by a database engine to access a data page from its buffer cache. Buffer cache is like a temporary storage area in the database’s memory. When a query needs data, the engine first checks the buffer cache to see if the required page is already there. If it is, the data is retrieved from memory, which is much faster than reading it from disk.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*03mKCPAx2hxK1F0eFPh48w.png" /></figure><p>Now lets create a copy table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xZotiJF3xdJJCq1HMvMHAw.png" /></figure><p>If we run the timer again, this time it took 16ms to run the code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/777/1*LAHr7Pe3TWlnAKUkOdfY1g.png" /></figure><p>And if I open the copied table you will notice there is no index on the table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*zx-mnKvTd26nfKTc_o5xOw.png" /></figure><p>On this table we would create Clustered Columstore Index by simply right clicking on the index.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*wNZpX0phTemNbKsK5tbnXg.png" /></figure><p>Now after creating it, when I rerun the timer. You will notice that it took 0ms and has 0 logical reads. See, how fast it is with clustered columnstore index.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*rr3i9oJvfu6gjxDEviZ53g.png" /></figure><p>If you want to create it out of scratch, then you can write it in the following manner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ek0evTC_RT2RGNqPZ-BhpQ.png" /></figure><p>However, if we want to create Non-clustered Index we will define the columns on which we want to set the index.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qswjZnqci8ZbXwsd2LINfQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=975a12ddad8d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Difference between Self Hosted IR and AutoSolve IR in Azure Cloud]]></title>
            <link>https://medium.com/@noueruzzaman/why-we-need-self-hosted-integration-runtime-to-connect-move-data-from-on-premise-to-azure-cloud-d7a47d7d5a54?source=rss-340fea656be9------2</link>
            <guid isPermaLink="false">https://medium.com/p/d7a47d7d5a54</guid>
            <dc:creator><![CDATA[Nouer Uz Zaman]]></dc:creator>
            <pubDate>Tue, 24 Sep 2024 17:20:18 GMT</pubDate>
            <atom:updated>2024-09-24T17:21:49.736Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wOTS3l5RPSAkd2DKsZR1hQ.png" /></figure><h4>So, what is Integration Runtime (IR)?</h4><p>Integration Runtime is a fundamental component of Azure Data Factory (ADF) which enables data <strong>integration </strong>between various<strong> data sources</strong> and <strong>destinations </strong>(sink) using Transformation activates (like Data Flows and Data Control). In other words it is like a bridge which connects ADF to underlying systems by using computation power. So, in other words whenever you are moving data from e.g. blob storage to SQL server using copy and transformation activity within ADF then you are using compute power and computer power is provided by Integration Runtime.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/1*NaBCb-KwwGaJ0qcFA9Oobg.png" /></figure><p>Within Azure infrastructure, there are 3 IRs which the user can use to connect to data sources. 1st is <strong>Azure Autoresolve</strong> IR(which is the default choice), 2nd is<strong> Self-hosted</strong> IR and 3rd is <strong>Azure-SSIS</strong> IR.</p><h4><strong>Autoresolve/Azure Integration Runtime</strong></h4><p>Autoresolve Integration Runtime is a feature within Azure Data Factory that automatically creates and manages Integration Runtime resources based on your data factory activities. It simplifies the process of setting up and configuring Integration Runtimes, especially for users who are new to Azure Data Factory or have complex data integration pipelines. If there are no Integration Runtime set up Azure uses Autoresolve Integration Runtime as a default Integration Runtime.</p><p>Autoresolve IR is called by <strong>Random IP addresses</strong>. One of the reasons for this is because resources within the Azure environment is distributed across different data centres or regions and this leads to varying IP addresses.</p><p>If you want to create IR manually and configure it then you would be using <strong>Azure </strong>IR. It is same like Autoresolve IR its just that it provides more control. You set it up by going to <strong>Linked Services</strong> in ADF or Synapse and when you create new linked service it will show default Integration Runtime you can change this by clicking on new and adding Azure IR.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/1*WtqFS9ycR-hGXKCVgOSX_w.png" /></figure><p>Now, when you are working with <strong>private network</strong>, it cannot be reached from any Random IP address or in other words Private network will not allow the connection to form through due to firewall settings.</p><p>Private network can be reached via <strong>Self-Hosted</strong> IR. Self-Hosted IR will connect to a <strong>Fixed IP address</strong>. When you use use a Fixed IP address it gets <strong>whitelisted </strong>by the firewall of your database server and his allows it to form connection. Now, when I mentioned whitelisted what I meant is that the Fixed IP address restricts access to only certain resources so only few computers can access this IP address.</p><p>The 3rd type, Azure-SSIS IR is used when you want to lift and shift existing SSIS packages to Azure Cloud without significant code changes and this type connects via Random IP address like Autoresolve IR.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7a47d7d5a54" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>