<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shachi Kaul on Medium]]></title>
        <description><![CDATA[Stories by Shachi Kaul on Medium]]></description>
        <link>https://medium.com/@shachikaul35?source=rss-ff148a0c040e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*MFsiHSAjl53GFzDSSrm_qQ.jpeg</url>
            <title>Stories by Shachi Kaul on Medium</title>
            <link>https://medium.com/@shachikaul35?source=rss-ff148a0c040e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 23:23:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shachikaul35/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Quantum Entanglement: The Twins in Quantum Realm]]></title>
            <link>https://shachikaul35.medium.com/quantum-entanglement-the-twins-in-quantum-realm-a0bbf1aac84d?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0bbf1aac84d</guid>
            <category><![CDATA[emerging-technology]]></category>
            <category><![CDATA[quantum-computing]]></category>
            <category><![CDATA[quantum-physics]]></category>
            <category><![CDATA[quantum-entanglement]]></category>
            <category><![CDATA[quantum]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Tue, 28 May 2024 06:13:44 GMT</pubDate>
            <atom:updated>2024-05-28T06:13:44.387Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gDIL9Xhgmg9ssEOVai0wFA.png" /><figcaption><a href="https://stock.adobe.com/in/search?k=%22nuclear+fusion%22&amp;asset_id=537881222">PC</a></figcaption></figure><p>What if you could instantaneously affect a particle on the other side of the universe just by changing the state of its twin here on Earth? OR Imagine a world where information travels faster than the speed of light, where two particles remain connected no matter how far apart they are. Well, welcome to the strange quantum realm of quantum entanglement. When I first learned about quantum entanglement, it felt like stepping into a science fiction novel.</p><p>And when an object is found in <strong>multiple states at the same time until they are measured, </strong>that’s another interesting concept of quantum superposition. To understand it more, find the article <a href="https://medium.com/geekculture/can-particles-be-in-two-places-at-once-the-weird-quantum-superposition-c76cd8cc8d93">here</a>.</p><h3>What is Quantum Entanglement?</h3><p>Entanglement is a phenomenon where particles are interconnected and influence each other’s states regardless of their distance. So the instant the state of one particle changes, it reflects some change to another particle at that same instant even if they are miles or millions of years apart.</p><p>Eg,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/255/1*CYhm0zYl4iEXb0SqX_Rx_Q.jpeg" /></figure><p>Let’s imagine miles apart two quantum particles A and B entangled with each other. A change in the spin of Particle A +1/2 changed B’s spin -1/2 implying states are spontaneously linked. It was instant communication, perhaps can say faster than the speed of light to adjust in perfect sync. Followed the non-locality property as it defies the rule for normal space and time.</p><p>So, before the measurement of quantum particles is made, they are in the states of superposition and entanglement. To know about superposition, here’s the <a href="https://medium.com/geekculture/can-particles-be-in-two-places-at-once-the-weird-quantum-superposition-c76cd8cc8d93">article</a>.</p><h3>EPR(Einstein, Podolsky, Rosen) Paradox</h3><ul><li>In 1900s, few scientists started giving evidence for quantum mechanics such as Bohr’s atomic model, Broglie’s wave-particle duality, Heisenberg’s uncertainty principle and Schrödinger’s Wave Equation.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*R0XHii1q9tjh3hXhfxraLg.jpeg" /></figure><ul><li>But Einstein marked quantum mechanics as incomplete. As per him, instant influence is impossible because his theory of relativity states that no influence can be transmitted faster than the speed of light. He coined the “Spooky at a distance” statement. Will get to know later why. Einstein introduced hidden variables since properties of quantum objects remain determined until one measures them and being hidden we couldn’t measure these variables. But they must exist according to EPR.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X8jRLdpdNsPCgamsKDeoDg.png" /></figure><ul><li>That time who could dare to go against Einstein? Bohr said that Einstein was wrong as there were no hidden variables and that the fact that entanglement seemed to create these weird correlations between particles is something we had to accept as reality. 30 years later, John Bell set up an experiment that involved running experiments again and again on pairs of entangled particles. This investigates how strong the correlation is between the outcome of measurements. Bell proved that quantum mech predicted stronger statistical correlations in the outcome of these measurements than any hidden variable theory possible could. When Bell’s exp was first done in a lab in 1970, it showed that there was no sign of hidden variables and that indeed outcome was determined only by the act of measurement itself.</li><li>And so further experiments explained the existence of Quantum Mechanics</li></ul><h3>Spooky Action @ Distance by Albert Einstein</h3><p>The titled statement was coined by Einstein as according to him the instant impact of entangled particles regardless of distance is questionable. It violates the classical physics law of the theory of relativity which states that nothing can travel at the speed of light.</p><p>So w<strong>hy isn’t this “spooky action at a distance “ then? Because </strong>Quantum non-locality is an alternative to this.</p><h3>How to create Quantum Entanglement?</h3><p>Create a pair of photons when an atom is given extra energy i.e. atom emits energy by emitting 2 photons when extra is given.</p><p>Pair Production: Create a pair of entangled particles using a crystal. Split a photon into two.</p><p>Photon entanglement: Use non-linear crystal as when a photon interacts with it, it splits into 2 photos with entangled photons, with obviously having correlated properties.</p><h3>Applications of Entanglement: Beyond the Lab</h3><ul><li><strong>Quantum Cryptography:</strong> Uses entangled particles to securely distribute cryptographic keys (Quantum Key Distribution), and detect eavesdropping in communication channels.</li></ul><blockquote>Today companies such as <a href="http://www.bbn.com/technology/cyber_security/quantum_cryptography">BBN Technologies</a>, <a href="http://www.toshiba.eu/eu/Cambridge-Research-Laboratory/Quantum-Information-Group/Quantum-Key-Distribution/Toshiba-QKD-system/">Toshiba</a> and <a href="http://www.idquantique.com/">ID Quantique</a> use QKD to design ultra-secure networks. The <a href="http://www.newscientist.com/article/dn4914-entangled-photons-secure-money-transfer.html">first bank transfer using entangled QKD</a> went ahead in Austria in 2004. This system promises to be highly secure because if the photons are entangled, any changes to their quantum states made by interlopers would be immediately apparent to anyone monitoring the key-bearing particles.</blockquote><ul><li><strong>Superdense Coding</strong>: Allows the transmission of two classical bits of information using only one qubit, doubling the capacity of communication channels.</li><li><strong>Quantum Teleportation</strong>: Enables the transfer of quantum states from one location to another without physical transmission of the particle itself.</li></ul><p>etc</p><h3>References</h3><ul><li><a href="https://www.smithsonianmag.com/science-nature/five-practical-uses-spooky-quantum-mechanics-180953494/">Five Practical Uses for &quot;Spooky&quot; Quantum Mechanics</a></li><li><a href="https://bigthink.com/starts-with-a-bang/quantum-entanglement-weirder/">Quantum entanglement just got a whole lot weirder</a></li></ul><p><em>Feel free to follow this author if you liked the blog because this author assures you to back again with more interesting Data Science related stuff.<br></em>Thanks,<br>Happy Learning! 😄</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0bbf1aac84d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can Particles be in Two Places at Once? The Weird Quantum Superposition]]></title>
            <link>https://medium.com/geekculture/can-particles-be-in-two-places-at-once-the-weird-quantum-superposition-c76cd8cc8d93?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/c76cd8cc8d93</guid>
            <category><![CDATA[quantum]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[superposition]]></category>
            <category><![CDATA[emerging-technology]]></category>
            <category><![CDATA[quantum-computing]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Mon, 08 Jan 2024 09:46:45 GMT</pubDate>
            <atom:updated>2024-05-20T12:11:56.362Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3k2oAydUA3Cz_cl4FiPkeg.png" /><figcaption><a href="https://www.comicbasics.com/all-loki-variants-from-loki-show-ranked-by-power/">Source</a></figcaption></figure><p>What if I say that there is another variant of yours somewhere in the timeline? What if the science fiction we watch in the Marvel series named Loki does exist? 😵 Some things in quantum mechanics seem very strange. Is it possible that the same object can be in different places? Not one after another but at the same time? 😮 If yes, then why don’t we see it in everyday life (classical world)? The answer is simple, a quantum system requires a very specialized environment and when it gets measured, it is so sensitive that it comes out of it and collapses to one state.</p><p><strong>In Simple Words</strong></p><p>A concept in quantum mechanics describes that an object can be in <strong>multiple states at the same time until they are measured</strong> while in a classical system, there is one well-defined state at a given time.</p><p>So, the electrons, photons, or atoms can exist in more than 1 location simultaneously at least until the measurement is made i.e. something can be “up” and “down” at the same time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/1*MFB-BGnlM-N_N7GUzOpPVQ.png" /><figcaption><a href="https://www.google.com/imgres?imgurl=https%3A%2F%2Ft3.ftcdn.net%2Fjpg%2F05%2F38%2F88%2F62%2F360_F_538886200_VIrvd3t5eRy0NxUhKPEs4JY28HmNUy6c.jpg&amp;tbnid=i-4f1V1pSfOBoM&amp;vet=10CAIQxiAoAGoXChMI8K-OjJjxggMVAAAAAB0AAAAAEA8..i&amp;imgrefurl=https%3A%2F%2Fstock.adobe.com%2Fsearch%2Fimages%3Fk%3Dsuperposition&amp;docid=T-Fz39YvePWT5M&amp;w=575&amp;h=360&amp;itg=1&amp;q=superposition%20illustration&amp;ved=0CAIQxiAoAGoXChMI8K-OjJjxggMVAAAAAB0AAAAAEA8#imgrc=JTU7J5J_rOchiM&amp;imgdii=z3aYse0UdNLPAM">Source</a>, Figure 1</figcaption></figure><p>During superposition, particles having different states can be rethought as the different outcomes with the probability observed. Once measured, the quantum system collapses into one state. These states are referred to as “<strong>wave function</strong>” which are combined through superposition and on measurement collapse into a single state. We’ll get to know more about wave function later.</p><p><strong>Mathematical representation</strong></p><p>In easy terms, something that has more than 1 solution for an equation. Eg, x² = 9 where x=-3 and x=3.</p><ul><li>Superposition is the combination of all possible states in the Herbert space before it collapses to one state during the measurement of a quantum system.</li><li>The nature of the outcome is always probabilistic, unlike classical physics. There we have a determined solution but here it’s always a probability value of a particular thing to happen. This value is calculated using the wavefunction.</li><li><strong>Wave function</strong> represents a quantum state which contains information about the particle’s position, momentum, and other observable properties. The square of the absolute value of the wavefunction (|Ψ|²) gives the probability density of finding a particle at a specific location.</li></ul><blockquote>Superposed wave equation, |ψ⟩ = α |ψ1⟩ + β |ψ2⟩</blockquote><p>where <br>|ψ⟩ : Qubit state,<br>(α,β): complex numbers, <br>(|α|²,|β|²): probability amplitude for finding system in states (ψ1,ψ2) respectively.</p><p>Hence, the qubit state of 0 and 1 i.e. |ψ⟩ = α|0⟩ + β|1⟩</p><p>In Figure 1, Qubit has 0 (spin down) and 1 (spin up) as 2 possible options for spin direction at the same time.</p><p><strong>What happens on measurement?</strong></p><p>Before we measure(observe) the system, it’s in a superposition state that is in multiple different states at the same time. When we measure, superposition collapses into one well-defined state giving us the illusion of always one state in our classical world. This well-defined one has concluded with the squared magnitude of coefficients i.e. probability of each state.</p><p><strong>How does Quantum Computer leverage Superposition?</strong></p><p>Quantum Computer has quantum gates which create and manipulate the superpositions of states. This sounds a bit strange and interesting. But similar to logic gates and circuits, we have quantum gates and quantum circuits in quantum mechanics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/607/1*U2u87BRQdj5b-n72kVMYZw.png" /><figcaption>Quantum Computer (<a href="https://www.newscientist.com/article/2390568-why-havent-we-got-useful-quantum-computers-yet/">Source</a>)</figcaption></figure><p>The superposition property of qubits is what gave the power of exponential parallelism to Quantum Computers. Quantum parallelism provides exponential speedup than classical computers.</p><p>There are other such Quantum Mechanics concepts such as entanglement and interference. Find an article <a href="https://medium.com/p/a0bbf1aac84d/edit">here</a> to understand entanglement and interference will also be covered soon in later blogs. Till then, try to think if you could also have your variant somewhere. 😉</p><p><em>Feel free to follow this author if you liked the blog because this author assures you to back again with more interesting Data Science related stuff.<br></em>Thanks,<br>Happy Learning! 😄</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c76cd8cc8d93" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/can-particles-be-in-two-places-at-once-the-weird-quantum-superposition-c76cd8cc8d93">Can Particles be in Two Places at Once? The Weird Quantum Superposition</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[9 Psych Effective Ways to Emotionally Detach]]></title>
            <link>https://shachikaul35.medium.com/9-psych-effective-ways-to-emotionally-detach-e210524c205f?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e210524c205f</guid>
            <category><![CDATA[life]]></category>
            <category><![CDATA[peace]]></category>
            <category><![CDATA[life-lessons]]></category>
            <category><![CDATA[life-hacking]]></category>
            <category><![CDATA[love]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Wed, 28 Dec 2022 18:15:08 GMT</pubDate>
            <atom:updated>2023-05-09T03:11:31.009Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Someone asked, “Why do I feel so much? I want to forget that person but can’t. Feels like a pendulum letting that person go/come in my mind.”</blockquote><blockquote>I answered, “it&#39;s just the expectations and emotions playing with you.”</blockquote><p>THE MIND. It plays a very vital role in reacting to your body. It listens to your thoughts and hormones get released which ultimately shape your body. A happy person isn’t just mentally fit but physically too. Research says that mental/emotional pain is worse than physical pain which can even cause some illnesses/diseases. <br>It&#39;s easier said than done to get past your person. Some or the other time when that vulnerable nerve hits and you have your flashbacks again. It&#39;s never about deleting but in fact making it so old that it starts appearing blurry. In life, it&#39;s about chapters getting added but never deleted. They may get old that they almost seem forgotten. Stop replaying the memories in your mind and make those old chapters fresh. No emotion is bad or good. It is how we manage that emotion that makes us more emotionally intelligent. :)</p><p>There are often times when you are left with no other choice but to forget that person to live in peace. Letting go is never easy, especially for the person you loved the most. Some tips and tricks which could be useful to ease your pain and get over the attachment. Though it also depends on a person’s nature.</p><ul><li><strong>Realize a strong reason</strong> to get detached and replay it if you feel so intense to go back.</li><li><strong>Stop over-thinking</strong>. Do not chase your thoughts unless you can’t do anything about them in real. The human brain has a tendency to create the simplest thing into the most complex just by thinking unnecessarily more.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*8MyvriZ1eP1U9KGO87PHHw.gif" /></figure><ul><li>Go out to the <strong>positive vibes place</strong>. Could be a temporary solution but a step towards thinking with a clear and peaceful mind. Go out for a walk, surround yourself with nature or give a break around mountains, beaches, or greens.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*8-XyZ21_W72buEn-TvHowg.gif" /></figure><ul><li><strong>Meditation and Yoga</strong> definitely help. But this will work when you are not so high in instability state because for meditation also you need to have a moderate level of peace. Yoga is about breathing so focusing on your breath helps to maintain your blood and stress levels.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*yi8iPsRM-a317CojzCio0A.gif" /><figcaption><a href="https://tenor.com/view/meditation-hippie-patrick-patrick-star-spongebob-squarepants-gif-5888486">Source</a></figcaption></figure><ul><li>It’s believed that the overwhelming state is the best moment when you can find yourself more creative. <strong>Utilize this flooding energy</strong> into positive which could be useful for you. After all, in a later stage of life, you could say that at least it helped me to achieve this. <br>Perhaps perfect time to focus on yourself more. Go for <strong>upskilling</strong> yourself. <strong>Learn new things. Create new good habits. Perhaps a new hobby. Hit a gym. Travel and make new contacts.</strong> Do something which includes the engagement of both your mind and body together.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*j-R8_SMwjya1qJEn8juGpg.gif" /><figcaption><a href="https://giphy.com/explore/learn-something-new-every-day">Source</a></figcaption></figure><ul><li>Watch out for some <strong>positive videos</strong> which act as an external push to hope and think differently from a different perspective.</li><li>This time you wanted to be alone in solitude but the fact is if you are too emotional at the moment to handle yourself then give yourself some time. <strong>Hang out</strong> <strong>with people.</strong> Do not share the grievance because repeating it will gonna just multiply the impact (depending if that person is a solution-giver). After some time when coming back into your sane senses, give a thought on why you reacted so and think for a solution.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*ksuWeheOqJgDLAr4-hSicg.gif" /><figcaption><a href="https://tenor.com/view/team-hanging-out-hanging-out-team-mag-team-magazine-gif-7528483">Source</a></figcaption></figure><ul><li>Sometimes writing the stuff out not only release your emotions but also help you in bringing towards a solution. So start <strong>journaling</strong> and pen down the emotions to not overload them inside you.</li><li><strong>Give a call to some of your old buddies</strong> with whom you lost contact for years. Trust me, it gives joy to both the sides and perhaps you feel more connected now than before.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*jhH_AKMOfAe1o46QyzJBQA.gif" /><figcaption><a href="https://tenor.com/search/friends-call-gifs">Source</a></figcaption></figure><p>Most importantly, <strong>do not forget your self-worth</strong>. It&#39;s all in your mind. Play psychologically with it.</p><p>It’s a saying,</p><blockquote>Do not even joke about yourself because your body doesn’t know the difference. Your subconscious mind captures the words as energy and words cast spells that’s why it’s called spelling. So, change the way you speak about yourself.</blockquote><p>It’s just a matter of your emotions and hormones. You are unbelievably strong and a director of your own life. You have come so far. Do not let it get thrashed because of some another person, just as of granted.</p><p><em>Feel free to follow this author if you liked the blog because this author assures you back again with more interesting life lessons and mantras.<br>Thanks,<br>Peace</em>! 😄 👼</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e210524c205f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Microsoft Power BI Series 101 — Extract, Transformation and Load (ETL)]]></title>
            <link>https://shachikaul35.medium.com/microsoft-power-bi-series-101-extract-transformation-and-load-etl-c5e60af75026?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5e60af75026</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Thu, 15 Dec 2022 17:23:40 GMT</pubDate>
            <atom:updated>2022-12-15T17:23:40.582Z</atom:updated>
            <content:encoded><![CDATA[<h3>Microsoft Power BI Series 101 — Extract, Transformation and Load (ETL)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1szZV44Z8hYffpEW" /><figcaption>Photo by <a href="https://unsplash.com/@inakihxz?utm_source=medium&amp;utm_medium=referral">Iñaki del Olmo</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Power BI is a Business Intelligence tool offering reporting and analytical capability. It could be defined as:</p><blockquote>“ <strong><em>The Power BI Platform</em></strong><em> is a Microsoft cloud solution with various software services, connectors and apps that help to consolidate and ingest data from multiple unrelated sources, perform various pre-processing, transformations and finally utilize them to deliver useful reports.”</em></blockquote><p>It comes with various services such as PowerBI Desktop, PowerBI Service(Web App) and Mobile PowerBI. This blog covers the ETL ways to be done via the Web App and Desktop version. Check<a href="https://k21academy.com/microsoft-azure/data-analyst/what-is-power-bi/"> this page</a> for more elaborated info.</p><h3>What is ETL?</h3><p>ETL is abbreviated from Extraction, Transformation and Load. A process of ingesting (extraction), transforming and then loading the transformed data into a warehouse. So basically <strong>moving data from any kind of source to the destination</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*cidKziixLjFMJTJn6MZgbA.png" /><figcaption><a href="https://rivery.io/blog/etl-vs-elt/">Source</a></figcaption></figure><p><strong>Extraction</strong>: Data in varied formats of relational (RDBMS), NoSql, text, files, XML etc can be extracted by using Power BI connectors.</p><p><strong>Transformation</strong>: Process of converting the extracted raw data into a certain standard that can be used for creating a data model and reporting.</p><ul><li>Cleaning, sorting and filtering operations</li><li>Feature engineering and selection</li><li>Bringing a column to a specific standard</li></ul><p><strong>Load</strong>: Transformed data is loaded into the data warehouse</p><h3>Is Power BI an ETL?</h3><p>Microsoft Power BI is <strong>not an ETL; it’s a BI tool that follows the ETL methodology</strong>. It’s a Business Intelligence and Analytical tool that creates interactive reports and dashboards.</p><p>So, after the loading activity, a destination in Power BI is not a warehouse that is there in an ETL process, instead, it’s always a Tabular model. It implies that the loaded data will always be in a structured format. So, loading data creates a dataset in PBI.</p><h3>ETL in Power BI</h3><p>The data extraction phase is achieved via Power BI API connectors such as SQL Server, Text/JSON, Excel, Azure SQL Database etc.</p><p>Power BI provides some ETL roles and features:</p><ul><li>Can connect from <strong>varied data sources</strong></li><li>There is a <strong>separate visual editor</strong> to practice the <strong>data transformation</strong></li><li>The transformation steps done visually are written into an M query language; can be seen via Advanced Editor</li></ul><p>Power BI, on the other hand, using <strong>Dataflows and Power Query</strong>, can only move the data to Power BI, creating datasets. Let’s see the practical steps for both ways.</p><ul><li><strong>ETL with Power Query Editor (Power BI Desktop)</strong></li></ul><p>Power Query maintains a step-by-step record of every action taken to ingest, transform and load data. This enables us to do a wide range of transformations right from removing necessary columns to adding up new ones with some transformations in columns.</p><h4><strong>Implementation</strong></h4><ol><li>Clicking on “Get Data” pops up a window asking for choosing a type of Power BI connector and fetch the respective source data</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6rIbryCcG5bDq7UvbJjuUw.png" /></figure><p>2. After loading the data, you will be navigated to the “Power Query Editor” window where you can perform various transformation techniques</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xzt6YQ0iAzKGII0BqVp_3Q.png" /></figure><p>Each of these transformation steps is captured as in steps under the Applied steps pane located toward RHS</p><p>Also, all these applied steps are recorded in M Query language which can be seen in Advanced Editor (highlighted in the image below)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_Jo-WVMfhP3Klb_TtjC2LA.png" /></figure><ul><li><strong>ETL with online PowerQuery Editor via Power BI Dataflows (Power BI Service)</strong></li></ul><p>Dataflows is another way to ingest your data; it’s a simple data pipeline that connects to the data source such as SQL Server DB, Sharepoint file etc. It <strong>can’t be created in the Desktop tool, instead only the cloud-service tool</strong> inside your workspace. But after creation, it can be utilized in the Desktop app too. To avail the Dataflow services, you need to have a <strong>premium license</strong>.</p><p><strong>Where the data from dataflows are stored?</strong><br>It is an independent data transformation item of your report. The tables out from dataflows are stored either in the internal Azure Data Lake Storage Gen2 (ADLS) or Dataverse.</p><blockquote>Dataflow is the data transformation in the cloud independent of the Power BI dataset.</blockquote><p>Schedule the refresh of dataflows so as to keep the report updated with the source data.</p><p><strong>Merits</strong></p><ul><li>It becomes difficult when we want to have the same transformational logic from a report in another one. <strong>Resuing the steps</strong> isn’t yet available! Here, Dataflows could be created having these steps and can be directly imported.</li><li><strong>Whenever new logic is created</strong>, dataflow just needs to be refreshed.</li></ul><h4><strong>Implementation</strong></h4><p>As said earlier, it can be only created via the PowerBI service.</p><ul><li>Go to your workspace and select the DataFlow option</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/1*kl271v0QNuo_PQxCk9AWbA.png" /></figure><ul><li>There are 4 ways to create a dataflow but for the showcase, go through the “Define new tables” option.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/317/1*D37dlFV0sjam3SDAvTp6aA.png" /></figure><ul><li>Select the appropriate data source. I have chosen Excel workbooks stored in my OneDrive.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/653/1*ZVabqsim3jfRQT0Ex2ruSA.png" /></figure><ul><li>Clicking on the Next above will hop into the below image where load the data.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*O8X0v3P5EgV3ElFscBQe_A.png" /></figure><ul><li>Also, you can add more transformation steps here such as filtering rows or other processing.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/653/1*17jtL_XC8Y_XNYfjXT87kA.png" /></figure><ul><li>Save &amp; Close the window. Give a name to the dataflow which resides in your workspace now.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/385/1*RvK9ZHSs5QXLr5PVRDRBoQ.png" /></figure><p>The transformation steps created via an Advanced Editor are stored in a dataflow. These steps can be applied to any report by simply importing this data flow.</p><h3>References</h3><ul><li><a href="https://www.sqlshack.com/an-introduction-to-power-bi-dataflows/">An introduction to Power BI Dataflows</a></li><li><a href="https://radacad.com/what-are-the-use-cases-of-dataflow-for-you-in-power-bi">What are Power BI Dataflows and their Use Cases?</a></li></ul><p><em>Feel free to follow this author if you liked the blog because this author assures you to back again with more interesting Data Science related stuff.<br></em>Thanks,<br>Happy Learning! 😄</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5e60af75026" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[So Be It!]]></title>
            <link>https://shachikaul35.medium.com/so-be-it-b8cba8e872c2?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8cba8e872c2</guid>
            <category><![CDATA[life]]></category>
            <category><![CDATA[life-lessons]]></category>
            <category><![CDATA[poem]]></category>
            <category><![CDATA[poetry]]></category>
            <category><![CDATA[inspiration]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Fri, 02 Sep 2022 17:01:18 GMT</pubDate>
            <atom:updated>2022-09-02T17:01:18.858Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*QpXqnpIrC2DxEfJ3w8llaA.png" /></figure><p>The Life.<br>You plan. You execute. <br>bearing some sweet fruits,<br>letting you feel all genius.<br>or <br>The bitter sour taste<br>ends up realizing...<br>Shit happens.<br>Well, so be it!</p><p>The Silent.<br>What made you?<br>Situations. Expectations. People. <br>Why not start adapting?<br>“Do not hold if it goddamn hurts, <br>because letting it go magically works.”<br>Anyway, so be it!</p><p>The Flashbacks. <br>Why aren’t they gone?<br>That nerve when it gets hits, <br>rewind all the emotions buried<br>with the nostalgia hitting hard.<br>The us-time moments.<br>The funny sweet-angry-happy emotions.<br>The never-to-be-broken promises.<br>Yes, things happen for a reason.<br>Well, so be it!</p><p>The Regrets. <br>Why aren’t there any? <br>’Cause, you did right<br>per the demand of that time.<br>So…<br>do not bother.<br>Time flies. <br>People change.<br>And, so be it!</p><p>The Faith. <br>Why still holding? <br>Even though, <br>Life twists and turns. <br>Time rolls its dramatic dice.<br>Still…<br>Sit back tight. <br>Because…<br>Deeper inside you still hope <br>for the best coming.<br>Mornings after Nights. <br>Beginnings after Endings.<br>Letting go makes the better ones in.<br>Well, so be it!</p><p>The High-time.<br>Chin up!<br>Held head high!<br>Be the North Star!<br>Because no matter <br>what’s done is done!<br>Let your shine intensifies. <br>Make yourself proud. <br>If that means living your life<br>in your own beautiful way, <br>SO WELL, GODDAMN, BE IT!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8cba8e872c2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Everyone Had a Story]]></title>
            <link>https://shachikaul35.medium.com/everyone-had-a-story-871d43fd1b?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/871d43fd1b</guid>
            <category><![CDATA[poetry]]></category>
            <category><![CDATA[life]]></category>
            <category><![CDATA[poetry-writing]]></category>
            <category><![CDATA[life-lessons]]></category>
            <category><![CDATA[poetry-on-medium]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Sat, 18 Jun 2022 10:51:15 GMT</pubDate>
            <atom:updated>2022-06-18T10:51:15.698Z</atom:updated>
            <content:encoded><![CDATA[<p>This poetry is dedicated to those who devoted once to something in life but the results got tossed opposite.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*Bi85JGTRvyFY8Cd0AmnzqA.png" /><figcaption><a href="https://liveboldandbloom.com/06/self-awareness/important-life-lessons">Source</a></figcaption></figure><p>Everyone had a story,<br>intense good or more worse,<br>buried in deep debris of history.<br>History where chapters though become old<br>but never got ceased or erased<br>and again may bring your life on hold. <br>Those chapters may get faded<br>but sooner or later life rolls a die<br>refreshing all your odds and olds<br>to never leave a chance to ‘sigh’. 😩</p><p>So,<br>though everyone had a story,<br>intense good or worse <br>but the choice is yours;<br>be a crying baby or a solution chaser<br>because its always your call<br>to bake the brand new stories <br>and let the bad ones be dumped forever. 😃</p><p>Even though everyone had a story,<br>though its hard to escape the history<br>but life is full of learnings and experiences.<br>So,<br>Stand on your feet,<br>Weave new chapters, <br>Take a few chances. <br>’Cause you may fall or rise again<br>but this time it’s guaranteed<br>you are more of a life-learned churned human<br>no matter what<br>will always held head high again! 🙌</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=871d43fd1b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Text Feature Extraction (3/3): Word Embeddings Model]]></title>
            <link>https://medium.com/geekculture/text-feature-extraction-3-3-word-embeddings-model-e98f3d270dce?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e98f3d270dce</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Mon, 03 Jan 2022 06:02:28 GMT</pubDate>
            <atom:updated>2022-01-03T06:02:28.324Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*x9UAQ8ucKTnwSHjgWk2UfA.jpeg" /><figcaption><a href="https://www.namesnack.com/guides/nature-photography-business-names">Source</a></figcaption></figure><p>In the previous parts of the Text Feature Extraction series, we have seen that in NLP the text-based data could be modeled. The feature extraction techniques such as <a href="https://shachikaul35.medium.com/text-feature-extraction-1-3-bag-of-words-model-649dbeeade79">BagOfWords </a>and TF-IDF were already discussed. So, here we will be learning about another technique called Word Embeddings. Word Embeddings has become a very crucial and interesting approach with a variety of its uses in building text models. Let’s find out more!</p><h3>Word Embeddings</h3><h4>Aim</h4><p>It aims to identify the semantics and contextual information of text which wasn’t possible in BOW and TFIDF.</p><h4>Detailed Study</h4><ul><li><strong>Intuition:</strong> Similar context (words) tends to have similar vector space or embeddings and are located closer to each other.</li><li><strong>What? </strong><br>- Technique to convert <strong>text into real-numbers vectors</strong> into a low-dimensional space. These vectors in low-D are called <strong>embedding</strong>. <br>- Word Embeddings aka Word Vectors to represent words<br>- It represents each word as a numerical vector with a specific dimension. These numbers sustain the similarity among various other word vectors that indirectly store the semantic info.<br>Eg, Girl = [ ..,..,..] and Women = [..,..,..] are closely related.</li><li><strong>Mapping </strong>of words into vector space</li><li>Since vectors, arithmetic operations are now possible in a text. <br>Eg, King-Boy+Girl = Queen<br>King vector (royal + male) is subtracted to Boy (normal + male) and added to Girl (normal + female) results into Queen (royal + female)</li><li>Let’s see <strong>how does this vector looks </strong>like.<br>Vector with a particular dimension let’s say 300 dimensions if using the English corpus (en_core_web_lg). These 300 are various possible features of a word in English. Each feature is a certain trait of that word that is not human interpretable but obtained by the ML model. Thus, can’t say exactly what describes. <br>Eg, For Queen, 300-D vector where numbers represent the magnitude of a feature as shown in Fig-1</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/221/1*v7SWSAMFxYvlXlnjG59sng.png" /><figcaption>Fig-1</figcaption></figure><ul><li><strong>Why did Word Embeddings come?</strong><br>Let’s bring up a scenario<br>Vocab words = 5000<br>A vector Queen projects at the 5th index in the vocab. Usually, text conversion into text in BOW and TFIDF uses the word count. <br>Queen = [0,0,0,0,1,0..00..00..0] with 5000-D<br>This results in the whole sparse vector with just 1 place non-zero value. Thus, comes Word Embedding. <br>So exact reasons:<br>1. <strong>Sparsity:</strong> Word Embedding uses real-value vectors of specific feature dimensions instead of 0s. The size of the embedding vector is smaller.<br>2. <strong>Semantics &amp; contextual:</strong> Learns by considering context words closer to your target word which wasn&#39;t possible in TF-IDF previous models.<br>3.<strong> Relationships and similarity:</strong> Vectors allow to perform arithmetic operations this it easier to find similar words. <br>4. Each word is not treated as a feature but infact a vector with a fixed dimension. Thus, algorithms of LSTM/GRU would not be complex and easy to train on.</li><li><strong>Applications:</strong> Sentimental Analysis, Speech Recognition, Information retrieval</li><li><strong>Utilization </strong>of Word Embeddings:<br> — Predicts the target/ context words<br> — Reduces dimensionality</li></ul><h4>Word Embeddings Umbrella</h4><p>Word Embedding is an umbrella term with the concept of pre-trained models (Word2Vec, GloVe) or embedding layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*_qHRQD2SY3Hol-lF2w6rPg.png" /><figcaption>Fig-2</figcaption></figure><h4>Pretrained Model</h4><p>- Uses pre-trained word vectors to build a model<br>- These word vectors came by training the embedding from a huge corpus<br>- Sklearm’s <strong>Gensim </strong>library provides pre-trained models and APIs such as Word2Vec, GloVe, FastText, etc.</p><h4>1.1 Word2Vec</h4><ul><li>Google’s Neural Network embedding model which trains the network to learn relations among words from a large corpus of text</li><li>It’s trained on 100 billion google news datasets having the vocab of 3 million words</li><li>It embeds the relations among words into a low-dimensional vector space</li><li>Produces word embeddings</li><li><strong>2-layer Neural Network</strong> of input, hidden, and output layer</li><li>Crux: Here just like any NN, the aim is not just to predict the probability but instead to get the trained weights (vector)of the hidden layer.</li><li>The weights of the hidden layer learned by the model are used as word embedding</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/461/1*BecfA72_331Ob6W0bnJrZg.png" /><figcaption>Fig-3</figcaption></figure><ul><li>It plays with predicting the probability of context words and the target word per the word2vec variant architecture</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*s8a8Dh0g1k-Vm55TG7DPzQ.png" /><figcaption>Fig-4, Word2Vec Variants Architecture: <a href="https://arxiv.org/pdf/1301.3781.pdf">Source</a></figcaption></figure><ul><li>After training, we get the trained vectors of each word in the vocabulary and the vectors closer (similar) to each other are placed closer in vector space.</li><li>Let&#39;s see the variants of the W2V model.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*P769v-ZSIIobHs-eAvIk-w.png" /></figure><h4>1.2 GloVe (Global Vectors)</h4><ul><li>My research found me a perfect definition which says—</li></ul><p><em>“GloVe is a count-based, </em><strong><em>unsupervised learning model</em></strong><em> that uses </em><strong><em>co-occurrence</em></strong><em> statistics at a </em><strong><em>Global level</em></strong><em> to model the vector representations of words.”</em></p><ul><li>Definition from the original paper <a href="https://nlp.stanford.edu/pubs/glove.pdf">https://nlp.stanford.edu/pubs/glove.pdf</a> states that —</li></ul><blockquote><em>GloVe is based on ratios of probabilities from the word-word co-occurrence matrix, combining the intuitions of count-based models resulting into capturing the linear structures used by methods like word2vec.</em></blockquote><ul><li><strong>Why named so? </strong><br>Because the mathematics of matrix is around considering <strong>global-level</strong> properties of the whole dataset while local properties are considered in Word2Vec models. It means that the models of Word2Vec brings out the statistical numbers considering words using window length but GloVe considers the whole dataset to create co-occurrence matrix.</li><li><strong>Main idea</strong><br>This word-embedding algorithm found by Stanford University Researchers works on a concept of depicting the relation between words based on statistics (probability and counts) using co-occurrence matrix.</li><li><strong>Mathematics in brief<br></strong>GloVe uses co-occurrence matrix where each value represents how those two words appear together. It considers co-occurrence probabilities as a ratio to differentiate two relevant words rather than the probabilities themselves.</li><li>It is <strong>trained on the co-occurrence matrix</strong> which signifies that how frequently a pair of words occur together in a corpus.</li><li>Let&#39;s understand some notations and the co-occurrence matrix</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*mlyEWNkrp4ww3flvGiLnaQ.png" /></figure><p>Probabilities and ratios from a large corpus</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*WYsyim29Jm21ULgUZ8Jo6Q.png" /></figure><p>The above table resembles that the P(probability) of solid in the context of ice is high but with steam is low. Since the numerator is a higher ratio of both is &gt;1 which is opposite for the word gas.</p><ul><li>You can download the pre-trained word vectors from <a href="https://nlp.stanford.edu/projects/glove/">https://nlp.stanford.edu/projects/glove/</a></li></ul><p><strong>Check out my </strong><a href="https://github.com/shachi01/NLP/blob/main/glove_v1.0.ipynb"><strong>GitHub repo</strong></a><strong> implementing the GloVe.</strong></p><h4>2. Embedding Layer</h4><ul><li>Instead of using pretrained model, you can train your own word embeddings uisng Keras Embedding layer.</li><li>Keras has an Embedding Layer to train neural network on textual dataset. It is a type of hidden layer specified after an input layer which takes the integer-encoded data.</li><li>Usually any textual data is comverted into numeric using some encoding techniques (lets say one-hot encoding). This creates dummy features for each categorical values. Seems like not feasible for big text data. Hecne, Embedding layer converts each word into fixed-length vector. It maps higher dimensional data into lower dimensional vector space.</li><li><strong>Uses of Embedding Layer:</strong><br>1. The embedding layer alone could be used as in <strong>another model </strong><br>2. Contributes in t<strong>ransfer learning</strong>; used to load pre-trained embedding model<br>3. Learned itself as a part of <strong>deep learning model</strong></li><li>Refer to <a href="https://medium.com/analytics-vidhya/understanding-embedding-layer-in-keras-bbe3ff1327ce">blog</a> for the implementation in brief</li></ul><h3>References</h3><p><a href="https://nlp.stanford.edu/projects/glove/">https://nlp.stanford.edu/projects/glove/</a><br><a href="https://jalammar.github.io/illustrated-word2vec/">https://jalammar.github.io/illustrated-word2vec/</a><br><a href="https://www.youtube.com/watch?v=Fn_U2OG1uqI">https://www.youtube.com/watch?v=Fn_U2OG1uqI</a></p><p><em>Feel free to follow this author if you liked the blog because this author assures you to back again with more interesting ML/AI-related stuff.<br></em>Thanks,<br>Happy Learning! 😄</p><p><strong><em>Can get in touch via </em></strong><a href="https://www.linkedin.com/in/kaul-shachi"><strong><em>LinkedIn</em></strong></a><strong><em>.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e98f3d270dce" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/text-feature-extraction-3-3-word-embeddings-model-e98f3d270dce">Text Feature Extraction (3/3): Word Embeddings Model</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Shore]]></title>
            <link>https://shachikaul35.medium.com/the-shore-73fb5f157cdb?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/73fb5f157cdb</guid>
            <category><![CDATA[poem]]></category>
            <category><![CDATA[love]]></category>
            <category><![CDATA[poetry]]></category>
            <category><![CDATA[poems-on-medium]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Fri, 24 Dec 2021 18:24:56 GMT</pubDate>
            <atom:updated>2023-03-02T18:37:21.829Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*vBMJTn1raV6t8NyqjDycZQ.jpeg" /><figcaption><a href="https://unsplash.com/photos/Pv7LzcSVsNg">Source</a></figcaption></figure><p>The spinning wheel of love at the shore,<br>When awfully flashbacks the moments you adore.<br>Reviving back the human long ago left,<br>All the way starts from how you met.<br>But better you come to your senses sooner,<br>Because life is not a sweet candy-gun shooter.<br>Reminds you of the fantasy you were earlier in<br>thinking the two shores as together<br>which in reality were so apart; I do believe in.</p><p>The chapter finally got closed<br>from the other side of the shore.<br>There is no way back to that damn door<br>The door to cherish those lovely moments again<br>When you know it’s all the fantasy,<br>bound to never happen again!<br>The pain, the hurt, the sleepless nights<br>Seems like a dark tunnel which I always fright.</p><p>Whatsoever, <br>It is worth remember<br>You are intelligent,<br>Yet you are beautiful.<br>You are fearless,<br>Yet you are unique.<br>You are priceless,<br>Yet you are simple.<br>You are all a precious masterpiece,<br>Yet in this weird hustle-bustle at your shore<br>You figured to finally attain the inner peace.</p><p>Stepping ahead walking the other shore, <br>leaving the footprints behind,<br>which I don&#39;t care about anymore!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73fb5f157cdb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[9 Reasons for Why IT Professionals Should Blog]]></title>
            <link>https://medium.com/geekculture/9-reasons-for-why-it-professionals-should-blog-54fa480978a7?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/54fa480978a7</guid>
            <category><![CDATA[blogging]]></category>
            <category><![CDATA[blogging-tips]]></category>
            <category><![CDATA[developer-stories]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Fri, 12 Nov 2021 06:32:58 GMT</pubDate>
            <atom:updated>2021-11-12T06:32:58.931Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*TFUwH7WRlv2vYDIeSzfkKw.jpeg" /></figure><p>Why being an IT professional should I blog?<br>How can blogging contribute to my personal brand?<br>Can blogging open more opportunities for me in the market?<br>If you also have likewise questions...<br>then this place is the best<br>to escort you towards your solution. 😃</p><p>I started blogging long years back which really acted like a rocket launcher. Why? Because it boosted me to learn and explore more along with increasing my network. It not only improved my articulation power but also crystal cleared the concepts I write.<br>There are various reasons that why any learner especially a developer should blog. So, let’s cut to the chase to explore them.</p><h4>1. Improvise Story-Telling</h4><p>Understanding the concept is easier than articulating the same to others. Blogging enhances your articulation skill where you need to think of how to start and end your topic while keeping the audience engaged. Story-telling comes with experience and so wherever you go, if you are not good at it then your work may seem less significant to others.</p><p><strong>Golder Rule:</strong> A project with creative story-telling will popularise you more.</p><h4><strong>2. Writing &amp; Communication Skill</strong></h4><p>Blogging enhances your writing skills since it makes you think about the structure of framing your content. It should be designed in such a way that an audience should feel in sync with the story. This ultimately furnishes your communication and writing skills.</p><p><strong>Golden Rule:</strong> More your language is simpler, the more it would be understood and people would love to read. 😉</p><h4><strong>3. Quick Refresher</strong></h4><p>I often forget the concepts where I invested enough time to understand them. At the time of implementing it, I have to refresh the concepts by again investing an ample amount of time. Here blogging will prevent you from studying from the scratch since a quick read of your very own content will save you time.</p><p><strong>Golder Fact: </strong>Reading your own words is much quicker and easier to grasp than from the other sources. 💁</p><h4>4. Deepen the Roots of Knowledge</h4><p>When you want to scribble your very own understanding, you search for the words to articulate it. And the secret of getting more words is to have enough knowledge which ultimately urges you to explore various sources of information. This, at last, gives you more ideas and intensifies your existing knowledge too.</p><p><strong>Golden Fact:</strong> Writing ▶️ More Investigation ▶️More Ideas ▶️More Knowledge</p><h4>5. Happy to Help</h4><p>How does it feel when your content is getting fruitful to people? It’s a feeling which may boost you to write more since creating content wasn’t only useful for you but also for others in need which might have created opportunities for them. <br>My relatable personal experiences are so sweet that it really boosts me to write more just like what I am doing now. I get positive comments/messages on LinkedIn from people saying that how my blogs were so useful to them.</p><p><strong>Golden Fact:</strong> Your happiness in publishing your work gets doubled when someone shows gratitude for making life easier by creating such <em>simple-to-understand</em> content. 👼</p><h4>6. Reinforcement Learning</h4><p>Feedback on your content helps to self-analyze the writing and makes you aware of scribbling more smartly the next time. Also, sometimes you mistakenly mislead the content which could be highlighted by the feedbacks given and ultimately enhance your learning too.</p><p><strong>Golden Rule:</strong> Be open to criticism because you grow more from the critics. ❤️</p><h4>7. Self-Branding</h4><p>It’s YOU. Your blog. Your ownership. A perfect marketing strategy to make yourself visible to a broader network of people. This demonstrates who you are and embrace the opportunities which may also boost your career path.</p><h4><strong>8. Resume Building</strong></h4><p>Everyone knows that blogging needs proper research, time, and a box full of soft/hard skills. Thus, mentioning this in your resume enhances your self-brand and gives a good impression too.</p><h4>9. Network and Opportunities</h4><p>It can be implicitly understood that publishing blogs give you higher visibility and especially at this time of the digital era you can reach across the globe. Blogging not only increases your connections but also the chance of getting more opportunities.</p><p>Now, let’s talk about something which should be followed by any blogger. Atleast, it has made my life much easier to scribble. 😅</p><h4>1. Open To Criticism</h4><p>Always open your arms for criticism because that’s what could be your booster to write much better than before. Making your article reach maximum people increases your chances to grow more. Why? Because everyone has their own perspective and feedback/comments from them will give you an idea to look from a different angle the next time you publish.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FzgW89swZx5ZNEkJnSo%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fonepeloton-peloton-selena-samuela-zgW89swZx5ZNEkJnSo&amp;image=https%3A%2F%2Fmedia1.giphy.com%2Fmedia%2FzgW89swZx5ZNEkJnSo%2Fgiphy.gif%3Fcid%3D790b7611c6532ec703fc4aff275db2889b4521464e51ba17%26rid%3Dgiphy.gif%26ct%3Dg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/590edc514ff9501331850d080ad5c70a/href">https://medium.com/media/590edc514ff9501331850d080ad5c70a/href</a></iframe><h4>2. Nobody is Perfect</h4><p>Wanna make a quick reality check? Nobody is perfect!<br>To see the perfect colors of your work, you need to accept the fact that nobody is perfect. We all learn from our experiences to become a better version of ourselves. It’s not a shame but instead a journey to reach our goal towards perfectionism.</p><p><strong><em>So! Forget your comfort zone! And start writing! </em>😊</strong></p><p><em>Feel free to follow this author if you liked the blog because this author assures you to back again with more interesting ML/AI-related stuff.<br></em>Thanks,<br>Happy Learning! 😄</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=54fa480978a7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/9-reasons-for-why-it-professionals-should-blog-54fa480978a7">9 Reasons for Why IT Professionals Should Blog</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Text Feature Extraction (2/3): TF-IDF Model]]></title>
            <link>https://medium.com/geekculture/text-feature-extraction-2-3-tf-idf-model-c3a8f7a92bc9?source=rss-ff148a0c040e------2</link>
            <guid isPermaLink="false">https://medium.com/p/c3a8f7a92bc9</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[tfidf-vectorizer]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Shachi Kaul]]></dc:creator>
            <pubDate>Tue, 29 Jun 2021 09:02:07 GMT</pubDate>
            <atom:updated>2025-02-08T09:50:31.392Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*dudfjJ-iDfR4WzYjahhgXw.jpeg" /><figcaption><a href="https://unsplash.com/s/photos/books-and-laptop">Source</a></figcaption></figure><p>In Natural Language Processing, any text-based problem needs to be converted into a form that can be modeled. A simple text could be converted into features with various techniques such as Bag of Words (BOW), TF-IDF, or Word embeddings. In the <a href="https://medium.com/geekculture/text-feature-extraction-1-3-bag-of-words-model-649dbeeade79">last blog</a> of the Text Feature Extraction series, we studied about CountVectorizer from scratch and its usecase on text classification. We deciphered there that it has a major drawback of lack of semantic meaning. <strong>CountVectorizer </strong>considers the word count to create a feature hence it <strong>doesn&#39;t take the sentence structure and order</strong> into account. Also, it results in a large sparse matrix. Thus TF-IDF comes into the picture.</p><p>This blog series projects the detailed TF-IDF technique. So, let’s get started.</p><h4><strong>Aim</strong></h4><p>The TF-IDF technique gives you the relevant term. This relevant term is the one by which the whole context can be understood instead of reading the whole text.</p><h4>Intuition</h4><ul><li>The word occurring multiple times implies its importance (TF).</li><li>But at the same time if it appears across multiple docs too frequently then it may not be relevant (IDF). These words we can refer to as stopwords such as the, this, etc.</li></ul><h4>Theory &amp; Concept</h4><ul><li>TF-IDF (Term Frequency-Inverse Document Frequency) depicts the importance of a word. It is the computation of the dot product of TF and IDF. Before this, let’s understand the terms individually first.</li><li>Term Frequency (TF):<br>- It demonstrates the importance of the word to a doc with an intuition that the more the term in doc means higher the importance is.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/373/1*Mlws6f7IVUrYq16pWrIwFw.png" /><figcaption>equation-1</figcaption></figure><ul><li>Inverse Document Frequency (IDF):<br>- Shows how a term is actually relevant. It is not necessary that term frequently in some docs could be relevant such as stopwords (the that, of, etc). Stopwords do not reveal the context and thus these should be avoided. IDF works in such a way that it ignores them. <br>- It penalizes the word appearing frequently across docs<br>- The IDF score is higher for the relevant term while lower weight to the stopword<br>- It considers natural logarithmic function aka log e</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/373/1*pm_UDKx71xCZqdLTmmYrmQ.png" /><figcaption>equation-2</figcaption></figure><ul><li>Nutshell, TFIDF value relates to doc while IDF depends on corpus</li><li>Computing TF-IDF manually is different from Sklearn’s TF-IDF. <br><strong>Difference:</strong> TF term remains the same while IDF term differs. Let’s dive in! <br><strong><em>Standard TF-IDF</em><br></strong>The standard notation is shown below where N is docs across corpus and n is doc having term m.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/173/1*nkn9gLX4rH8tKY4431sldw.png" /><figcaption>equation-3</figcaption></figure><p><strong><em>Sklearn TF-IDF</em><br></strong>By definition, Tfidf should work as the above formula but Sklearn provides some more advanced computation. TfidfVectorizer and TfidfTransformer compute differently than the standard as follows:<br>- Adding 1 to the numerator <br>- Adding 1 to the denominator to prevent zero division<br>- Adding 1 to the whole log term as a constant for smoothing<br>- Resulting TF * IDF vectors then normalized with L2 (Euclidean)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/173/1*2iTNn68Z4WKSUBY2hgtDqw.png" /><figcaption>equation-4</figcaption></figure><p>Curious for more, visit this <a href="https://stackoverflow.com/questions/24032485/difference-in-values-of-tf-idf-matrix-using-scikit-learn-and-hand-calculation">StackOverflow link</a> and a <a href="https://towardsdatascience.com/how-sklearns-tf-idf-is-different-from-the-standard-tf-idf-275fa582e73d">blog</a> with in-depth detail.</p><h4>Process</h4><p>Let’s cut to the chase and study the steps to build a TF-IDF model. We will be following the standard notation <em>equation-3</em>.</p><p>corpus = “She is a wanderlust”, “She is lovely”</p><ol><li><strong>Compute TF</strong>: Refer to equation 1.</li></ol><p>TF_doc2 (“is”) = 1/3 <br>TF_doc2(“lovely”) = 1/3</p><p><strong>2. Compute IDF</strong>: Refer to equation-3.</p><p>IDF(“is”) = log (2/2) = 0<br>IDF(“lovely”) = log (2/1) = 0.30<br>As can be seen clearly that the weightage of “is” is less than the “lovely”. Thus lovely seems more relevant.</p><ul><li><strong>Dot product of TF and IDF word:</strong></li></ul><p>TF-IDF (“is”) = TF . IDF = (1/3) * 0 = 0<br>TF-IDF (“lovely”) = (1/3) * 0.3 = 0.09</p><p>The results showed that the word “is” is irrelevant while “lovely” holds some importance. Reading just the word “lovely” distinct the sentence.</p><p>Result summary as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/514/1*yntwSSsJGiCgQ_0DdN4__Q.png" /><figcaption>table-1</figcaption></figure><h4>Implementation</h4><p>To understand the TF-IDF model, let’s first see how to implement manually then will for Sklearn implementation.</p><ol><li><strong>Manually</strong></li></ol><ul><li>Let’s create our corpus of sentences and convert them into lowercase to not differentiate between “this” and “This”.</li></ul><pre>doc = &quot;She is a wanderlust”, “She is lovely”<br>#Convert into lowercase<br>doc = list(map(str.lower, doc))</pre><ul><li>Create a bag of words for each doc in the corpus</li></ul><pre>cv = CountVectorizer()<br>count_occurrences = cv.fit_transform(corpus)</pre><pre>#For doc1<br>bagOfWords_1 = dict.fromkeys(cv.get_feature_names())<br>for ind,key in enumerate(bagOfWords_1):<br>    bagOfWords_1[key] = count_occurrences.toarray()[0][ind]<br>bagOfWords_1<br>Out[2]:<br>{&#39;is&#39;: 1, &#39;lovely&#39;: 0, &#39;she&#39;: 1, &#39;wanderlust&#39;: 1}</pre><pre>#For doc2<br>bagOfWords_2 = dict.fromkeys(cv.get_feature_names())<br>for ind,key in enumerate(bagOfWords_2):<br>    bagOfWords_2[key] = count_occurrences.toarray()[1][ind]<br>bagOfWords_2<br>Out[3]:<br>{&#39;is&#39;: 1, &#39;lovely&#39;: 1, &#39;she&#39;: 1, &#39;wanderlust&#39;: 0}</pre><ul><li>Compute TF</li></ul><pre>def compute_tf(bow, doc):<br>    tf_dict ={}<br>    doc_count = len(doc)<br>    for word, count in bow.items():<br>        tf_dict[word] = count/doc_count<br>    return tf_dict<br>tf_doc1 = compute_tf(bagOfWords_1, corpus[0].split(&#39; &#39;))<br>tf_doc2 = compute_tf(bagOfWords_2, corpus[1].split(&#39; &#39;))<br>tf_doc1</pre><pre>Out[4]:<br>{&#39;is&#39;: 0.25, &#39;lovely&#39;: 0.0, &#39;she&#39;: 0.25, &#39;wanderlust&#39;: 0.25}</pre><ul><li>Compute IDF</li></ul><pre>def compute_idf(docs):    <br>    N = len(docs)<br>    idfDict = dict.fromkeys(docs[0].keys(),0)<br>    for doc in docs:<br>        for word, val in doc.items():<br>            if val &gt; 0:<br>                idfDict[word] +=1 <br>    <br>    for word, val in idfDict.items():<br>        # standard notation<br>        idfDict[word] = math.log(N / float(val))<br>        #sklearn notation<br>        #idfDict[word] = (math.log((N+1) / (val+1))) + 1<br>    return idfDict<br>idfs = compute_idf([bagOfWords_1,bagOfWords_2])<br>idfs</pre><pre>Out[5]:<br>{&#39;is&#39;: 0.0,<br> &#39;lovely&#39;: 0.6931471805599453,<br> &#39;she&#39;: 0.0,<br> &#39;wanderlust&#39;: 0.6931471805599453}</pre><ul><li>Compute TF * IDF</li></ul><pre>def compute_tfidf(tf,idf):<br>    tfidf = {}<br>    for word, tfVal in tf.items():<br>        tfidf[word] = tfVal * idf[word]<br>    return tfidf<br>tfidf_doc1 = compute_tfidf(tf_doc1, idfs)<br>tfidf_doc2 = compute_tfidf(tf_doc2, idfs)<br>tfidf_doc1</pre><pre>Out[6]:<br>{&#39;is&#39;: 0.0, &#39;lovely&#39;: 0.0, &#39;she&#39;: 0.0, &#39;wanderlust&#39;: 0.17328679513998632}</pre><p><strong>2. Sklearn</strong><br>The above scratch steps could be done in just a few lines of code. Scikit-learn provides a library called TfidfVectorizer which computes the Tfidf weights.</p><pre>vectorizer = TfidfVectorizer()<br>vectors = vectorizer.fit_transform(corpus).todense()<br>vectors<br>&quot;&quot;&quot;<br>matrix([[0.50154891, 0.        , 0.50154891, 0.70490949],<br>        [0.50154891, 0.70490949, 0.50154891, 0.        ]])<br>&quot;&quot;&quot;</pre><h4>Drawbacks</h4><ul><li><strong>Context understanding</strong><br>BagOfWords and TFIDF techniques lack the understanding of context. TFIDF could decipher the sentence structure but not the context.</li><li><strong>Large vocabulary</strong><br>In the case of a large vocabulary, features become so voluminous that it challenges memory and time.</li></ul><p>To overcome the above cons, a new feature extraction came called Word Embedding. Let’s learn about this in <a href="https://medium.com/geekculture/text-feature-extraction-3-3-word-embeddings-model-e98f3d270dce"><strong>Text Feature Extraction (3/3)</strong></a>: Word Embedding Model.</p><p><strong>Check out my </strong><a href="https://github.com/shachi01/NLP/blob/main/TFIDF_Sklearn_Scratch.ipynb"><strong>GitHub repo</strong></a><strong> summarizing all the code demonstrated here.</strong></p><p><strong>Also, feel free to grasp more on how to develop a TF-IDF model to predict the movie reviews sentiments via this </strong><a href="https://github.com/shachi01/NLP/blob/main/TFIDF_MovieReviews.ipynb"><strong>GitHub repository</strong></a><strong>.</strong></p><h3>References</h3><ul><li><a href="https://www.freecodecamp.org/news/how-to-process-textual-data-using-tf-idf-in-python-cd2bbc0a94a3/">How to process textual data using TF-IDF in Python</a></li><li><a href="https://towardsdatascience.com/how-sklearns-tf-idf-is-different-from-the-standard-tf-idf-275fa582e73d">How Sklearn’s “TF-IDF” is different from the standard “TF-IDF”?</a></li></ul><p><em>Feel free to follow this author if you liked the blog because this author assures you to back again with more interesting ML/AI-related stuff.<br></em>Thanks,<br>Happy Learning! 😄</p><p><strong><em>Can get in touch via </em></strong><a href="https://www.linkedin.com/in/kaul-shachi"><strong><em>LinkedIn</em></strong></a><strong><em>.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c3a8f7a92bc9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/text-feature-extraction-2-3-tf-idf-model-c3a8f7a92bc9">Text Feature Extraction (2/3): TF-IDF Model</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>