<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lera Tsayukova on Medium]]></title>
        <description><![CDATA[Stories by Lera Tsayukova on Medium]]></description>
        <link>https://medium.com/@ltsayukova?source=rss-8eb8ece974bf------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*zwknjD9ZunwGmflqtclKAA.jpeg</url>
            <title>Stories by Lera Tsayukova on Medium</title>
            <link>https://medium.com/@ltsayukova?source=rss-8eb8ece974bf------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 05:26:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ltsayukova/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Distance Measures for Machine Learning]]></title>
            <link>https://ltsayukova.medium.com/distance-measures-for-machine-learning-5455ce0919f7?source=rss-8eb8ece974bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/5455ce0919f7</guid>
            <dc:creator><![CDATA[Lera Tsayukova]]></dc:creator>
            <pubDate>Wed, 30 Jun 2021 19:39:32 GMT</pubDate>
            <atom:updated>2021-09-23T03:09:37.221Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*nSBuKX1m3dHtZbgYNWIXKA.jpeg" /></figure><p>What is a <strong>distance measure?</strong></p><ul><li>A distance measure is simply a means of calculation between two points or objects. An objective score that summarizes the relative difference between two objects in a domain space.</li></ul><p>Why is it important to know what a distance measure is?</p><p>-Distance measures have many applications in machine learning, for both supervised learning and unsupervised learning.</p><p>Two such instances are:</p><ol><li>K Nearest Neighbors (supervised)</li><li>K-Clustering (unsupervised)</li></ol><p>In the KNN algorithm, a prediction is made for new points by calculating the distance between the new example (in this case row) and all examples (other rows) in the dataset.</p><ol><li><strong>Euclidean Metric</strong></li></ol><blockquote>Euclidian distance takes two points and determines the shortest distance by measuring the hypotenuse of the triangle between those points.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/105/0*3LztQGewUYWwxcTO.png" /></figure><p>between two elements</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/303/0*01U0n6bge34SuSOH.png" /></figure><p>and</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/295/0*oggNV7KDwvh36xDg.png" /></figure><p>is given by</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/0*la66MZjRu1a0nRXB.png" /></figure><p>Where Euclidian Distance usage :</p><ul><li>L2 Normalization</li><li>Ridge</li><li>Root Mean Squared Error</li></ul><p>The calculation for Euclidian Distance is simple. Measure the shortest distance between two points. However, this metric is not applicable in all scenarios. Imagine the instance below: You are trying to get from point 1 to point 2. Now, unless you are a bird or a genius with a hovercraft board, Euclidian distance is not a practical calculation for the the shortest distance to get from p1 to p2. Why?</p><p>Euclidian distance in this example measures A, which is given by the formula above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*COGsIXj9PfuZ3z3Qilmy_A.png" /><figcaption>Portland Grid (image from Google Earth) using Euclidian Metric</figcaption></figure><p><strong>Euclidian Distance </strong>in this example measures A, the hypotenuse of the triangle between two points p1 and p2.</p><p>If we can imagine the origin point p1 as being (0,0) and the destination point p2 as (3,4) then we can calculate the Euclidian distance as follow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*qoaKCZTaOf8D5GoLAbBoPw.png" /><figcaption>Euclidian Distance formula for (0,0) (3,4)</figcaption></figure><p>Naturally, no one wants to calculate these formulas by hand so below is an example of the SciPy way of handling this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4pgXjbz8oVaXCwZcTEza2Q.png" /><figcaption>Calculating Euclidian Distance using SciPy</figcaption></figure><p><strong>2)Manhattan Metric</strong></p><p><em>The sum of the absolute differences between points across all dimensions</em>.</p><p>Given by the Formula:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/762/1*M-QiwWIu_D5aE9YhnJP0CQ.png" /><figcaption>Manhattan Distance Formula</figcaption></figure><p>Manhattan Distance usage:</p><ul><li>L1 Normalization</li><li>Lasso</li><li>Mean Absolute Distance</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*2zuWNjPiZTwBPRWX1O-E_w.png" /><figcaption>Portland Grid (image from Google Earth) using Manhattan Metric</figcaption></figure><p><strong>Manhattan Metric<br></strong>- This metric does not use diagonals, only horizontal and vertical measures.</p><p>Here the distance is calculated by B and C.</p><p>If we can imagine the origin point p1 as being (0,0) and the destination point p2 as (3,4) then we can calculate the Manhattan distance as follow:</p><p><strong>||X||= |3–0| +|4–0| = 7</strong></p><p>As you can see in the examples above, the Manhattan Distance is calculated in a way that replicated how a taxicab drives between Manhattan city blocks to reach its final destination!</p><p>Again, here is the SciPy example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yxfe3LntP27794PIpjTB0Q.png" /><figcaption>Calculating Manhattan Distance using SciPy</figcaption></figure><p><strong>3)Minkowski Metric</strong></p><p><em>The generalized form of Euclidean and Manhattan Distance.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/1*0_v4yQVxq-uWpNtWU_uEZA.jpeg" /><figcaption>German mathematician Hermann Minkowski</figcaption></figure><p>We typically think about distance when it comes to measuring the distance between or within cities. The Minkowski distance measures distance “in a space where distanced can be represented as a vector that has length.”</p><p>Naturally, one wonders does a map have vector space? Let’s use this definition and start by going back to our map example. The distance between the two points is the vector A, which connects p1 and p2. We could also combine multiple vectors to create a route that connects more than two points along the route (imagine an Uber Eats ride that makes another stop between the pickup point and your house, no priority delivery here). The <em>normed </em>part of the vector simply means that it has a length and a vector wouldn’t be a vector if it had no length or negative length. So in this sense, we have met the requirements of the definition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*JVp2GHxOmQhApIQA8Evb6w.png" /><figcaption>Minkowski Distance</figcaption></figure><p>p = 1, Manhattan Distance</p><p>p = 2, Euclidean Distance</p><p>p = ∞, Chebychev Distance</p><p>In using the formula for Minkowski distance, we simply insert p=1 for Manhattan or p=2 for Euclidian Distance. Although it is technically defined for any number, p is rarely used for except for 1, 2, or infinity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*YkXcaQAa0pLXS3nd_5t4_w.png" /><figcaption>Minkowski Distance measurement</figcaption></figure><p>The way distances are measured by the Minkowski metric of different orders between two objects with three variables ( In the image is displayed in a coordinate system with x, y, z-axises).</p><p>Again, let us cheat a bit and employ Minkowski Method in SciPy:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o52n94Dkzozs--hpuqs0xQ.png" /><figcaption>Calculating Minkowski Method using SciPy</figcaption></figure><p>Now, that we have covered the <strong>Euclidean Distance</strong>, the <strong>Manhattan Metric,</strong> and the <strong>Minkowski</strong> (Chebychev) <strong>Distance</strong>, let’s do a short recap:</p><ol><li>Euclidian Distance</li><li>Manhattan Metric</li><li>Minkowski Distance</li></ol><p>References:</p><ul><li><a href="https://www.analyticsvidhya.com/blog/2020/02/4-types-of-distance-metrics-in-machine-learning/">Understanding Distance Metrics Used in Machine Learning</a></li><li><a href="https://machinelearningmastery.com/distance-measures-for-machine-learning/">4 Distance Measures for Machine Learning - Machine Learning Mastery</a></li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FIZW7U0Ksiwg&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DIZW7U0Ksiwg&amp;image=http%3A%2F%2Fi.ytimg.com%2Fvi%2FIZW7U0Ksiwg%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/304d0262008ac5ea5f0e3d45bdd3abeb/href">https://medium.com/media/304d0262008ac5ea5f0e3d45bdd3abeb/href</a></iframe><p><a href="https://dataaspirant.com/five-most-popular-similarity-measures-implementation-in-python/">Five most popular similarity measures implementation in python - Dataaspirant</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5455ce0919f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Science behind Music: Coding edition]]></title>
            <link>https://medium.com/geekculture/the-science-behind-music-coding-edition-151cf66bb6cc?source=rss-8eb8ece974bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/151cf66bb6cc</guid>
            <category><![CDATA[mozart-effect]]></category>
            <category><![CDATA[magenta]]></category>
            <category><![CDATA[music]]></category>
            <category><![CDATA[lo-fi]]></category>
            <category><![CDATA[coding]]></category>
            <dc:creator><![CDATA[Lera Tsayukova]]></dc:creator>
            <pubDate>Mon, 14 Jun 2021 19:24:28 GMT</pubDate>
            <atom:updated>2021-09-22T19:51:43.056Z</atom:updated>
            <content:encoded><![CDATA[<p>As Aristotle one famously mused, music “makes the hearts of men glad: so that on this ground alone we may assume that the young ought to be trained in it.”</p><p>No doubt, there has been a heap of research, particularly in the last 40 years, done regarding the effects of music on the human psyche. In particular, researchers have been interested in determining to what extent music influences creativity, comprehension, and memory.</p><p>If you retain nothing else, let this blog convince you, my reader, that incorporating music into your coding life can only be beneficial. Just as in machine learning models, the vast amount of variables has made it difficult for scientists to definitively establish a direct impact between music and its promising effects. Nevertheless, the research is hopeful:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*SIXH0iSFkgRLrNLtYTUbFg.jpeg" /></figure><ol><li>Music improves comprehension.</li></ol><p>Known as the <strong>Mozart</strong> effect, and the arousal-mood-hypothesis state that listening to background music can foster learning,</p><p>A study done in Germany concluded: “<em>Due to the Mozart effect, comprehension will be influenced positively and directly by background music”.</em></p><p>2. It may even improve memory.</p><p><em>“There is a lack of research investigating the interaction between listening to background music and working memory capacity:” </em>(Lehmann, Seufert 2017)</p><p>Oh no. Did my entire theory just go flying out the window? Well, there is a highly popular theory, that floats among scientific circles, which concludes that music negatively effects memory. However, Lehmann and Seufurt point out here that not only has little research been done on the the direct impact of music and memory but that studies which have been published and point to a negative correlation have largely <strong>failed</strong> to distinguish between different genres of music.</p><p>Luckily, there is a whole host of research that has since rectified this problem. A study done by scientists at Trevecca Nazareth University, examined the effects of two music genres — classical and rap — on memory. Subjects were dived into three groups, classical, rap, and silence and then given a memory task to complete.</p><p>There were two hypotheses.</p><p>A) Hypothesis 1: predicts that there would be a difference between the three groups (classical, rap, and silence) on a memory task score. The first hypothesis was supported.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/1*Hc9lXCbHU7DgJV5GVINa8Q.png" /></figure><p>The data was analyzed using a one-way ANOVA test which showed a significant difference between the groups, with a p-value of 0.02.</p><p>B) Hypothesis 2: predicts the classical music condition would produce the best performance on a memory. This hypothesis was also supported.</p><p>The group exposed to classical music competed the memory task with fewer flips than the other groups.</p><p>So, clearly the difference of genres of music DOES matter. Unfortunately, even though classical music has proven benefits, it is not a highly popular genre among younger generations. Fortunately, there is a genre of music which has promising feedback.</p><p>3. Music can transform your environment to be more conducive to your coding experience; meet Lo-Fi.</p><p>Lofi music:</p><ul><li>Lo-fi → Low Fidelity music</li><li>In technical terms Lo-fi refers to low quality music, most often referring to the imperfections that would typically be deemed as “errors” during the recording process.</li><li>It’s fuzzy</li><li>Examples: muffled noises, background effects, soothing weather sounds</li><li>mellow rhythm somewhere between 70 and 95 bpm</li></ul><p>Lo-fi has been used in music therapy:</p><ul><li>Michael Viega, a music therapist at Montclair State University, uses Lo-fi beats to helps his patients in stressful environments. His research on using lo-fidelity music to help, <em>“people achieve hi-fidelity and clarity on individual, community, cultural, and spiritual levels of existence”</em>, is linked below.</li><li>Viega took noises that were typically considered stressors, bustling city sounds, bustling nurse stations, beeping of medical machines, and transformed those noises to a level of subtlety noticeable enough to be nostalgically soothing when combined with a backbeat.</li></ul><h4>The Roots of Lo-Fi Music:</h4><ul><li>Although Lo-Fi music can span across all genres, its foundations are rooted in hip hop.</li><li>The genre has actually been around for much longer than Youtube has.</li></ul><p>Now that we’ve covered the research on the benefits of music — specifically Lo-fi — how it relates to coding, and what the origins are let’s see (better yet hear) some examples!</p><p>Well, if you want to create your own Lo-Fi beats: meet Magenta!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nWJEBx2xuMLYKfDFGPEJQw.png" /></figure><ul><li><strong>Magenta</strong></li><li>a customizable LoFi player, from Google that is powered by their TensorFlow System.</li><li>Magenta’s source code on Github</li><li>click on objects in the moods and customize the levels and tempo to your nostalgic hearts desire</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1qV571YYqWcaQoTNYfRktw.png" /><figcaption>Magenta’s customizable source code</figcaption></figure><p>But, if you would rather just jump right into it, here are some LoFi playlists:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FyP2FZNP6eZk%3Fstart%3D3%26feature%3Doembed%26start%3D3&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DyP2FZNP6eZk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FyP2FZNP6eZk%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d1784023c42b9b269c00d4901e157009/href">https://medium.com/media/d1784023c42b9b269c00d4901e157009/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FbdQuy2oe5kU%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DbdQuy2oe5kU&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FbdQuy2oe5kU%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2f97eee494a5b4fd476ba66e92af57ca/href">https://medium.com/media/2f97eee494a5b4fd476ba66e92af57ca/href</a></iframe><p>For more information check out the following:</p><p><a href="https://scholar.utc.edu/cgi/viewcontent.cgi?article=1214&amp;context=mps">https://scholar.utc.edu/cgi/viewcontent.cgi?article=1214&amp;context=mps</a></p><ul><li><a href="https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01902/full">The Influence of Background Music on Learning in the Light of Different Theoretical Perspectives and the Role of Working Memory Capacity</a></li><li><a href="https://voices.no/index.php/voices/article/view/2228">Listening in the Ambient Mode: Implications for Music Therapy Practice and Theory</a></li><li><a href="https://voices.no/index.php/voices/article/view/2228/1983">Listening in the Ambient Mode: Implications for Music Therapy Practice and Theory</a></li><li><a href="https://magenta.tensorflow.org/lofi-player">Lo-Fi Player</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=151cf66bb6cc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/the-science-behind-music-coding-edition-151cf66bb6cc">The Science behind Music: Coding edition</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pandas Tricks:]]></title>
            <link>https://ltsayukova.medium.com/pandas-tricks-5963a8722d2e?source=rss-8eb8ece974bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/5963a8722d2e</guid>
            <category><![CDATA[pandas-dataframe]]></category>
            <category><![CDATA[jupyter-notebook]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Lera Tsayukova]]></dc:creator>
            <pubDate>Thu, 03 Jun 2021 16:39:14 GMT</pubDate>
            <atom:updated>2021-06-03T16:39:14.450Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Eiu4aPI2z6T9CsWs5pvIhg.jpeg" /></figure><h3>Pandering to the Data Science Student Masses</h3><p>There are a<strong> variety of ways to do things in pandas </strong>— translation: This is both a<em> blessing</em> and a <em>curse</em>. The novice user of Pandas (ie. me) can easily get confused. So while there exists probably more than one way to do any of the following these are the methods I found the least intimidating.</p><h4>Create a Random DataFrame (Quickly!):</h4><ul><li>Say you want to give an example or how to do something in Pandas, but you don’t want to bother scrolling through endless folders to find an existing dataset to work with. Solution: Create one with Pandas!</li><li>Using the — pd.random.rand() — you can quickly generate an example DataFrame:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LpEu0WY6dJPsWgTm9LnHNg.png" /><figcaption>Optionally pass in a string list to columns=</figcaption></figure><p>Pandas by default will generate a DataFrame of numbers for both the column and row names, but you can pass it a list of strings as shown up in order to make it easier to work with. The input generates:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RE3ApQZpll3Ko4qlEqEu3A.png" /></figure><p>Let’s say you decided you hate the column names, whether they are of your choosing or the columns of another dataset whose names make as much sense as Carol Baskin’s alibi….</p><p>You can quickly rename any Pandas DataFrame columns by inputting df.rename() and passing in a dictionary. Don’t forget to specify which axis you are working on!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w4rdzhMi4kFT82nXBgLaqQ.png" /></figure><p>And just like that we have renamed (almost) all our columns:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FIf8Q90-q9vDGprQOpoTyQ.png" /><figcaption>Okay fine, after we renamed HALF our columns:</figcaption></figure><p>We can also add a prefix, or get rid of a prefix if need be:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RzVF5gCdm1IajQwBU0xj1g.png" /><figcaption>Adding prefix to all of our columns</figcaption></figure><p>Ahhh this is terrible. CHANGE IT BACK!!</p><p>Not to worry: this mistake can easily be remedied by using df.columns.str.replace() method as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r7jUzYj1K7JKYuW-4lTQ6w.png" /><figcaption>Using the df.columns.str.replace() feature</figcaption></figure><h3><strong>2). D-Types:</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/346/1*Fe1pu7t0ZyVVKkgos1yHvA.jpeg" /></figure><p>Next, we examine D-Types and how to interact with them in order to get more out of our data exploration. For this we will be using our movies dataset:</p><p>In any dataset, we usually have a mixture of datatypes or D-types which sometimes makes it increasingly frustrating to work with and infuriating to discover after you have already error-red out. Believe me I share your rage…As such here are some quick and dirty examples of how to filter out d-types:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ZwFR_gxllE3wtMuDf6pXw.png" /></figure><p>Here in our movie dataset we can see a variety of objects, integers, and floats.</p><p>Let’s say you ONLY wanted to look at the numerical values in this dataset. To do this simply employ the following: df.select_dtypes(include=) as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nrd5wuqSumoOkIfhEWsJxg.png" /><figcaption>Using df.select_dytpes(include=””) feature</figcaption></figure><p>Luckily, this feature also works in reverse! Let’s use the <strong>‘exclude’</strong> feature to <strong>only include numbers:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wsYkMWAtJ3VBPZOyE6EmhQ.png" /><figcaption>Using select_dtypes(exclude= ) feature</figcaption></figure><h3>3) Subsetting DataFrames</h3><p>Lastly, lets look at Pandas DataFrames themselves, namely how to create subsets of a DataFrame.</p><p>If for any reason you want to split up your DataFrame, Pandas has an answer to that. It will allow you to create two subsets of data according to the parameters you set, whether that be 75:25 or 60:40 you get the idea.</p><p>Let’s go back to our movie dataset. It has 5043 rows. For whatever reason we don’t need this many and have decided that we would like ehh roughly 4000 movies instead. In order to subset our ‘movies’ DataFrame we call the following function: df.sample(frac= , random_state= ). In practice it looks much like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F1a25BDXpZWgaycdLkTHNw.png" /><figcaption>How to subset DataFrame using df.sample()</figcaption></figure><p>The ‘random_state’ simply splits up the data set randomly which the ‘frac’ takes the percentage of the data set you want to subset. Now we can create a second subset with the remaining 20% of our movies data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zp-PpFPBj42GoVH5stlVoQ.png" /><figcaption>Creating a second subset by dropping our first subset from our original DataFrame using df.drop()</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lvruFHMRz0FO4Fdc_gup4w.png" /></figure><p>Here we can see the index numbers of our two movie subsets. And confirm that their concatenation will add up back to the original DataFrame.</p><p><strong><em>Note: This method will only work if you have unique id’s for your index.</em></strong></p><p>Hopefully, the above methods have been insightful or at the very least offer some help to the struggling data science students out there. I feel your pain.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5963a8722d2e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python Tips & Tricks:]]></title>
            <link>https://ltsayukova.medium.com/python-tips-tricks-14a3294b82fa?source=rss-8eb8ece974bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/14a3294b82fa</guid>
            <dc:creator><![CDATA[Lera Tsayukova]]></dc:creator>
            <pubDate>Wed, 03 Mar 2021 19:31:20 GMT</pubDate>
            <atom:updated>2021-03-03T19:31:20.088Z</atom:updated>
            <content:encoded><![CDATA[<h3>Ice Cream and Pickles</h3><p>For those of us just starting out in our data science journey, working with Python and Jupyter Notebooks can be a bit daunting. Codespeak sometimes reads like Japenese and trying to search for useful tips among the plethora of StackOverflow information can be be utterly overwhelming. The following is dedicated to fellow food lovers and those on lunch break simply looking for tips to make your life easier and to optimize your time.</p><p>Here are two tools I found useful:</p><h3>1) Ice Cream</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*Y-OezTx_95k8sESELagOHA.jpeg" /></figure><p><strong>Ice Cream </strong>is a python library that takes the print() function and optimizes it, making your output more readable.</p><p>Do you ever get tired of writing print()? Especially when you need to see more than one output at a time?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*efGoyf8uTfWHc2UImphOYA.png" /></figure><p>It gets especially tedious when you want to see what that output is referring to. For example, what does 30 belong to?</p><p>Well, we could do this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0P-UsyiGa7pYd229cjMWYg.png" /></figure><p>Notice, how we have to write out print( ‘num1’ , num1) to see it in the output?</p><p>This is where Ice Cream comes in handy!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DgFRCvjhHG1obKjczuTBIg.png" /><figcaption>^^^ Icecream outputs automatically include the referenced variable.</figcaption></figure><p><strong>So why use icecream?</strong></p><ul><li>It allows us to see the variables that our output references, which saves confusion, especially when you have many outputs.</li><li>Its shorter than typing out ‘print’. Time is money!</li></ul><h3><strong>2)Pickling</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*42rfg8xJIUjF38NBqoIJoQ.png" /></figure><blockquote><strong>Pickling:</strong></blockquote><blockquote>“a serialized, byte-wise .pkl file that preserves a Python object precisely and exactly”</blockquote><p><strong>What is Pickling?</strong></p><p>- A serialization process for Python data structures</p><p>-Useful for when you need to store an object and retrieve it later.</p><p><strong>Well, why not just read in a file (JSON, CSV)? Is this not the same thing?</strong></p><ul><li>Yes and no.</li></ul><p>We have likely all experienced the incredible amounts of frustration (i.e. blinding rage) that occurs when saving/reading in a file. Maybe you saved the the dataframe to a .csv but the file was altered in memory somehow. Now you have the pleasure of starting from scratch to clean your data set, and between delimiters, dropping rows and removing headers etc., you have wasted valuable time. **Sigh**</p><p>Pickling is exactly what is sounds like; an easy way to PRESERVE and reuse any Python object.</p><p>Intrigued? Heres a quick tutorial:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DD94iSU4YEf_HYDOx1_fKg.png" /><figcaption>Step 1: Import Pickle</figcaption></figure><p>In this case, we are declaring a dictionary as our object to Pickle. This however could be anything: a dataframe, a matrix, a fitted model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dsVWosuO_xZSyzGSgfWitA.png" /><figcaption>Step 2: Opening the file for writing</figcaption></figure><p>Three things here:</p><ol><li>Note that in Step 2, the filename does <strong>not</strong> need to have an extension.</li><li>“<strong>WB</strong>” — simply stands for Write Binary (The file will be written in byte objects).</li><li>If you would like to specify where the file is being saved to, you can instead swap the ‘filename’ with a file path.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EcnoZMxly7FcIdENWZOJ3Q.png" /><figcaption>Step 3: Putting the jar on the Lid</figcaption></figure><p>Pickle.dump() takes two arguments:</p><ul><li>the name of the object to be pickled (in this case our dictionary)</li><li>the file where the object is being pickled (outfile)</li></ul><p>You will now see your pickled file ‘food.pkl’ saved in the same directory that you are already working in, unless you specified another filepath.</p><p>Now, how to <strong>unpickle</strong> a file:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vj4698D2bEOlS-thVowJug.png" /><figcaption>Step 4: Unpickling a File</figcaption></figure><p>To unpickle a file we simply do the opposite of what we did to pickle our dictionary in the first place.</p><ul><li>Outfile becomes infile,</li><li>‘WB’ → ‘RB’ (meaning read binary)</li><li>Then close the infile.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LNHN3hl9IMbUFnxJjLoFGw.png" /><figcaption>Our food dictionary unpickled.</figcaption></figure><p>Note: This only work for Python and is not compatible across other programming languages.</p><ul><li>There is version of pickling written in C — cPickling — making it up to a 1,000x faster. Very useful for large datasets.</li><li>Pickling can prove to be especially useful when working with machine learning algorithms, saving time and energy (both yours &amp; your computers) from rewriting a model and retraining it all over again.</li><li>While Pickling works for most objects it does not work for all. Lambda functions are one of these which can’t be serialized , however there is a workaround for using lambda functions by simply using the <strong>dill </strong>package instead of pickle.</li><li>Be aware! You are more likely to to import malicious code from a pickled file. As a rule of thumb, do not use pickling when the intended object to pickle contains sensitive data.</li></ul><p>_________________________________________________________</p><p>For more checkout:</p><p><a href="https://betterprogramming.pub/dont-fear-the-pickle-using-pickle-dump-and-pickle-load-5212f23dbbce">https://betterprogramming.pub/dont-fear-the-pickle-using-pickle-dump-and-pickle-load-5212f23dbbce</a></p><p><a href="https://www.datacamp.com/community/tutorials/pickle-python-tutorial">https://www.datacamp.com/community/tutorials/pickle-python-tutorial</a></p><p><a href="https://www.youtube.com/watch?v=CU1jOL6nv8I">https://www.youtube.com/watch?v=CU1jOL6nv8I</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14a3294b82fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>