<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Amsavalli Mylasalam on Medium]]></title>
        <description><![CDATA[Stories by Amsavalli Mylasalam on Medium]]></description>
        <link>https://medium.com/@amsavallimylasalam?source=rss-a3eaf02e1753------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*gOEN3DUWv5cYwjWB3q3X6g.png</url>
            <title>Stories by Amsavalli Mylasalam on Medium</title>
            <link>https://medium.com/@amsavallimylasalam?source=rss-a3eaf02e1753------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 10:57:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@amsavallimylasalam/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Interactive Data Visualisation with Plotnine using the grammar of graphics in Python]]></title>
            <link>https://medium.com/aatomz-research/interactive-data-visualisation-with-plotnine-using-the-grammar-of-graphics-in-python-86fe5a63218a?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/86fe5a63218a</guid>
            <category><![CDATA[ggplot]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[variablz]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Wed, 13 Sep 2023 18:48:41 GMT</pubDate>
            <atom:updated>2023-09-13T18:49:49.600Z</atom:updated>
            <content:encoded><![CDATA[<h3>ggplot using python; A beginner guide</h3><p>Unlock the world of data visualization with my beginner’s guide to ggplot in Python. If you’ve ever wanted to create stunning and informative graphs, charts, and plots effortlessly, this is where you begin your journey.</p><p>In this article, we’ll take you through the fundamental concepts of ggplot, a powerful data visualization library, and show you how to harness its capabilities to transform your data into compelling visual narratives.</p><p>Whether you’re a newcomer to Python or a seasoned programmer looking to dive into the world of data visualization, this guide will equip you with the knowledge and skills to create impactful visualizations with ease.</p><p>The basic structure of a ggplot consists of several components:</p><ol><li>Data: The data that you want to visualize.</li><li>Aesthetic mapping: This specifies how variables in the data should be mapped to visual properties of the plot, such as the x and y axes, colour, shape, and size.</li><li>Geometric objects: These are the visual elements that represent the data, such as points, lines, bars, and polygons.</li><li>Scales: These control how the data is represented on the axes, such as the range of values and the labelling of tick marks.</li><li>Facets: These allow you to create multiple plots based on subsets of the data, such as creating separate plots for different groups or categories.</li></ol><p>To create a ggplot, you start by specifying the data and aesthetic mapping using the ggplot() function, and then add layers of geometric objects, scales, and facets using various functions such as geom_point(), scale_x_continuous(), and facet_wrap(). The resulting plot can be customized in many ways using various arguments and options.</p><p>Overall, ggplot is a powerful tool for data visualization that allows users to create high-quality, customizable plots that can help to reveal insights and patterns in their data.</p><h3>Installation</h3><p>Plotnine can be installed using pip, a package manager for Python.</p><p>!pip install plotnine</p><h4>Importing Libraries:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*9ntUCl4bXaPHFC5Y7c1B2g.png" /></figure><h4>Importing Data:</h4><p>Let&#39;s start creating visualisations, For that I am importing <a href="https://www.kaggle.com/datasets/dnyaneshyeole/flipkart-smartphones-dataset/download?datasetVersionNumber=1">flipkart_smartphones</a> dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rhA5_9w89_rlE7ieuD5KhQ.png" /></figure><h4>Scatter plot to show the relationship between ratings and discounted price using ggplot</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jn5_p0Vq4CbcUfzkrgMCHg.png" /></figure><p>This code creates a scatter plot using ggplot function of plotnine package. The plot displays the relationship between original_price and discounted_price variables, coloured by brand.</p><p>Here’s a breakdown of the code:</p><ul><li>ggplot(df, aes(x=&#39;original_price&#39;, y=&#39;discounted_price&#39;, color=&#39;brand&#39;)): The ggplot function initializes a new plot object, which is then customized with subsequent layers. The aes function maps variables to visual aesthetics. Here, original_price is mapped to the x-axis, discounted_price is mapped to the y-axis, and brand is mapped to colour.</li><li>geom_point(): This layer adds points to the plot. Each point represents a device.</li><li>labs(x=&#39;Original Price&#39;, y=&#39;Discount price&#39;): This function sets the x-axis and y-axis labels.</li><li>theme_bw(): This function sets the theme of the plot to a simple black-and-white scheme.</li></ul><h4>Box plot to show distribution of discounted prices by brand:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YebjnK0Ff0EfTAS2VzmFgA.png" /></figure><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RZYCw-1OQqCfQd4uHYFjeA.png" /></figure><p>This code creates a boxplot visualization to show the distribution of discounted prices for the top 10 brands in the dataset.</p><p>the ggplot function is called to create the plot. The aes function is used to map the brand variable to the x-axis and the discounted_price variable to the y-axis, and the fill argument is set to brand to fill the boxes with colours corresponding to each brand.</p><p>The geom_boxplot function is called to create the boxplot. The ggtitle, xlab, and ylab functions are used to add a title and labels to the plot.</p><p>Finally, the theme function is used to adjust the x-axis text to be vertical and aligned to the right.</p><h4>Histogram to show the distribution of discounted prices:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ovvqC4oWGr8ZOJUP_GN5ag.png" /></figure><p>This code creates a histogram of the distribution of discounted prices in the dataframe df. The histogram bins are set to a width of 5000 using binwidth=5000. The histogram bars are filled with blue color and have white borders using color=&#39;white&#39;, fill=&#39;blue&#39;. The transparency of the bars is set to 0.5 using alpha=0.5.</p><p>The plot is given a title using ggtitle(&quot;Distribution of Discounted Prices&quot;) and the x and y axes are labeled using xlab(&quot;Discounted Price&quot;) and ylab(&quot;Count&quot;). The aes() function is used to specify that the histogram should be plotted using the discounted_price column.</p><h4>To compare the distribution of Original Prices and Discounted prices across different battery types. We can use the facet option to create separate plots for each battery type.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QnFR3lyKfn5E6aLGW8wfAg.png" /></figure><p>This code creates a scatter plot with discounted price on the x-axis and the original price on the y-axis, with the points coloured by brand. It also uses the facet_wrap function to create separate plots for each level of the battery_type variable.</p><ol><li>(ggplot(df1, aes(x=&#39;discounted_price&#39;, y=&#39;original_price&#39;, color=&#39;brand&#39;)) starts the plot, setting the x-axis to discounted_price, the y-axis to original_price, and the colour of the points based on brand.</li><li>geom_point() adds the points to the plot.</li><li>facet_wrap(&#39;~battery_type&#39;) creates separate plots for each level of battery_type.</li><li>labs(x=&#39;Discounted Price&#39;, y=&#39;Original Price&#39;) adds axis labels to the plot.</li><li>theme_bw() sets a black-and-white theme for the plot.</li></ol><h3>Conclusion:</h3><p>In this article, We have explored the capabilities of ggplot, a Python library for data visualization. We have seen how to create interactive visualizations using geometric shapes, aesthetics, and statistical transformations. We have also explored the facet option, which allows us to create subplots that display different subsets of the data.</p><p>By using the ggplot library, analysts can create highly customizable and interactive visualizations that can help them to better understand their data and communicate insights to others.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86fe5a63218a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/interactive-data-visualisation-with-plotnine-using-the-grammar-of-graphics-in-python-86fe5a63218a">Interactive Data Visualisation with Plotnine using the grammar of graphics in Python</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Beginner Guide for Identifying and Handling Outliers]]></title>
            <link>https://medium.com/aatomz-research/a-beginner-guide-for-identifying-and-handling-outliers-c761798fec4a?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/c761798fec4a</guid>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Tue, 03 Jan 2023 13:15:40 GMT</pubDate>
            <atom:updated>2023-01-03T13:15:40.593Z</atom:updated>
            <content:encoded><![CDATA[<h4>For Aspiring Data Scientists</h4><p>Have you ever noticed, while going through the rain, that one person walks casually without an umbrella and does not worry about anything while everyone is running away from the rain? Exactly that person is the outlier 😅. Fun apart, Let’s talk technically…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gjkbz-x1RExwHQk_aQ4IEg.gif" /><figcaption>Handling Outliers — Credits: Aatomz</figcaption></figure><p>Outliers are values within a dataset but unusual to other typical values for that attribute. Outliers may imply variabilities in a measure, observed errors, or newness.</p><h4>Should I Remove all the Outliers?</h4><p>According to the effect of outliers, we can decide whether we can keep it or drop it. If outliers negatively affect the result of an analysis, you can drop it, and else If outliers may be required information for data analysis, you can keep it.</p><p>Anomaly detection is used to find and identify outliers and helps to prevent fraud, opponent attacks, and network intrusions that can compromise your company’s future.</p><p>If you’re confident that the outlier is virtually skewed and incorrect, or if the data set is large enough that removing it won’t hurt the data, then it’s safe to be dropped.</p><p>I usually run the results with &amp; without the outlier to see if there’s any substantial difference and finalize the best outcomes rather than dropping it.</p><h4>What is the cause of outliers in the dataset?</h4><p>The most common causes of outliers in the datasets are Typo Errors, Measurement Errors, Data Processing errors, and Naturally-occurring Errors.</p><h4>Handling Outliers</h4><ol><li>Sort Values</li><li>Box Plot Visualization/Inter-Quartile Range (IQR)</li><li>How to Graph Your Data to Find Outliers</li><li>Z- Score Calculation</li></ol><h4><strong>1. Sort Values</strong></h4><p>In the Sorting method, You can sort numeric variables from low to high and observe for too-low or too-high values. Mark any drastic values that you find. Sorting values is easy to check whether you need to analyze specific data points before using other methods.</p><h4>2. Box Plot Visualization/Inter-Quartile Range (IQR)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/551/1*n1e53Ki1B5nvJL-EOGv7eg.png" /></figure><p>Box plot visually exhibits the statistical summary of a dataset, Min, Q1, Q2 (median), Q3, and Max. Even it indicates the outliers according to the IQR method. When an outlier is detected, the whisker will correspondingly change to the upper limit (Q3+1.5*IQR) or lower limit (Q1–1.5*IQR).</p><p>Quartile 1 (Q1) represents 25th percentile<br>Quartile 2 (Q2) represents 50th percentile<br>Quartile 3 (Q3) represents 75th percentile</p><h4>Implementation of IQR</h4><p>To explain the IQR method, I am taking <a href="https://www.kaggle.com/datasets/sulianova/cardiovascular-disease-dataset/download?datasetVersionNumber=1">the Cardiovascular Disease datase</a>t from kaggle. Let’s import the library and read the file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m3ZjpXKnArnpUFk4Z8Qwdg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xVOcNRd8G0RI3kpP1WjPIQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qq2KIah0smDjRCh4zvZLCQ.png" /></figure><p>Input:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*AY9upviUZLAmHd5FP_NfNg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nmlum5e5g_cnr-5Zdk2spg.png" /></figure><p>From this box plot, you can now analyze the existence of outliers; let’s create a function of the IQR method to detect and remove outliers for these features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gFJmIdcvcmcdq_TeaXqTyg.png" /></figure><p>Removing outliers first, then visualize these features using a boxplot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AkNw29V8foGjg0Ii-54Qew.png" /></figure><h3>3. How to Graph Your Data to Find Outliers</h3><p>Three graphical methods can be applied to find outliers. Outliers can be highlighted using scatterplots, histograms, and boxplots.</p><p>Using Graphical Methods, I am finding Outliers for the Height attribute.</p><p>Input:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d-Gmc9oYyrlzNSS2Law_TA.png" /></figure><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zGcxcseM1gBHOtKzGtcdKg.png" /></figure><p>Histograms make it simple to identify outliers. For instance, the extreme left point in the preceding graph is an outlier.</p><p>The box plot graph shows that values above 200 and below 130 depict outliers.</p><p>A point over 1.5 times the interquartile range above or below the third quartile is a convenient way to define an outlier.</p><h3>4. Z-Score Method to identify Outliers</h3><p>Z-score helps us to understand if a data value is greater or smaller than the mean and how outlying it is from the mean. Z score conveys how many standard deviations away a data point is from the mean.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*tQEO2UWPGxeD2snnLYRwvQ.jpeg" /></figure><p><strong>Z &gt; ± 3 OUTLIERS</strong></p><p>A data point is significantly different from the other data points if its z score is more than 3. Such a data point like that may be an outlier.</p><p>Implementing calculate Zscore function for height attribute</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S2RngqUdMAp9ttwb02zJbA.png" /></figure><p>This article shows commonly-used processes of identifying outliers, then demonstrates how these outliers may end up in a dataset to build valuable insights.</p><p>Handling outliers is a complex process. However, for analytical people, it’s so exciting 👩‍🏫.</p><p>Connect with me for data science talks</p><p><a href="https://www.linkedin.com/in/amsavalli-datascientist/">https://www.linkedin.com/in/amsavalli-datascientist/</a></p><p>Thanks &amp; Regards</p><p>AMSAVALLI — Data Scientist</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c761798fec4a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/a-beginner-guide-for-identifying-and-handling-outliers-c761798fec4a">A Beginner Guide for Identifying and Handling Outliers</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[These 3 books may change your way of visualizing the data!]]></title>
            <link>https://medium.com/aatomz-research/these-3-books-may-change-your-way-of-visualizing-the-data-a8eb934b26aa?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8eb934b26aa</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[books]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Thu, 10 Nov 2022 07:54:48 GMT</pubDate>
            <atom:updated>2022-11-10T07:54:48.798Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h9Bxj6L--ah4lKtenff5WQ.png" /><figcaption>3books may change your way of visualizing the data — credits: aatomz</figcaption></figure><p>Data visualization is a method to visualize statistical data to communicate the insights and patterns behind the data. However, plotting the various charts with statistical plots only makes sense sometimes.</p><p>Good data analysts/data scientists used to open the secret behind the data with their expertise. However, beginners also gain that expertise through reading well-written books from experienced Authors.</p><p>I used to upgrade my data visualization skills by reading various books, and Here are some of my recommendations to upgrade your skills in data visualization.</p><h4>1. How to Choose the Right Data Visualization by <strong>Mike Yi ✍️</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/263/1*6n2FAHaTpjvAnZKKrqDwDw.jpeg" /></figure><p>If you need to outline your data using visualization but aren’t sure what kind of visualization to use, this book will recommend chart types to try.</p><p>In this book, chart types have been divided into seven major categories based on visualization roles. such as,</p><ol><li>Display the data (Raw Numbers) in a Single value chart, Single value with indicator, Bullet chart, and Table.</li><li>Charts showing change over time are explained in Line Charts, Sparkling, Connected Scatter plots, Bar charts, and Box plots.</li><li>Charts for showing part-to-whole composition described by Pie charts, Doughnut charts, Waffle charts/Grid Plots, Stacked Bar charts, Stacked Area charts, and Steam graphs.</li><li>Charts depicting flows and processes explain Funnel Chart, Parallel Sets Chart, Sankey Diagram, and Gantt chart.</li><li>Data distributions are explained by Bar charts, histograms, Density curves, Box plots, Letter-Value plots, and Violin plots.</li><li>Comparing values between groups include Bar Chart, Grouped Bar Chart, Lollipop Chart, Dot Plot, Line Chart, Sparkling, RidgeLine, Box Plot, Letter-Value Plot, and Violin Plot.</li><li>Observing relationships between variables include Scatter Plots, Bubble charts, Connected Scatter plots, Dual-axis bar line Plots, Grouped Bar charts, Heatmap, 2-d density Curves, and Dendrogram.</li><li>Examining geographical data include Scatter maps, Bubble maps, 2-d histograms, Isopleth/contour maps, Connection maps, Choropleths, and cartograms.</li></ol><p>Download the book 📕 using this <a href="https://cdn2.hubspot.net/hubfs/392937/How-To-Choose-The-Right-Data-Visualization.pdf">link</a>.</p><h4>2. Storytelling with Data: A Data Visualization Guide for Business Professionals by Cole Nussbaumer Knaflic ✍️</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*3YhJVVMF3kj5LC4ZJtqcqA.jpeg" /></figure><p>If you are a newbie in visualization and want to generate good charts and ethics on fundamentals of storytelling in a business circumstance, this is an important book to start learning the primary principles. Specifically, you’ll learn about</p><ul><li>The importance of context</li><li>Choosing an effective visual</li><li>Clutter is your enemy!</li><li>Focus your audience’s attention</li><li>Think like a designer</li><li>Dissecting model visuals</li><li>Lessons in storytelling</li></ul><p>Through this book 📖 you can improve reasoning and analytical 🧐 skills for Data Visualization.</p><p>My favorite is chapter -6, which explains the dissecting model visuals. In this chapter, you will learn the importance of fascinating model visuals.</p><p>Overall this book is to best Visualise raw numbers.</p><p>Here is the link to purchase this book - <a href="https://www.amazon.com/gp/product/1119002257/ref=as_li_tl?ie=UTF8&amp;tag=jphwang06-20&amp;camp=1789&amp;creative=9325&amp;linkCode=as2&amp;creativeASIN=1119002257&amp;linkId=f6b671d22a6da52b421d4e3d49379996">Amazon S</a>ite</p><h4>3. Good Charts Workbook: Tips, Tools, and Exercises for Making Better Data Visualizations by Scott Berinato ✍️</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*OXvrtBpKQFm47itfKRbK0A.jpeg" /></figure><p>Good Charts Workbook gives ideas and exercises to help you practice DataViz. Each chapter has an overview of a topic, some examples, practice questions, and an “answer key” that shows his thought process but offers an open mind to alternatives.</p><p>The Good Charts Workbook provides tools, exercises, and practical insights that help you gain your skills.</p><p>How is the workbook organized? Two core sections make up the book.</p><p><strong>Part 1. Build Skills</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*FFMxMKl9DZInoREo_NzTGA.jpeg" /></figure><p>Build Skills part includes: Controlling Color, Crafting for Clarity, Choosing Chart Types, Practicing Persuasion, Capturing Concepts</p><p><strong>Part 2: Make Good Charts</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*DU4b88xlHexS9xU4qcIHOg.jpeg" /></figure><p>Make Good Charts include: Talk, Sketch, Prototype, The Monthly Report, and The Plastic Problem Presentation.</p><p>I would recommend this book 📕 for beginner to intermediate-level data visualization practitioners.</p><p>👉<a href="https://www.amazon.in/Good-Charts-Workbook-Exercises-Visualizations-ebook/dp/B078VFRDPC/ref=cm_cr_arp_d_pl_foot_top?ie=UTF8">Amazon</a></p><p>Still, there are many excellent books available for Data Visualization. Also, recommend your favorite data visualization book in the comment.</p><p>For more insights on data visualization and data science, follow me on LinkedIn.</p><p><a href="https://www.linkedin.com/in/amsavalli-datascientist/">https://www.linkedin.com/in/amsavalli-datascientist/</a></p><p>Thanks &amp; Regards</p><p>Amsavalli</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8eb934b26aa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/these-3-books-may-change-your-way-of-visualizing-the-data-a8eb934b26aa">These 3 books may change your way of visualizing the data!</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[These 7 categorical plots forced me to love Seaborn Library.]]></title>
            <link>https://medium.com/aatomz-research/these-7-categorical-plots-forced-me-to-love-seaborn-library-b4aff2bb2201?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4aff2bb2201</guid>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[seaborn]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Wed, 19 Oct 2022 14:22:21 GMT</pubDate>
            <atom:updated>2022-10-19T14:22:21.943Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zArfNOhLpujoUifJSagcOA.png" /><figcaption>Love with Seaborn: Credits (Aatomz)</figcaption></figure><p>Plotting categorical variables is always a challenging task for data scientists. However, there are dozens of libraries that do the magic of plotting. I am gonna explain the categorical plots which forced me to love Seaborn Library.</p><p>Click 👉🏻 <a href="https://www.kaggle.com/datasets/ishandutta/early-stage-diabetes-risk-prediction-dataset/download?datasetVersionNumber=1">here</a> to download the Early stage diabetes risk prediction dataset from Kaggle to reproduce this code.</p><h4>Import Libraries and Data</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*2XWt-WhVxKW99CVihAcJtQ.png" /></figure><p>Load data file into Pandas Data Frame.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GGQndiXC6wfQrqLD7KB21Q.png" /></figure><h4>1. Paired Bar Plot</h4><p>Paired Bar Plot is the vertical bar that separates two discrete, intimately related set of data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TiIc-HvQrmgOq3eQ4C79Iw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OVegmAzsd2wLq6VsRV9NxQ.png" /></figure><h4>2. Count Plot:</h4><p>Countplot () is used to express the counts of observations in each categorical bin using bars.</p><p>Let&#39;s create a count plot for Gender and partial paresis as a hue parameter. And plot horizontally to make more space for category labels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3FTYRBzYw-4qVZl00f-ApQ.png" /></figure><h4>3. Stripplot:</h4><p>A StripPlot is a scatterplot for separate categories. It’s suitable for multivariate analysis.</p><p>The issue of point overlapping in a strip plot can be partially fixed by setting the alpha parameter that regulates point transparency.</p><p>Let&#39;s create a strip plot for the Visual Blurring and Age feature, a vast strip plot for gender.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a9CMBGPoCu3f4Nq9_94WiQ.png" /></figure><h4>4. SwarmPlot:</h4><p>SwarmPlot is the categorical scatterplot with points adjusted to be non-overlapping.</p><p>In Swarmplot, we can show all observations and some representation of the underlying distribution.</p><p>Let&#39;s consider this example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5_viqB1cK9fL1xRtLXQ6mw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CZzFpU_SLHw_yMvy7EqfGw.png" /></figure><h4>5. Box Plot:</h4><p>A box plot or whisker plot is the primary statistical tool for visualizing data, and it&#39;s a distribution plot concerning categories.</p><p>Consider the below example for a box plot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JdXsQmOzVTX0zi9z8qIUaQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DhFeiNVJuSqS8sNvcMCW4Q.png" /></figure><h4>6. ViolinPlot:</h4><p>ViolinPlot is the categorical distribution plot, a combination of the ba ox plot and kernel density estimate.</p><p>The below example shows the distribution of Delayed Healing quantitative data across several levels of Age and Gender categorical variables such that those distributions can be compared.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wVF-NdKrXDDcaOTBt6YF2g.png" /></figure><h4>7. Displot:</h4><p>A display figures out the distribution of observations. We can use a displot display or distribution plot ta in histogram form.</p><p>This example shows how Age is distributed along with the gender category feature.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_4gbx5PBoaNGeksMykWRrw.png" /></figure><p>While comparing to the Matplotlib library, the Seaborn library allows us to analyze our data rapidly and comfortably in simple steps. That’s why I mostly end up with the Seaborn library while plotting Categorical variables.</p><p>What is your favorite for Categorical variable plots? Let me know in the comments.</p><p>For more insights from data visualization, follow me on LinkedIn.</p><p><a href="https://www.linkedin.com/in/amsavalli-datascientist/">https://www.linkedin.com/in/amsavalli-datascientist/</a></p><p>Thanks &amp; Regards</p><p>Amsavalli Mylasalam</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4aff2bb2201" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/these-7-categorical-plots-forced-me-to-love-seaborn-library-b4aff2bb2201">These 7 categorical plots forced me to love Seaborn Library.</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Become a Pro in Scatter Plot Visualization]]></title>
            <link>https://medium.com/aatomz-research/become-a-pro-in-scatter-plot-visualization-6aad24a6fb89?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/6aad24a6fb89</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[matplotlib]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Wed, 28 Sep 2022 16:12:12 GMT</pubDate>
            <atom:updated>2022-09-28T16:12:12.961Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Matplotlib Guide for Creating Scatter Plot</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ytmrGeHVL8W7cw7RWDFgxw.png" /><figcaption>Become a Pro in Scatter Plot Visualization (Credits: Aatomz)</figcaption></figure><p>The primary purpose of a scatter plot is to determine the relationship between two variables (bivariate). Scatterplots play a very important role in regression algorithms to find the relationship. Also, In classification problems, scatter plot help to identify the correlation between the features.</p><p>Here I am gonna explain how to create such a basic scatterplot, and also I am gonna explore how to customize it with labels, markers, and colors using matplotlib</p><p>You can download the diamond data set from this <a href="https://www.kaggle.com/datasets/shivam2503/diamonds">link</a> for reproducing this code.</p><h3>Creating Scatter Plot with Diamond dataset</h3><p>Import the necessary libraries</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*a0u9Qvvj2OGsEknM3DWYdg.png" /></figure><p>Load data file into Pandas Data Frame</p><pre>df=pd.read_csv(‘diamonds.csv’)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qg1vRsN9MINdLc4rzKND_w.png" /></figure><h3>Create a scatter plot.</h3><p>Let&#39;s see if carat and price are correlated using a scatter plot (sample for 620 rows) in Diamond Dataset.</p><p>With the help of a scatter plot, we can find Bivariate Analysis. ie, here, the relationship between the carat and the price of the diamond can visualize through a scatter plot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Mv49BoZYX7ft9aRt2ON80A.png" /><figcaption>Scatter Plot</figcaption></figure><p>Wow! we’ve created a scatter plot to find the relation between the carat and the price of the diamond. Yet! It feels something missing.</p><p>Yeah! without the axis label, it may be challenging to understand the relationship. So let’s see how to add those axis labels.</p><h3>Setting Label</h3><p>Here we are setting the X and Y label title using set_xlabel and set_ylabel methods, respectively. The fontdict parameter helps to customize the label formats.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*b3RXfZLW4kDxD480DjnlBg.png" /><figcaption>CODE: Setting Label in Scatter Plot</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mD5JTpqJTRD3O6_BVDnjVQ.png" /><figcaption>Scatter Plot Between Carat and Price of Diamonds</figcaption></figure><p>Now it looks cool; however, the first thing our client asks is what this plot is for. So if you don’t want to answer for it, just put the title for the plot. Indeed it’s a mandatory step 😅</p><h3><strong>Setting Title</strong></h3><p>using the set_title method, we can set the title for the plot. here also, we can customize the font of the title with the fontdict parameter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rwxoKoa0f_n4QMy0oSDLHA.png" /><figcaption>Code: Setting Title</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3PYts5nxgUXcv-tM_9v1oQ.png" /></figure><p>So we have checked the primary representation of the scatter plot so far. Let me explain some advanced options for scatter plots.</p><h3>Setting Marker: size, color, shape, and edge color</h3><p>We can also customize the marker in the scatter plot. Here I am gonna explain how we can resize and change the shape and color of the markers in the scatter plot.</p><p>Use the <strong>s</strong> parameter to change the size of the marker.</p><p>Use the <strong>c</strong> parameter to change the color of the marker.</p><p>Use the <strong>marker</strong> parameter to change the type of marker. For more marker style, check out this <a href="https://matplotlib.org/stable/api/markers_api.html">link</a>.</p><p>Use <strong>edgecolor</strong> parameter to change the color of the edge of the marker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*29ErrdDEht9ZgCXpZgJSOA.png" /><figcaption>Code: Marker Type, Size , and Colour</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iO1nKLoF770BNPX3rwomHA.png" /></figure><h3><strong>Adding Annotations</strong></h3><p>To add annotations in the scatter plot, we have to do the following steps.</p><p>⚽️ Store all the annotations in a list in order with the sequence of the points to be displayed.<br> ⚽️Draw the scatter plot.<br> ⚽️ Using a for loop, annotate each point.</p><p>consider the following example: For this, I filtered the premium diamonds from a diamonds data frame</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nX7Bw9fpkGncv-jpxs7Pnw.png" /></figure><p>step1: In the premium diamond dataset, I chose the clarity feature for annotations and stored this as a list.<br>step2: Drawing scatter plot: In this xaxis=carat, yaxis=price.<br>step3: Created for loop to annotate clarity feature.</p><p><strong>Input:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zva6g9dMKHIhT-RUag4GgQ.png" /></figure><p><strong>Output:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tHj7Jtn_b7VAVd-YSep1kw.png" /></figure><p>So, I hope next time you plot your scatter plot, you will search for my article. If you like this, don’t forget to clap.</p><p>If you want to learn more options, let me know in the comments, and I will update how to do those features in this article.</p><p>Follow me on LinkedIn for more insights on Data Visualization</p><p><a href="https://www.linkedin.com/in/amsavalli-datascientist/">https://www.linkedin.com/in/amsavalli-datascientist/</a></p><p>Thanks &amp; Regards</p><p>Amsavalli</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6aad24a6fb89" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/become-a-pro-in-scatter-plot-visualization-6aad24a6fb89">Become a Pro in Scatter Plot Visualization</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Let Lambda do Feature Engineering]]></title>
            <link>https://medium.com/aatomz-research/let-lambda-do-feature-engineering-ac9e52fb6a5e?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac9e52fb6a5e</guid>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Sat, 17 Sep 2022 19:29:58 GMT</pubDate>
            <atom:updated>2022-09-17T19:29:58.063Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oQQDDSpegpx9he_XcjX_Rw.jpeg" /></figure><p>We all know a lambda function can have any number of arguments but can have only one expression. It is one of the famous one-liner functions in Python programming.</p><p>By using this lambda function with Pandas, we can do many feature engineering operations. In this article, I am gonna explain some of the use cases.</p><p>I am using the students&#39; performance dataset from Kaggle. For reproducibility, you can download the link <a href="https://www.kaggle.com/code/ahmedameenhassan/students-performence/data">here</a>.</p><h4>Let’s import the libraries and dataset</h4><pre>import pandas as pd<br>import numpy as np</pre><pre>df=pd.read_csv(‘students_performance.csv’)<br>df</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5h3OJUXayxzhbB0mRAMs3Q.png" /></figure><h4>1. Discretization</h4><p>Here I am gonna discretize the writing score values into categories (Outstanding and Satisfactory) based on their score. So I’ve applied lambda with a unary conditional statement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kp2MWV57SREhRoXKzJZ41A.png" /></figure><h4><strong>2. Data Aggregation</strong></h4><p>One of the best ways to aggregate and create new features is using the lambda function.</p><p>Here we can calculate the total score from multiple features such as math score, reading score, and writing score.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*owWOl30scG7oC_9a2O-1Pg.png" /></figure><h4>3. Filter Rows</h4><p>Sometimes, we need to apply lambda expression to filter the dataset. Suppose we need to filter the rows based on the percentage value. (E.g. percentage &gt; 80%) . Here the score value is not available in percentage, and we could not filter using the direct method. So applying lambda expression to filter the data is the best way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YdfUA-ZOATdQtllxiMde0A.png" /></figure><h4><strong>4. Binary Encoding:</strong></h4><p>Let’s apply the lambda with the combination of the map to encode the lunch feature into binary values (standard — 1, free/reduce — 0)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s7BNsyy62AJ_ReVC0KI4NA.png" /></figure><h4>5. Categorical Encoding</h4><p>Let’s convert the categorical feature ‘race/ethnicity’ into numeric values through the lambda function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HP9i1iKfyE_3jeZd3xcDGw.png" /></figure><h3><strong>6. Normalization:</strong></h3><p>Normalization refers to replot real-valued numeric attributes into a 0 to 1 range.</p><ul><li>Zscore</li><li>Min-Max</li></ul><h3><strong>Zscore</strong></h3><p>The z-score calculates how many standard deviations a data point is away from the mean.</p><p>Here I am finding the Zscore value using lambda for math_score</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oGm-_0MIWsY2Td1Hhkqh4g.png" /></figure><h3><strong>Min-Max Normalisation</strong></h3><p>In Min-Max Normalization, the minimum value of that feature gets transformed into a 0, the maximum value gets transformed into a 1, and every other value gets transformed into a decimal between 0 and 1.</p><p>Here I am applying this function with Lambda to the reading_score feature.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xKlcAm0I0h_MpKKAmPBPRA.png" /></figure><h3>7. IQR</h3><p>Using the Interquartile Rule, we can find the outliers.</p><p>Once we find the values of the first and third quartiles, the interquartile range is very easy to calculate.</p><p>Here I am using Lambda to find outliers for writing scores.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rVGf0EgOYdR_A3iKl-LTiQ.png" /></figure><p>So this is how Lambda is a powerful function to apply with Pandas to reduce most of the Feature engineering operations to one line. However, be cautious while using lambda for a sequence of operations since It may lead to reading complexity.</p><p>For more data science thoughts, follow me on LinkedIn.</p><p><a href="https://www.linkedin.com/in/amsavalli-datascientist/">https://www.linkedin.com/in/amsavalli-datascientist/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac9e52fb6a5e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/let-lambda-do-feature-engineering-ac9e52fb6a5e">Let Lambda do Feature Engineering</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Stunning Python Libraries you should know to work on Data Science projects.]]></title>
            <link>https://medium.com/aatomz-research/stunning-python-libraries-you-should-know-to-work-on-data-science-projects-86a5c04b76a3?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/86a5c04b76a3</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Tue, 16 Aug 2022 17:18:54 GMT</pubDate>
            <atom:updated>2022-08-16T17:18:54.097Z</atom:updated>
            <content:encoded><![CDATA[<p>As a data scientist, data analysis and manipulation is a day-to-day job. Knowing the right library for the right tasks tremendously reduces the working hours. So, I thought to share the 3 primary libraries I used to work for my regular tasks for various purposes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_tpdHSEW6VuFewYJUvoA8Q.jpeg" /><figcaption>Credits to Aatomz</figcaption></figure><h3>1. Pandas Library</h3><p>pandas were released on 11 January 2008</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/228/1*L_FgditNOU2jIOcfzn8q3g.png" /></figure><p>pandas library 📚 is one of the most used and widespread libraries 📚 in Python. Pandas library is most required for data manipulation necessary for data analysis 🧐 or machine learning.</p><p>Pandas Library helps us to work on structured data optimally for data structures and functions. The name panda does not imply the animal 🐼 ; it expresses Panel Data which means a structured dataset. Data frame and series are the two main classes to work on pandas. Python with Pandas is used in a wide range of different fields.</p><p><strong>Why should you use Pandas Library?</strong></p><ul><li>The Pandas library provides a systematic method to manage and explore data.</li><li>Alignment and indexing are one of the best methods in pandas library.</li><li>Pandas library provides tools for loading data into in-memory data<br>objects from different file formats.</li><li>The different file format we can import into pandas are Comma-separated values (CSV), XLSX, ZIP, plain Text(txt), JSON, XML, HTML, Images, Hierarchical Data Format, PDF, DOCX, MP3, MP4, SQL</li><li>Handling of missing data is integrated within pandas libraries.</li><li>Handling of missing data is integrated within pandas libraries.</li><li>Using pandas features, we can easily clean 🧼 up our data.</li><li>Reshaping and pivoting of data sets.</li><li>Label-based slicing, indexing, and subsetting of large data sets.</li><li>Filter, Sort, and Transpose</li><li>Function Application like Lambda, Aggregate, Group by, Map, Transform<br> And pipe.</li><li>Pandas can help to combine, concatenate, join and merge data.</li><li>Pandas play an essential role in Descriptive Statistics and Random sampling.</li></ul><p><strong>The downside of the Pandas library</strong></p><ul><li>Poor closeness for 3-Dimensional Matrices. We cannot efficiently process the image data using the Pandas library.</li><li>Pandas have a very steep learning curve. There are too many functionalities available in Pandas, and it will be a time-consuming process to learn.</li><li>To process big datasets is limited due to out-of-memory errors in pandas.</li><li>Slow, limited multicore algorithms for large datasets</li></ul><h3><strong>2. Dask Library</strong></h3><p>Dask library was released on 8 January 2015</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*FmXQxoesC8MeFYop5mJkdQ.jpeg" /></figure><p>To deal with extensive data sets and parallel computing, the best one ☝️ is with Dask Library.</p><p>For parallel computing, Dask is the extendable open source python 🐍 library.</p><p><strong>Why should you use the Dask Library?</strong></p><ul><li>Dask is familiar due to parallelizes NumPy and pandas data-frame.</li><li>Dask Runs hardy on clusters with 1000s of cores.</li><li>Dask is suitable for fast numerical algorithms.</li><li>With python concurrent futures, Dask supports a real-time task framework.</li><li>The higher-level Dask API is Dask Array or Dask Delayed, Dask ML, Dask Bags, and Dask DataFrame.</li></ul><p><strong>The downside of Dask</strong></p><ul><li>Dask is not good at optimizing complex SQL queries.</li><li>Index, Sort, and shuffle Operations are not good at Dask parallel computing.</li></ul><h3>3. Polars Library</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/1*x40mC-XiWMDYiby6SnKKBw.png" /><figcaption>Credits to: www.pola.rs</figcaption></figure><p>The Polars project was started in March 2020 by Ritchie Vink.</p><p>Polars is a DataFrame library in the rust programming language and uses Apache Arrow as a foundation.</p><p>Polars is the wrap speed data frame library for python and rust.</p><p>Polars does not use an index for the data frame.; it utilizes an apache arrow why because the apache arrow is efficient in areas like load time, memory usage, and computation.</p><p><strong>Why should you use Polars Library?</strong></p><ul><li>Polars library provides a fast and easy way to work with a large dataset.</li><li>Polars is a data manipulation and analysis library written in rust with APIs in Python.</li><li>polars library gives full Support for numerical calculations.</li><li>String manipulation and data frame operations like filtering, joining, intersection, and aggregations such as groupby can be made accessible using the polars library.</li><li>Parallelization, optimizing CPU, Arrow2 framework makes polar so fast.</li><li>when building data pipelines, polars is the best tool.</li></ul><p><strong>The downside of polars</strong></p><ul><li>polars is not much efficient in compatibility.</li></ul><p>I have written about 3 primary libraries I used to work on a day-to-day job, but it doesn’t mean I won’t use other libraries. As data scientists, we should always be up to date with the tech. Even there are dozens of libraries available on the market now. I have tried many, but these 3 are most attracted me to my data science job. What is your favorite library on data analysis tasks?</p><p>Thanks &amp; Regards</p><p>Amsavalli</p><p>Connect with me on LinkedIn for more data science insights!</p><p><a href="https://www.linkedin.com/in/amsavalli-datascientist/">https://www.linkedin.com/in/amsavalli-datascientist/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86a5c04b76a3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/stunning-python-libraries-you-should-know-to-work-on-data-science-projects-86a5c04b76a3">Stunning Python Libraries you should know to work on Data Science projects.</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hard to imagine  data analysis without the aggregate functions.]]></title>
            <link>https://medium.com/aatomz-research/hard-to-imagine-data-analysis-without-the-aggregate-functions-27ba65b26068?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/27ba65b26068</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-analyst]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[pandas]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Tue, 02 Aug 2022 01:38:34 GMT</pubDate>
            <atom:updated>2022-08-02T01:38:34.618Z</atom:updated>
            <content:encoded><![CDATA[<p>Aggregation is one of the most useful descriptive analysis functions to perform complex statistical steps more efficiently. The primary purpose of the aggregate is to summarize the group of data to analyze the statistical summaries such as mean, standard deviation, variance, etc.</p><p>I quickly demonstrate various methods of using an aggregate function in Pandas in this article. For reproducing the code, I’ve attached the data set <a href="https://www.kaggle.com/datasets/iamsouravbanerjee/software-professional-salaries-2022/download?datasetVersionNumber=14">link</a> here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VltI76UcWH3IEJ7DLjtQGg.jpeg" /></figure><p>Let’s load the data set using Pandas</p><pre># Import Pandas Library<br>import pandas as pd</pre><pre>df=pd.read_csv(‘IT_salary.csv’)<br>df.head(10)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JacniuHAKhTKAI4bmMlcJw.png" /></figure><h3><strong>1) Use the aggregate function to find the statistical summary of one variable</strong></h3><p>Here we are finding the average salary using Pandas aggregate function</p><pre>value = df[‘Salary’].aggregate(‘mean’)<br>print(round(value,2))</pre><p>Output:</p><pre>695387.21</pre><h3>2) To find the statistical summary of one/many variables by grouping one variable</h3><p>Here we are finding the statistical summary of each variable by grouping the Job Title Variable. Remember, the statistical summary only works on the numerical columns, and hence other columns are ignored.</p><p>2.1. Let’s find out the mean value for each category in JOB Title</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OYlKVh4SyI34wZo9Kqb0xg.png" /></figure><p>2.2. Let’s find out the maximum values of each variable for each group of JOB title</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*stwOMctTcsNFNuh2bhnb2Q.png" /></figure><h3>3) Reporting various statistical summaries</h3><p>Let&#39;s find both minimum and maximum salary by grouping job-title.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_qs56slhUZmp3Z2xbKeD3Q.png" /></figure><h3>4) Reporting various statistical summaries from various features</h3><p>Here I am applying multiple aggregate functions(count, min, max) at the same time on various features (Company name and Salary) by grouping one variable (Job Title)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/986/1*tgBZUXZ6JTXxKnuX-rQ6aQ.png" /></figure><h3>5) Group By Multiple Features</h3><p>We can group multiple features using Pandas</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GzThFmGn4h5ZE5_Z5LWDvQ.png" /></figure><h3>6) Aggregate function with Group by Multiple Features</h3><p>Also, we can group multiple variables to find the statistical summary of one/all variables. Here I am grouping the mean and sum salary based on JOB title and company</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8zO4WljHktC1RVzcGlsQ4Q.png" /></figure><h3>7) Use dictionary function to report various statistical summaries from various features</h3><p>Let’s find out the maximum rating and sum of salary by grouping the Company Name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*9IOqaNcNozeHr6WmJOAJhw.png" /></figure><p>So finally, in this article, I have described various pandas aggregation tricks to summarize the descriptive statistics of given data frame. Without the aggregate functions, it is really hard to analyze the summary of group of data. Also, there are many more options you can play around with aggregate function, However, for efficient programming, have a good practice on various dataset.</p><p>Happy Learning!</p><p>To learn more from me, follow me on LinkedIn</p><p><a href="https://www.linkedin.com/in/amsavalli-datascientist/">Amsavalli Mylasalam - Data Science Technical Assistant - aatomz research | LinkedIn</a></p><p>Regards</p><p>Amsavalli</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27ba65b26068" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/hard-to-imagine-data-analysis-without-the-aggregate-functions-27ba65b26068">Hard to imagine 🥹 data analysis 📊without the aggregate functions.</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[You’ll never forget Shallow Copy and Deep Copy in Python after reading this…]]></title>
            <link>https://medium.com/aatomz-research/youll-never-forget-shallow-copy-and-deep-copy-in-python-after-reading-this-b2bef9aa57a?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/b2bef9aa57a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[numpy]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[amsa]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Thu, 14 Jul 2022 08:15:20 GMT</pubDate>
            <atom:updated>2022-07-14T08:15:20.782Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Iz-hdsTA4oOeXDuRU2psw.jpeg" /><figcaption>Deep Copy Vs Shallow Copy — Image Credits: <a href="http://www.aatomz.com/">aatomz research</a></figcaption></figure><p>Indeed, one of the most common interview questions I’ve been asked in the Python interviews is the difference between Shallow copy and Deep Copy.</p><p>Copying the compound objects, such as those containing lists, class instances or nested lists in Python/NumPy, is too dangerous.</p><p>To understand why it is too dangerous, we should know the mechanism behind various copy operations. So In this article, I am going to explain it.</p><p>Various methods to copy an object in Python</p><ol><li>Using Reference</li><li>Using Shallow copy</li><li>Using Deep Copy</li></ol><h3><strong>1. Using Reference (Both Pain and Benefits)</strong></h3><p>We can use the assignment operator (=) to get a copy of the existing object. It won’t create a new object, but it will create a reference to the existing object. It means that whatever the changes we are doing with the original object will affect the reference object or vice versa.</p><p>For example, If you beat any cat, both the original and reference cat feels the pain. If you give food to any cat, both original and reference cat will have food. i.e., both pain and benefits for both cats</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*peiAunIuNV6BMVn2X4uOJA.jpeg" /><figcaption>Reference: Image Credits to <a href="http://www.aatomz.com/">aatomz research</a></figcaption></figure><p>Input:</p><pre>import pandas as pd<br>import numpy as np</pre><pre>original_array=np.array([5,6,7,8,9])<br>ref_array=original_array</pre><pre>print(f&#39;Original: {original_array}&#39;)<br>print(f&#39;Reference: {ref_array}&#39;)</pre><p>Output:</p><pre>Original: [5 6 7 8 9]<br>Reference: [5 6 7 8 9]</pre><p>Now we are changing the particular element in the original array using the assignment operator and notice that the value has changed in both the original and reference object.</p><pre>original_array[2]=4</pre><pre>print(f’Original:{original_array}’)<br>print(f’ref_array:{ref_array}’)</pre><p>Output:</p><pre>Original:[5 6 4 8 9]<br>ref_array:[5 6 4 8 9]</pre><h3><strong>2. Using Shallow Copy (Only Pain, No Benefits)</strong></h3><p>We can add new items into the shallow copied object, but we cannot modify the existing items.</p><p>For example, We can give food to the copied cat, but it doesn’t mean we are giving food to the original cat. However, if we beat mocked cat with the hammer, the original cat also feels the pain 🤣🤣🤣 i.e., only pain, no benefits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ImLTwzT_q7ZQQobz-fCUbg.jpeg" /><figcaption>Shallow Copy — Image Credits to <a href="http://www.aatomz.com/">aatomz research</a></figcaption></figure><p>The copy method copies the reference of the compound objects, i.e., it is the same as the reference method but only for nested items. Therefore, this method is called shallow copy or partial copy.</p><p>So, appending a new item to the copied compound objects won’t affect the original object, but modifying the copied nested items will affect the original object.</p><p>Input:</p><pre>original=np.array([1,3,5,7])</pre><pre>new=copy.copy(original)</pre><pre>print(f&#39;Original :{original}&#39;)<br>print(f&#39;New Value :{new}&#39;)</pre><p>Output:</p><p>This means it will create new and independent object with same content.<br> To verify this, we print the original and new.</p><pre>Original :[1 3 5 7]<br>New Value :[1 3 5 7]</pre><p>In this below example, We are appending an item to the original object, but this change will not affect the copied item and vice versa.</p><pre>original=np.append(original,[9])</pre><pre>print(f&#39;After appending 9 the Orginal if: {original}&#39;)<br>print(f&#39;New Value is : {new}&#39;)</pre><p>Output:</p><pre>After appending 9 the Orginal is: [1 3 5 7 9]<br>New Value is : [1 3 5 7]<br></pre><p>Now we modify the existing element in the original object and notice that it will also reflect in the copied item and vice versa.</p><p>Input:</p><pre>import copy</pre><pre>old = [[1,3,5],[2,4,6]]<br>new = copy.copy(old)</pre><pre>old[1][1] = 8</pre><pre>print(“Old list:”, old)<br>print(“New list:”, new)</pre><p>Output:</p><pre>Old list: [[1, 3, 5], [2, 8, 6]]<br>New list: [[1, 3, 5], [2, 8, 6]</pre><h3><strong>3. Using Deep Copy (No Pain, No Benefits)</strong></h3><p>Python/NumPy has a deep copy function to avoid this pitfall while copying the object.</p><p>After the deep copy, whatever the modification changed in copied object won’t affect the original object or vice versa, i.e., no pain and no benefits for the original/copied cat 🤷🏻‍♀️</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*csgvfaXBguShGfdfl4HL2A.jpeg" /><figcaption>deep copy: Image credits to <a href="http://www.aatomz.com/">aatomz research</a></figcaption></figure><p>The deep copy function copies each item recursively from the original array into the copied array. Therefore the nested objects won’t be affected because of the modification done to the original object or vice versa.</p><p>Let’s check the below example. First, we modify the element from the original object and notice that the deep copied item won’t get affected.</p><p>Input:</p><pre>import copy</pre><pre>old = [[1,3,5],[2,4,6]]<br>new = copy.deepcopy(old)</pre><pre>old[1][2] = 8</pre><pre>print(“Old list:”, old)<br>print(“New list:”, new)</pre><p>Output:</p><pre>Old list: [[1, 3, 5], [2, 4, 8]]<br>New list: [[1, 3, 5], [2, 4, 6]]</pre><p>I hope you enjoyed the concept of shallow and deep copy methods in Python/NumPy.</p><p>Don’t forget to clap if you like it; I also welcome your suggestions.</p><p>Follow me on <a href="https://www.linkedin.com/in/amsavalli-datascientist/">LinkedIn</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b2bef9aa57a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/youll-never-forget-shallow-copy-and-deep-copy-in-python-after-reading-this-b2bef9aa57a">You’ll never forget Shallow Copy and Deep Copy in Python after reading this…</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pandas filtering methods to solve most of the data analysis tasks]]></title>
            <link>https://medium.com/aatomz-research/pandas-filtering-methods-48f778005f1c?source=rss-a3eaf02e1753------2</link>
            <guid isPermaLink="false">https://medium.com/p/48f778005f1c</guid>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Amsavalli Mylasalam]]></dc:creator>
            <pubDate>Mon, 20 Jun 2022 08:03:31 GMT</pubDate>
            <atom:updated>2022-07-11T07:26:12.298Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uQ3ouK14w7N5UMC6tKankg.jpeg" /></figure><h3><strong>Introduction:</strong></h3><p>Filtering is one of the key 🔑 operations of filtering datas from raw data set for the data analyst.In this article I am explaining different filtering methods available in pandas.</p><p>Here I’ve used most famous Titanic dataset for filtering operations, so that you can easily reproduce the code. Please download the data file from <a href="https://github.com/mwaskom/seaborn-data/blob/master/titanic.csv">here</a>.</p><pre>import pandas as pd</pre><pre>df=pd.read_csv(‘titanic.csv’)<br>df.head()</pre><p>Output</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wxid5LXdffyYNKvB3RPnfA.png" /></figure><h4>Filtering Rows using Conditions:</h4><h3>1) Location (loc) function</h3><p>Here I am using pandas location function to filter only the 3rd class passengers.</p><p>Here I am using pandas location function to filter only the 3rd class passengers.</p><p>Input:</p><pre><br>dfnew=df.loc[df[‘class’]==’Third’]<br>dfnew</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nbfbGDZ9CTwMzprEFIzfKA.png" /></figure><h3><strong>2) Query Function</strong></h3><p>The query() function is used to filter the columns of a DataFrame with a boolean expression.</p><p>Listing out only male passenger using Query Function</p><p>Input:</p><pre>newdf = df.query(“sex== ‘male’”)<br>newdf</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ookb8nFSTvNDHrRVkJwcgg.png" /></figure><h3>3) Directly Filter From DataFrame</h3><p>Filter Female alive passengers using Data frame way instead of loc method.</p><p>Input:</p><pre>dfalivefemale = df[(df.alive == “yes”) &amp; (df.sex == “female”)]<br>dfalivefemale</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A4s0eM1RAX3UFR6JMF1Vvw.png" /></figure><h4><strong>4) DataFrame Filter by rows position and columns name</strong></h4><p>Let’s filter first 10 rows and only Class and Who columns</p><p>Input:</p><pre>df.loc[df.index[0:10],[“pclass”,”who”]]</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sIT6mAm77XO5YwAT0e1qXA.png" /></figure><h4>5) Use Lambda function to Filter Rows based on Conditions</h4><p>Here I’m filtering the Rows where the Fare greater than 7.0000</p><p>Input:</p><pre>df.loc[lambda df: df[‘fare’] &gt; 7.0000]</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JSpU9tWlsSRH7kYlnxTSyQ.png" /></figure><h4>6) Filter One Column values based on Condition in another Column</h4><p>Listing out column ‘alive’ value not equal to ‘Yes’ whose town is “Southampton”</p><p>Input:</p><pre>df.loc[(df.alive != “yes”) &amp; (df.embark_town == “Southampton”)]</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jlfczc2zxQn85GV4eqXxXw.png" /></figure><p><strong>7) The tilde operator negates the values in the DataFrame:</strong></p><p>Filter columns of ‘Who’ is ‘child’ using tilde operator</p><p>Input:</p><pre>childpassanger= df[~df[‘who’].str.contains(‘man’)]<br>childpassanger</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U4aRq8IffxYPHnJcBJNKkQ.png" /></figure><p><strong>8) Filter Non-Missing Data in Pandas Data frame</strong></p><p>Input:</p><pre>nonmissingdata= df[df.deck.notnull()]<br>nonmissingdata</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zZjfqxyMkPfB7-PkQcAolw.png" /></figure><h4>9) List Comprehension Method for Filtering</h4><p>Let’s filter the rows of male who did not survive</p><p>Input:</p><pre>df.iloc[[index for index,row in df.iterrows() if row[‘sex’] == ‘male’ and row[‘survived’] == 0]]</pre><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IrwDoN8mE3neqbTRdAoOPA.png" /></figure><p><strong>Conclusion:</strong></p><p>I believe this article gave you lot of insights to you yet still there are lot I’ve missed such us filtering dates and time. I will cover those topics in upcoming articles.</p><p>If you enjoyed the article, Like and share it.</p><p>Follow me on <a href="https://www.linkedin.com/in/amsavalli-datascientist/">LinkedIn</a> for more insights.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=48f778005f1c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/pandas-filtering-methods-48f778005f1c">Pandas filtering methods to solve most of the data analysis tasks</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>