<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arunkumar N on Medium]]></title>
        <description><![CDATA[Stories by Arunkumar N on Medium]]></description>
        <link>https://medium.com/@kumarnarun6?source=rss-969ea81e348f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*LwNr6g_QyPJ5345he0Wfvg.jpeg</url>
            <title>Stories by Arunkumar N on Medium</title>
            <link>https://medium.com/@kumarnarun6?source=rss-969ea81e348f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:21:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kumarnarun6/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Revealing the Hidden Insights of NFL Big Data Bowl 2023 through Univariate Data Visualization]]></title>
            <link>https://medium.com/aatomz-research/revealing-the-hidden-insights-of-nfl-big-data-bowl-2023-through-univariate-data-visualization-25f0e44f105f?source=rss-969ea81e348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/25f0e44f105f</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[aatomz]]></category>
            <dc:creator><![CDATA[Arunkumar N]]></dc:creator>
            <pubDate>Sat, 31 Dec 2022 05:52:56 GMT</pubDate>
            <atom:updated>2022-12-31T05:52:56.424Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PZxxxYrnDcPntoPeMQDYQQ.jpeg" /><figcaption>credits: <a href="https://edition.cnn.com/2022/09/08/sport/nfl-glossary-terms-spt-intl/index.html">edition</a></figcaption></figure><p>Let’s dive into some sports. It’s American football this time. This article will explore the power of univariate analysis in understanding and analyzing data. Univariate analysis is a statistical technique that analyzes the distribution of a single variable. By applying univariate analysis to the NFL Big Data Bowl 2023 dataset, we can gain valuable insights into the characteristics and patterns of the data.</p><p>This article will go through the techniques and approaches used in univariate analysis and how they can be applied to the NFL Big Data Bowl 2023 dataset to reveal hidden insights and trends. We hope this article will provide a solid understanding of the importance and applications of univariate analysis in data science.</p><h3>Importing the required libraries</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qZA6A6MTYheBeEvmFG39Ng.png" /></figure><p>Here, I’ve used “pandas.set_option” to scroll through all the columns in the dataset.</p><p>I have taken the recently trending Kaggle NFL Big Data Bowl 2023 dataset. You can download it from <a href="https://www.kaggle.com/competitions/nfl-big-data-bowl-2023/data">here</a> for reproducibility. I have already optimized and merged the dataset. You can refer to my Kaggle <a href="https://www.kaggle.com/code/arunkumardsci/merging-and-optimizing-the-nfl-data">notebook</a> if you like to reproduce this article.</p><h3>Reading the optimized file.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/990/1*kGq56FJQAgK3BHNSUcXgdw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*OSDN0dEOHWoT0qrN97Bblw.png" /></figure><p>We have approximately 8 million rows and 70 columns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ybfHtp0lzYqn1lyZdkH7Yw.png" /></figure><p>From the info(), we can see that I have converted all float64 and int64 datatypes to float32 and int32, respectively.</p><h3>Customizing the Plot Fonts — Font dictionary</h3><p>For customizing the size and color of axes labels, we use the fontdict option in Matplotlib. We can put this directly into code or store it in a variable to change it as needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*BoVPFcXN1c9FvxNPY_DHHQ.png" /></figure><h3>Box Plot</h3><p>Through research and experimentation, these columns contain a reasonable number of outliers. I have chosen numerical columns to see how the values are distributed. I preferred to show all the plots in a single view by using the subplots option to reduce scrolling.</p><p>Since the data is bigger and we are also using loops, it is safer to use gc.collect(), a garbage collector that removes unwanted memory.</p><p>The option &quot;plt.tight_layout()&quot; automatically adjusts the spacing between the subplots, so there is no alignment problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SfilrmhuGPefy-zW834eQA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TKUlgwsuKZiIGWjdUQG_Xw.png" /></figure><p>From the above box plots, we can able to see,</p><p>Both the home team and the visiting team score mostly between 4 and 18 points. The maximum score of both the home and visiting teams is around 38. We rarely see teams&#39; scores above 40, which are outliers.</p><p>The number of yards to go for one down is in the range of 5–10. If this value is low, the offensive team dominates and pushes forward. If this value is high, the defensive team has not let the offensive push forward for the goal.</p><p>We can see many outliers for the distance traveled from the initial time point. After 0.8 yards, all are outliers, and most players traveled between 0 and 0.5 yards.</p><h3>KDE Plot</h3><p>A kernel density plot is a graphical representation of the distribution of a continuous variable. It is a smoothed version of a histogram, which estimates the underlying probability density function of the data.</p><p>It gives us insights, which I will explain in the histogram part. Now let’s enjoy the plot. Some plots look like the mask of Batman 😅😅😅. When compared to other plots, this one took more time to process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XDr0kL4o6hEs0tW89JvBDg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jb4WslL8BWzYql7raaalog.png" /></figure><h3>Histogram</h3><p>A histogram is a graphical representation of a dataset that shows the frequency or proportion of the data falling within different intervals or bins. It is a valuable tool for visualizing the distribution of a dataset and identifying patterns or trends in the data.</p><p>In the histogram, we can use the &quot;bins&quot; option to group the values into several bars. It makes our plots look crisp and clean.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R68BzDi_YqTK4KuqDLlgjQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bUc2BvbWh0OGzIwim2rUeQ.png" /></figure><p>From the Histogram, we can see that,</p><p>Players are mostly in the middle of the ground as the count of ‘x’ and ‘y’ maximum for the middle values</p><p>Most players are in the weight categories 180–220 pounds and 320 pounds</p><p>The orientation of the players and angle of player motion are mostly 90 degrees or 270 degrees, as the players of both teams are facing each other 180 degrees apart.</p><h3>Count Plot</h3><p>A count plot is a graph that displays each unique category&#39;s frequency of occurrences in a categorical dataset. It is often used to visualize the distribution of categorical data.</p><p>I wanted to share this with you while working on this plot. Everything about this plot was completed, but not in this order. I was thinking about how to order these bars from maximum to minimum. Then I saw the option &quot;order,&quot; where we can give the order of categories in a list. We can’t do it manually since the categories are different for different columns. I have sensed why we should not use list comprehension since we can use lists in &quot;order.&quot; I gave it a try, and it worked out. I have enjoyed that moment since I applied what I learned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OGqQirRR1jbuc9X5Ew4m-g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GlCUqFkzQJU8M_uW95D_Pg.png" /></figure><p>From the plots, we can see,</p><p>Most of the time, there are six defenders in the box where they can directly tackle the offense.</p><p>Teams achieved 1 down most number of times.</p><p>Most games were played at 13:00 Eastern time.</p><p>The shotgun formation is the most preferred offensive formation by the teams.</p><p>Most of the time, the pass result is &quot;C-Completed.&quot;</p><p>The majority of the defenders cover man to man.</p><h3>Count plots for many categories</h3><p>Since ‘foul name’ has more than 25 categories, I have split it into the Top 5 and bottom 5 for clear visibility, or they would be rushing with one another.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EtEA96gKBbKkNC2P47NRHg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tvlJvqjpA6FWG-jwEBgX2g.png" /></figure><p>From this plot, we can see that,</p><p>Offensive holding, defensive pass interference, and defensive holding are the most common fouls committed by players.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uUXVZbTtrKrKyJ0tpApuLg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k_tublpuDid9ZosGBlk9Ew.png" /></figure><p>From this plot, we can see that,</p><p>The two most common fouls committed by players are illegal substitution and disqualification.</p><h3>Word Cloud</h3><p>A word cloud visualizes the most common words in a text or a group of texts. It is created by displaying the words in such a way that the size of each word is proportional to its frequency in the text.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RzKwV5spGx3FlfyHektTLw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/402/1*wCb2qlpXK44ZmJYeJTJ_3Q.png" /></figure><p>From this word cloud, we can see,</p><p>Teams like KC, MIA, TEN, and TB have played the most games.</p><h3>Conclusion</h3><p>Overall, this univariate analysis provides a valuable starting point for further exploration and analysis of the data and will be a helpful resource for beginners. Still, relationships, comparisons, and multivariate analyses are yet to be covered, and we will see that in the upcoming article.</p><p>My Kaggle notebooks on the NFL Big Data Bowl 2023 <a href="https://www.kaggle.com/code/arunkumardsci/merging-and-optimizing-the-nfl-data">Optimization</a> and <a href="https://www.kaggle.com/code/arunkumardsci/univariate-analysis-of-nfl-big-bowl-2023-data">Univariate analysis</a></p><p>Code for optimization and merging the data You can use my <a href="https://github.com/kumarnarun/NFL_bigdata_bowl2023_Kaggle/tree/main/nfl-big-data-bowl-2023/optimized_dataset">GitHub</a> repo.</p><p>For more data science insights, connect with me on LinkedIn.</p><p><a href="https://www.linkedin.com/in/arunkumar-data-scientist/">https://www.linkedin.com/in/arunkumar-data-scientist/</a></p><p>Thanks &amp; Regards</p><p>Arun Kumar</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25f0e44f105f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/revealing-the-hidden-insights-of-nfl-big-data-bowl-2023-through-univariate-data-visualization-25f0e44f105f">Revealing the Hidden Insights of NFL Big Data Bowl 2023 through Univariate Data Visualization</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Choose the super speed iteration to process large datasets-Part 2]]></title>
            <link>https://medium.com/aatomz-research/choose-the-super-speed-iteration-to-process-large-datasets-part-2-c19e85ba1d3?source=rss-969ea81e348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/c19e85ba1d3</guid>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[numpy]]></category>
            <dc:creator><![CDATA[Arunkumar N]]></dc:creator>
            <pubDate>Tue, 22 Nov 2022 16:45:51 GMT</pubDate>
            <atom:updated>2022-11-22T16:47:13.903Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eD7fmOdW7t2Gtg-Zx50Gug.png" /><figcaption>Super Speed Iteration — Part 2 (Credits: Aatomz Research)</figcaption></figure><p>This is the continuation of Part 1 of the article<a href="https://medium.com/aatomz-research/choose-the-super-speed-iteration-to-process-large-data-sets-bec7ad560348"> <strong>&quot;Choose the super speed iteration to process large datasets.”</strong></a> The following 6 methods are the most efficient, considering their performance compared to previous methods. So let’s go and enjoy the process.</p><h4>1. Eval</h4><p>From the execution time seen below, we can say that the Eval is one of the dynamic functions of Pandas, especially for doing arithmetic operations and comparison operations. The performance will be faster only when the data frame contains more than 10k rows. This method will be beneficial when dealing with huge datasets. Because the execution time is less, the timeit function executes the operation 7*100 times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ff7XNVHxcWXjFtXINa-gw.png" /></figure><h4>2. Assign</h4><p>This method is even faster when compared to Eval. We can assign many columns in the existing data frame by using this function. You can see that Eval and Assign functions run 7*100 times by the timeit function. Both functions have approximately similar run times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fd-o-uTYXQ3LeJx3jSWvcw.png" /></figure><h4>3. Div</h4><p>We came to the final four functions. With my research and experimentation, I have found that these four methods run 7 * 1000 times and give us the best possible run-time result we can get, which is the mean run-time result. This method, one of the Pandas data frame attributes, does the normal scalar arithmetic division of the columns or between two columns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_esh4SBd0aeMdp48Ezb3jw.png" /></figure><h4>4. Pandas Vectorization</h4><p>For numerical operations in a Pandas data frame with millions of rows, vectorization is the way to go. Especially for homogeneous operations, vectorization is very useful. In this case, the &quot;price&quot; and &quot;USD price&quot; columns have similar datatypes; hence, during vectorization, an array of price values and USD price values is created, and a division operation takes place simultaneously between the two arrays. So this method is more efficient and gives us faster results. The timeit function runs it 7 * 1000 times, as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vQTDaTZof73QCMX2G5S78w.png" /></figure><h4>5. Numpy Vectorization</h4><p>If the vectorization happens in Numpy arrays, it is Numpy vectorization. The Numpy arrays are dynamic and faster than Pandas arrays, as the Numpy array uses C. The execution time of the code is in microseconds, and that tells you how effective this method</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eeQlvbIyri87pdBkLlaNTw.png" /></figure><h4>6. Values</h4><p>This method returns the values of columns in the NumPy array, ignoring the labels of rows and columns. Numpy vectorization and values do similar functions, but the values method is a bit faster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cnygtYEbg1rwBrZqekC_MQ.png" /></figure><h4>Conclusion</h4><p>Here is the performance comparison for all methods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m8Up0aTp0nW25pCQfSt87A.png" /><figcaption>Performance comparison of Iteration Methods</figcaption></figure><p>There have been a lot of methods for doing the same operation. But which method is powerful and efficient in saving your precious time? Through this research and experimentation, I have found out that the “.values” method has a faster execution time compared to other methods. But I would suggest you use “.to_numpy()” as the official <a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.values.html">documentation</a> of Pandas&#39; recommended use instead of “.values” as it is deprecated.</p><p>I have done this research only on the numerical values. Still, there is much to know about how vectorization handles strings, and various operations are yet to be explored. Query the universe and learn more. Never give up on anything. Until the following article, Take care.</p><p>For more data science insights, connect with me on LinkedIn.</p><p><a href="https://www.linkedin.com/in/arunkumar-data-scientist/">https://www.linkedin.com/in/arunkumar-data-scientist/</a></p><p>Thanks &amp; Regards</p><p>Arun Kumar</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c19e85ba1d3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/choose-the-super-speed-iteration-to-process-large-datasets-part-2-c19e85ba1d3">Choose the super speed iteration to process large datasets-Part 2</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Choose the super speed iteration to process large data sets.]]></title>
            <link>https://medium.com/aatomz-research/choose-the-super-speed-iteration-to-process-large-data-sets-bec7ad560348?source=rss-969ea81e348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/bec7ad560348</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[numpy]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Arunkumar N]]></dc:creator>
            <pubDate>Mon, 14 Nov 2022 16:14:24 GMT</pubDate>
            <atom:updated>2022-11-15T06:46:17.286Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kpsmiOYvoXy16JHcoayVuQ.png" /></figure><p>Data Science is a fantasy job until you enter big data analytics 😅😅😅. I went through many challenges while working on my first big data project, and I realized the most critical challenge I had to deal with was time consumption. So I did much research to handle some processes efficiently and thought to share some of my research here.</p><p>In this article, we will see different methods for iterating over a pandas data frame and explore which way is faster and more efficient while handling larger datasets.</p><p>You can take any <a href="https://data.world/wfp/dc663585-4b6f-46ae-a6d6-b2f3e4ea32b5">dataset</a> that requires mathematical operations and functions. I have imported pandas for data analysis and NumPy for numerical operations through arrays.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/968/1*L1V1wl8iHdSXfPzxY-bscg.png" /></figure><p>I checked the number of rows and columns using shape and filtered the needed columns for easy operations and visibility.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*n8Xi1_6GwAZZF_0VRe-hYw.png" /></figure><p>I created a function for division operation as I needed the ratio between the Indian and USD prices. We will make a ratio column throughout all methods and check for their run time.</p><p>I have copied the original data frame by values to explain each method and for more clarity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*BTREsXzg611yAAW354jlAw.png" /></figure><h4>%timeit function</h4><p>It is one of the magic commands of the python Jupyter notebook used to measure the execution time of the code in the best possible way by taking the mean and standard deviation of the number of run times and providing us with the best runtime result.</p><h4>Iterrows</h4><p>As you can see, the runtime of the Iterrows method is probably one of the most extended run times I have ever seen after optimization. This method iterates over rows and returns index, series Paris. Here I have ignored the index by using (_,). Be careful while using this method as it does not preserve the datatype across the rows.</p><p>Here I have used Iterrows inside list comprehension for iterating over the ‘price’ and ‘USD price’ columns and does the divide operation by using the divide function as mentioned earlier and creating a ‘ratio1’ column in the df_iter_rows data frame. I have run it two times and created two columns for experimental purposes. As the run time is more extensive for iterrows, the time it functions runs is only 7*1 times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LEbMJzceAXaVnG0q1YsDZQ.png" /></figure><h4>Apply</h4><p>You can use apply functions in a variety of ways. Here I have used lambda inside the apply function to do operations between two columns. Usually, apply function is used for convenience, not performance, as you can see in the execution time below. It is faster than Iterrows but not faster enough.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t9lfLY4jbCuUDbgdzge3wA.png" /></figure><h4>itertuples</h4><p>This method is 100 times faster than the iterrows and 15 times faster than the apply() methods as it converts the data frame into a list of tuples and iterates over it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oxPXhDCIe9ibjTjq5A2dXA.png" /></figure><h4><strong>Zip</strong></h4><p>We use the membership function — ‘in’ wherever possible since it is precise and faster. Here I have used the Zip function to perform a division operation between two columns. This method gives us faster results than previous ones, and you can also note that it runs 7*10 times as the response to the Zip function much quicker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q3b7iJs_AdxH1-lk7tl-4g.png" /></figure><h4>Map</h4><p>It is one of the most efficient python functions because it is precise, faster, and consumes less memory. It also works very well in handling strings. It is much quicker when compared to previous methods. You can see that Zip and map have similar run times, but the map is a little bit faster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rXoHZVWP5EEXm_OHrqFh6Q.png" /></figure><h4>Vectorize</h4><p>It is one of the NumPy operations to map a function. It uses broadcasting methods of NumPy for optimizing memory and run time. When it comes to performance, it performed well when compared to previous methods. Itertuples, Zip, Map, and Vectorize all fall under similar run time zones run 7*10 times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5e14FFZE7mnQPfeIozdkcw.png" /></figure><h4>Conclusion</h4><p>From the above methods, we can conclude that Vectorize is faster for numerical operations. Zip and Map will be the go-to method in most cases, like handling many other functions having strings instead of numbers. Itertuples are also faster but not faster enough, like zip and map. I won’t recommend using iterrows and applying when the data frame size is larger since it is too slow.</p><p>I hope these methods will definitely help you while iterating the Pandas data frames. Still, there are even faster and more efficient methods yet to cover. We will see those things in the upcoming article.</p><p>For more data science insights, connect with me on LinkedIn. <a href="https://www.linkedin.com/in/arunkumar-data-scientist/">https://www.linkedin.com/in/arunkumar-data-scientist/</a></p><p>Thanks &amp; Regards</p><p>Arun Kumar</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bec7ad560348" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/choose-the-super-speed-iteration-to-process-large-data-sets-bec7ad560348">Choose the super speed iteration to process large data sets.</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[3 Effective Methods to Slice Your DataSet using Pandas]]></title>
            <link>https://medium.com/aatomz-research/3-effective-methods-to-slice-your-dataset-using-pandas-bf1fa2b2408d?source=rss-969ea81e348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf1fa2b2408d</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Arunkumar N]]></dc:creator>
            <pubDate>Sun, 16 Oct 2022 18:39:00 GMT</pubDate>
            <atom:updated>2022-10-16T18:39:00.286Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MyH_YMT3rsyBDTKN7iCIwQ.png" /><figcaption>3 Effective Methods to Slice Your DataSet using Pandas (Credits: Aatomz)</figcaption></figure><p>Pandas is one of the most valuable libraries for data analysis tasks. In analyzing the data deeply, we often slice the data as required. Here we will see how we can slice a data frame using pandas in the 3 most efficient ways.</p><p>I have taken the ‘Indian food prices’ dataset from the data world. You can download it from <a href="https://data.world/wfp/dc663585-4b6f-46ae-a6d6-b2f3e4ea32b5">here</a> for reproducibility.</p><p>Before analyzing any dataset, we need to optimize the dataset by converting data types of columns to needed data types and here I am loading the optimized dataset.</p><h3>Loading the dataset:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QW7jxiXh1jcKQPg6BY1_tQ.png" /></figure><p>We can do so many different things by following data frame operations like filtering, indexing, and so on. Here I have focused only on slicing methods.</p><h3>1. Slicing by iloc[]:</h3><p>Here I have sliced the first 5 rows and first 5 columns of the data frame. we have done integer location-based slicing. The initial 0:5 represents the row index, and the final 0:5 represents the column index</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O_dvgd7I7cB1biGMQq5WGQ.png" /></figure><h4>Slicing middle n rows:</h4><p>Here I have sliced the middle 5 rows. I have found the middle value of the index and used it inside the iloc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xQbD7n3VqSTBUOtAlfi40A.png" /></figure><p>I have chosen 5 rows from the middle row as a slicing point.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0MllR3kypVcUnIfffsHoow.png" /></figure><p>Here I have sliced the last 5 rows and chosen some specific columns using the lambda function. It will work without using the lambda function also.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y2K5OtGvU33OIvdGmrFm6g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*bXyoCXN-BqMUqNc7kgTlrA.png" /></figure><h3>2. Slicing by truncate():</h3><p>Here I have sliced from 50 to 100 rows by using the truncate option. By default, it will truncate the rows if you don’t mention the axis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lTrdeOh0BZsqoVJSWXWoJA.png" /></figure><h4>Slicing by DateTime:</h4><p>Let’s slice the data frame by setting the date as the index. Before doing that, I copied the data frame by its value so that the original data frame did not get affected while doing any operations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S1MU2U16m9R6MtDnF-PURg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CK_unX4nwMhbMtpDnlSDjA.png" /></figure><p>I have truncated the data frame by date from the year 2021 to 2022</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9F1uQC9jnkwtCMVO8MzlSQ.png" /></figure><h3>3. Slicing by loc[]:</h3><p>we can do a lot of operations using loc[] especially filtering, filtering the data frame by any conditions. I think it is one of the key attributes of the Pandas data frame. It takes values based on row labels and column labels.</p><p>Here I have sliced from 100th to the 200th row, and I have sliced the column in steps. I need the date and price columns only, so I opted to ignore the 11 columns in between them. We can do it for rows too.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aIHNUDO-iMTAh6dCDYNI-Q.png" /></figure><h4>Slicing by DateTime:</h4><p>I have sliced the data frame from the year 2000.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HdFyTpJ5YoShW6NsDnbZqg.png" /></figure><p>I have overlooked the prices from the year 1997 to 2000.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*fmx77-CUl2bgkJgw8Ufxqw.png" /></figure><p>We can also slice the columns along with rows. Here I have sliced the ‘unit’ column to the ‘price’ column for the period 2015–2016.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QX6ZGVmnk1TEJEPPzonVEg.png" /></figure><p>Here I have sliced all the columns after the ‘price’ column and found the ‘price’ and ‘USD price’ for 2010–2011.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2B1u44qiBlC3f8gY6GDHsQ.png" /></figure><p>We can use lambda for choosing specific columns. Here I have found the ‘market’ and ‘price’ for that particular time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/976/1*Is3gwkEJ6O6Gs3y8SN1wsg.png" /></figure><p>We can choose the specific columns without using the lambda function also. I am just showing you the options. You can follow any method. Here I have chosen ‘market,’ ‘commodity,’ and ‘price’ for 1994–2000.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*snVC7AHzwEaUs1TXAo8w7w.png" /></figure><p>In our day-to-day life, as a data-scientists, we play with datasets and extract valuable insights. To gather valuable insights, we need to analyze the data thoroughly. Here we’ve checked how the prices of each commodity are in different periods using simple slicing techniques.</p><p>I hope this article will give you some clarity about slicing. We can do many more analyses in the upcoming article.</p><p>Thanks</p><p>For more data science insights, connect with me on LinkedIn.</p><p><a href="https://www.linkedin.com/in/arunkumar-data-scientist/">https://www.linkedin.com/in/arunkumar-data-scientist/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf1fa2b2408d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/3-effective-methods-to-slice-your-dataset-using-pandas-bf1fa2b2408d">3 Effective Methods to Slice Your DataSet using Pandas</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How did I convert the 33 GB Dataset into a 3 GB file Using Pandas?]]></title>
            <link>https://medium.com/aatomz-research/how-did-i-convert-the-33-gb-dataset-into-a-3-gb-file-using-pandas-b21d8da205c0?source=rss-969ea81e348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/b21d8da205c0</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Arunkumar N]]></dc:creator>
            <pubDate>Sat, 17 Sep 2022 19:10:36 GMT</pubDate>
            <atom:updated>2022-09-17T19:10:36.200Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GpxIDx0Xa3DmftwmH1iQ0g.png" /><figcaption>credits to: aatomz</figcaption></figure><p>Optimizing the dataset memory is the first and critical step in any data science project. A dataset may contain thousand to millions of rows. We can load the smaller dataset quickly, but when the dataset gets bigger, our system may not be able to run the large memory files. So I thought to share how I converted the 33GB data file into 3GB for easy loading.</p><h4>Dataset</h4><p>I have taken the recently trending ‘American Express-Default prediction’ dataset. You can download it from<a href="https://www.kaggle.com/competitions/amex-default-prediction/data"> Kaggle</a> for reproducibility.</p><h4>Importing the Libraries</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8WY5hqGYAVnZYq9QBPMNvg.png" /></figure><p><a href="https://pandas.pydata.org/docs/getting_started/index.html">Pandas</a> is one of the most commonly used libraries in python for data analysis. We can use it for data processing such as manipulation, concatenating, joining, merging, statistical analysis of data, and many more.</p><p>A garbage<a href="https://docs.python.org/3/library/gc.html"> collector(GC)</a> frees up the space from the memory while handling extensive data. It removes unwanted things from memory.</p><p>The glob module extracts specific files using patterns from our system.</p><p>OS module interacts with the operating system and handles the file and its paths.</p><h4>Chunking</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sPHmIt6_KEkWHpt_42iN-w.png" /></figure><p>Here we can’t be able to load the data file generally as we do in pandas since the file size is heavier, causing a memory error. So we use the ‘chunk size’ option to split the files as we wish. Here I have chosen 5 lakh rows per chunk. Here ‘GC.collect()’ is crucial and vital since it saves you from a memory error. I got 23 chips after running this.</p><p>After chunking, see whether you can read the chunk and check the info.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GF1bwFV1ukjRj1PNFyNGsg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XuMsYjZrxoj5DVhpteIqPQ.png" /></figure><p>From the info, we can see that there are 190 columns with float64 in 185 columns. It is one of the common problems with Pandas always loading the float data as float64. By optimizing this memory, we could have reduced a part of the memory from our dataset. So we have to convert it to ‘float16’ or ‘float32’ to minimise memory usage. Here I have converted it to ‘float16’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LAJpJNQifhMQ1CS3RH8HRg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X8fc0hpVzpFKu-cj2gAGqQ.png" /></figure><p>You can see after converting to ‘float16,’ memory usage has been drastically reduced.</p><h4>Optimizing and concatenating</h4><p>Previously we read and optimized for a single chunk file. Now we are going to contact all 23 chunk files as well as optimize the memory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iPq3idO_lxSn4HCAz2Q_6g.png" /></figure><p>Here I have used ‘glob’ and ‘ os. Path. join’ method to access corresponding files by file path and its component, i.e. (“*.csv”). We have to read all the files by iteration. During iteration, we can use conditions to convert from ‘float64’ to ‘float16’. After converting, we have to save the chunks in a list. Then we can contact all the files from the list and keep them in a new data frame.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/1*3xw4k6FdpfUfXSb081dJpA.png" /></figure><p>Now you can see the details of the new data frame after concatenating. We can do data processing operations in this data frame quickly. We can read 11 million rows at an instance, which is initially impossible due to the bigger file size.</p><h4>Converting data frame to file format</h4><p>The optimized data frame could be converted into any file format. I recommend the feather format because of its lightweight, as you see below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/947/1*czBr8-QRBBdd4lbF3vbCxg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KoD-2zeL6SNnUsDVv9vG6A.png" /></figure><h4>Reading the file</h4><p>Now we can read the optimized feather format file without any memory error. We can do pandas operations, functions, data analysis, data manipulation, and more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nEtNBXB_pPROxbnofIhSSg.png" /></figure><p>We can do further optimization by analyzing ‘object ’ columns and converting them to ‘category’ or ‘string. We can do the same thing in other tools like dask and data table, but our aim is to deep dive into all the possibilities in the known territory and thus see pandas as a powerful tool. I hope this article will help handle the large datasets!</p><p>In case any queries or more clarification is needed, ping me on LinkedIn.</p><p>For more data science insights, connect with me on LinkedIn.</p><p><a href="https://www.linkedin.com/in/arunkumar-data-scientist/">https://www.linkedin.com/in/arunkumar-data-scientist/</a></p><p><a href="https://www.kaggle.com/code/arunkumardsci/memory-optimiztion-amex-dataset">memory_optimiztion_amex_dataset</a></p><p>Also, please check out my <a href="https://github.com/kumarnarun/Amex-Default-Prediciton">GitHub repo,</a> if you want to download the complete working code.</p><p>See you in my next data science article, babye!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b21d8da205c0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/how-did-i-convert-the-33-gb-dataset-into-a-3-gb-file-using-pandas-b21d8da205c0">How did I convert the 33 GB Dataset into a 3 GB file Using Pandas?</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Four Pattern Occurrences can Save Your Life during Text Mining Projects.]]></title>
            <link>https://medium.com/aatomz-research/four-pattern-occurrences-can-save-your-life-during-text-mining-projects-4a4a89f26a6e?source=rss-969ea81e348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/4a4a89f26a6e</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[text-mining]]></category>
            <category><![CDATA[data-mining]]></category>
            <category><![CDATA[aatomz]]></category>
            <dc:creator><![CDATA[Arunkumar N]]></dc:creator>
            <pubDate>Sat, 20 Aug 2022 20:04:14 GMT</pubDate>
            <atom:updated>2022-08-20T20:04:14.351Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZBaHtC1mXE1o6pcw76F6RQ.jpeg" /><figcaption>Credits to Aatomz</figcaption></figure><p>In this data-dominating world, we can extract the required information efficiently using Regular Expression (Regex) in any Programming Language. Suppose you are given a data-containing book full of contact information. If you want to extract the user’s mail-id alone, Regex will do that job in a few minutes.</p><p>However, practicing the Regex pattern occurrences is vital to retrieve the exact information. So, In this article, I am here to explain the 4 pattern occurrences from the regular expression that can save your life during text mining projects.</p><h4>Pattern 1: Zero or more occurrences (*)</h4><p>As the name suggests, it repeats the character or pattern zero or more times.</p><p>Before starting the code, we have to import the regex module.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*KLt1TBJvDIyLo8VeK9hVgQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NBs6y-uF_h7PWwDCBItPFw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2O7W_k4W1PfFBQyZF9J4dQ.png" /></figure><p>In the first example, we take random text as input, and then we create a pattern to find something in the text. For finding, we use re.findall(pattern, txt) that finds all the pattern matches in the text. I have created a pattern that starts with ‘94’ and dot(.) indicates any character or wildcard character except newline, and our hero (*) indicates that the wildcard character could occur zero or more times, and the pattern should end with ‘8’. For this pattern, we got (9488488458), which is in the text as output. Here the pattern exactly matches the numbers in the text. If the text contains some letters between ‘94’ and ‘8’, it brings all of them.</p><p>In the second example, we are ignoring the wildcard character (.) dot. In this case, (*) indicates the number ‘4’ could occur zero or more times in between ‘9’ and ‘8’. we got two results. ‘944448’ has more occurrences of ‘4’ and ‘98’ has zero occurrences of ‘4’.</p><h4>Pattern 2: One or more occurrences (+)</h4><p>The character or a pattern repeats one or more times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E4ruWAFCIZm2Kvc5_XLjNw.png" /></figure><p>In this example, ‘\d’ represents numbers(0–9) in a string, and ‘+’ indicates one or more occurrences of the numbers. In the given input, there are many numbers. By using this pattern, we have extracted the numbers in the input. If ‘+’ is not included after ‘\d’, the output will be unique numbers.</p><h4>Pattern 3: Zero or one occurrence (?)</h4><p>It repeats the character or pattern zero or only one time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DBy7pEMgGfI5DRT2kFlBMw.png" /></figure><p>The above example (?) indicates zero or one occurrence of ‘i’ in the text. The first result, ‘fire’, is an example of one occurrence since one ‘i’ is present between ‘f’ and ‘r’. The result ‘free’ is an example of zero occurrences since there is no ‘i’ between ‘f’ and ‘r’.</p><h4>Pattern 4: The exact number of occurrences ({ })</h4><p>We use this for how often a character or pattern needs to be repeated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g0j1bHD0Nx0PI-wSV25bjQ.png" /></figure><p>In this example, we will collect all the mobile numbers from the given input. For this, we have to recognize the pattern for the mobile numbers shown in the text. Let’s do this. In the given text, a mobile number has 10 digits which are represented by ‘\d{10}’ or a 10-digit number with a code ‘+91-’, which is represented by ‘\+91-\d{10}’. Here ‘|’ means either or. To indicate ‘+’ as the raw character, we are putting ‘\’ before ‘+’ since it has special meaning in Regex, as we saw in the previous examples.</p><p>I hope I have given some valuable insights about Regex. Still, there are many incredible things to learn from it. Learn Regex and efficiently do code. See you all in the upcoming article. Keep your heads high. keep rising.</p><p>Cheers!</p><p>By the way! Would you like to Ask for more info? here is my LinkedIn</p><p><a href="https://www.linkedin.com/in/arunkumar-data-scientist/">https://www.linkedin.com/in/arunkumar-data-scientist/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4a4a89f26a6e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/four-pattern-occurrences-can-save-your-life-during-text-mining-projects-4a4a89f26a6e">Four Pattern Occurrences can Save Your Life during Text Mining Projects.</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a Python Library for Material Science]]></title>
            <link>https://medium.com/aatomz-research/build-a-python-library-for-material-science-83e32ef292a2?source=rss-969ea81e348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/83e32ef292a2</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[materials-science]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[materials]]></category>
            <dc:creator><![CDATA[Arunkumar N]]></dc:creator>
            <pubDate>Tue, 02 Aug 2022 02:37:32 GMT</pubDate>
            <atom:updated>2022-08-02T02:37:32.596Z</atom:updated>
            <content:encoded><![CDATA[<p>In this AI-driven world, technology has been changing on a day-to-day basis. From Aluminium frame shuttle racquets to highly flexible carbon fiber racquets. From heavy metals to lightweight composites in automobile and space industries. Everything is upgrading at a fast pace. So we must adapt to learn technical skills and apply them in our core field.</p><p>In this article, we focus on how we can develop a simple python library for Hall-Petch relation in the material science domain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AjPVtqa_BkCbKkvc-HcsmA.jpeg" /><figcaption>Python Library for Material Science: (Credits to: Aatomz Research)</figcaption></figure><h4>Grain and Grain boundary</h4><p>If we break a metal, we can see grains where atoms are arranged periodically. The collection of grains forms a solid. Each grain is separated by a grain boundary. The properties such as strength and hardness of a material can be influenced by the amount of grain and grain boundaries.</p><figure><img alt="This image showcases the grain and grain boundaries" src="https://cdn-images-1.medium.com/max/291/1*yt14IBZwkmI5bd0dFmbDrw.png" /><figcaption>Credits to: <a href="https://material-properties.org/">https://material-properties.org</a></figcaption></figure><h4>Hall-Petch Relation:</h4><p>The relationship between yield strength and grain size is given by</p><p>σy = σ∞ + k/√D where,</p><p>σy is the yield strength</p><p>σ∞ is a constant (i.e.) the yield stress for infinitely large grain size</p><p>k is a constant (i.e.) the locking parameter is the resistance of grain boundaries in stopping the dislocation</p><p>From this relationship, we can conclude that for smaller grain sizes, the yield strength will be maximum.</p><h4>Python Module (Library):</h4><p>Compilation of different functions or operations for a highly reusable purpose is a module. For example, a calculator does addition, division, and many processes. Here calculator is a module that contains all functions.</p><p>Like a toolkit, medikit whenever there is a need, we take and use it. In the same way, we can import the module whenever it is needed and utilize it.</p><p>Now let’s look at some essential formulas and how we can apply those in creating a module using python.</p><h4>1) Calculating grains per square mm = N</h4><p>We can calculate the grain count by using the formula,</p><p>N = (2(n-1)) × (100/M)2</p><p>where n is ASTM grain size number</p><p>M is Magnification</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PNgRqqpGfrmGNhXCJuuGEg.png" /></figure><p>It is nothing but a module for doing mathematical operations. Here we are importing that module since we are doing calculations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U0h-qqlTKTuQyFM1y88R3g.png" /></figure><p>Here in this code, we define (def) what we will do(calculate_graincount). This is known as defining a F<a href="https://www.programiz.com/python-programming/function">unction</a>. Inside the parenthesis(n, M) are <a href="https://stackoverflow.com/questions/47169033/parameter-vs-argument-python">Parameters</a>.</p><p>The sentence inside the triple quotation is <a href="https://www.programiz.com/python-programming/docstrings">Docstring</a>, where helpful information can be given.</p><p>The corresponding grain count formula is coded and returned the grain count so that we can use it further anywhere; otherwise, we can use it only once.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XurGZCGlvz00-Iqky7A-1A.png" /></figure><p>After running the code, we can call the function above and enter the desired values. Here the values(6,1) is <a href="https://stackoverflow.com/questions/47169033/parameter-vs-argument-python">Argument</a>. we can get the output after running the called function.</p><h4>2) Calculating average grain diameter = D</h4><p>We can calculate average grain diameter by D = 1/√N, where N is the of grains per square mm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*7Zzrom4S_qGA0bdOp8Lx4g.png" /></figure><p>Here math.sqrt won’t work if the math library is not imported.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ddMzjVHDOi3Zn4TucqPpBA.png" /></figure><h4>3) Calculating locking parameter = k</h4><p>It can be calculated from the graph by finding the slope when yield stress(σy) is plotted against D-1/2. It is the contribution of grain boundary toughness.</p><p>slope_k = (y2-y1)/(x2-x1)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4wRidpDPWdis2YaUQlaCHQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D2BBFC_BlFi-MFqweG74LA.png" /></figure><h4>4) Calculating yield strength = σy</h4><p>The stress at which plastic deformation occurs when a material loses its elasticity is the Yield strength. It is vital in any engineering and design application in the manufacturing and construction sectors. It can be calculated by using the Hall-Petch relation,</p><p>σy = σ∞ + k/√D</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9PZQO2pchz-GTrr2KgMZPQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xWLblgBZP8KsCSsvmemK_A.png" /></figure><p>After creating a module, we have to download it as a python file(.py) and save it.</p><h4>Permanently adding the module.</h4><p>We must add the created module permanently in the anaconda environment so we can import the library as we need.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*Tyd1NF5i9FnodsREaKZmdA.png" /></figure><h4>Importing Module</h4><p>Now we can import the created module and use it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XSYeTaWu-FlM4dfSslaiDA.png" /></figure><p>I hope this article helped you to gain some knowledge about creating a module in python for the benefit of the core engineering purpose where mathematical calculations are essential.</p><p>Awake the genius within you. See you in the upcoming article.</p><p>Cheers!</p><p>Follow me on LinkedIn.</p><p><a href="https://www.linkedin.com/in/arunkumar-data-scientist/">Arunkumar Nagarajan - Associate Content Analyst - Clarivate CompuMark | LinkedIn</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=83e32ef292a2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/build-a-python-library-for-material-science-83e32ef292a2">Build a Python Library for Material Science</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[3 Fundamental steps to build a Sign-Up Form with Python]]></title>
            <link>https://medium.com/aatomz-research/3-fundamental-steps-to-build-a-sign-up-form-with-python-94083b1d2d52?source=rss-969ea81e348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/94083b1d2d52</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[sign-up]]></category>
            <category><![CDATA[aatomz]]></category>
            <category><![CDATA[data-structures]]></category>
            <dc:creator><![CDATA[Arunkumar N]]></dc:creator>
            <pubDate>Thu, 07 Jul 2022 04:04:00 GMT</pubDate>
            <atom:updated>2022-07-07T04:04:00.882Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="3 Fundamental steps to build a Sign-Up Form with Python" src="https://cdn-images-1.medium.com/max/1024/1*rWVmTP7PJvZSMGlgI-tRFQ.jpeg" /><figcaption>CREDITS TO: <a href="http://www.aatomz.com/">aatomz research</a></figcaption></figure><p>For accessing any web application by users, a Sign-up page is inevitable. So it is essential to create an excellent sign-up form for storing the user information.</p><p>It’s an entry gate to access any website or app by registering user information. Here we are going to see how we can build a simple sign-up form using Python in 3 steps.</p><h4>1. Collecting User Inputs</h4><p>Initially, we are going to get input from the users. Any information or data sent to a system for processing is considered input. Here we will get Username, Mail-Id, and Phone number as inputs from the users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*joYRu8SLw_9tylvny5vY5Q.png" /></figure><h4>2. Storing User Information in Dictionary</h4><p>Once we get the user information, we need to store it somewhere. We use <a href="https://www.programiz.com/python-programming/dictionary">Dictionary</a> to achieve this. It is nothing but a data <a href="https://www.geeksforgeeks.org/python-data-structures/">structure</a> where we can keep a collection of data in a key: value pair, represented by curly braces { }.</p><p>Data are stored inside the curly braces separated by commas. Here we can see how can we store the given input data in a dictionary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/947/1*XFEqhn9JpkqUNm3I-Fv0xA.png" /></figure><p>Here ‘mail-id’ in the red indication is the key, and the value is on the right side after the colon. We can access data from the dictionary by using the key.</p><p>All user information is now stored in the ‘user1’ dictionary. If we run ‘user1’, all the user information is displayed in the output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qVxMb0MApdruspzDwfIaIA.png" /></figure><h4>3. Adding multiple users in a List:</h4><p>For storing many users’ information, we can use <a href="https://www.programiz.com/python-programming/list">List</a>, which is represented by a square bracket [].</p><p>For example, in the same procedure as user1, we can collect user2 details and store both user1 and user2 in the List.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/885/1*IqzJuj4akhXCN_PnBZjNpA.png" /></figure><p>This article is only to understand that how can we start building concept of Sign Up form. Yet, there are more things such as validating and storing user information in database are critical. I will write about it in upcoming articles.</p><p>However, I hope this article will help beginners understand some basics of building a sign-up form and storing user information.</p><p>Cheers!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=94083b1d2d52" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aatomz-research/3-fundamental-steps-to-build-a-sign-up-form-with-python-94083b1d2d52">3 Fundamental steps to build a Sign-Up Form with Python</a> was originally published in <a href="https://medium.com/aatomz-research">Aatomz Research</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>