<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nouran Ali on Medium]]></title>
        <description><![CDATA[Stories by Nouran Ali on Medium]]></description>
        <link>https://medium.com/@nouranalimohammed?source=rss-d192b4414494------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*sj2RC18924lcoH6-fD8oTg.jpeg</url>
            <title>Stories by Nouran Ali on Medium</title>
            <link>https://medium.com/@nouranalimohammed?source=rss-d192b4414494------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 01:43:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nouranalimohammed/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Dive Into Deep Learning — Part 3]]></title>
            <link>https://nouranalimohammed.medium.com/dive-into-deep-learning-part-3-5cb23ca6d204?source=rss-d192b4414494------2</link>
            <guid isPermaLink="false">https://medium.com/p/5cb23ca6d204</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[regression]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Nouran Ali]]></dc:creator>
            <pubDate>Tue, 07 Mar 2023 11:17:10 GMT</pubDate>
            <atom:updated>2023-03-07T12:45:04.015Z</atom:updated>
            <content:encoded><![CDATA[<h3>Dive Into Deep Learning — Part 3</h3><p>In this part, I will summarize <strong><em>section 3.6</em></strong> Generalization, section from page 86 to page 113 are all about coding and I found them very clear to understand so I recommend you read them before reading this part.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*07XuItz1q80RtDz2Pmz1aA.jpeg" /></figure><ul><li><a href="https://medium.com/mlearning-ai/dive-into-deep-learning-part-2-c91fb72276fd">Dive Into Deep Learning — Part 2</a></li><li><a href="https://medium.com/mlearning-ai/dive-into-deep-learning-part1-452295c1da32">Dive Into Deep Learning — Part1</a></li></ul><h3><strong>Generalization</strong></h3><p>The authors give an example of students who prepare for an exam, student 1 memorizes the past exams questions and student 2 discovers patterns in the questions, if the exam is <br>1. Identical to past exams: student 1 outperforms student 2.<br>2. Fresh questions: student 2 will perform better.</p><p>The goal of ML is to <em>discover patterns</em> and not simply memorize our training data, the fundamental problem is how to discover that pattern that<em> generalizes.</em></p><p>In real-life ML work, we fit models using a finite collection of data even with the most extreme scale, the number of available data points remains small.</p><p>An important note to keep in mind is that we must consider the risk of not discovering a generalizable pattern in the data after training and fitting. This introduces two new concepts:</p><p><em>Overfitting: </em>fitting closer to our training data than to the underlying distribution.<br><em>Regularization: </em>techniques for preventing overfitting.</p><h3>Training error and generalization error</h3><p>In supervised learning, we assume that training data and test data follow the <em>IID assumption: data is drawn independently from identical distributions.</em></p><p><em>training error Remp</em>: statistic calculated on the training data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*SSomolcz-Mm_yUzw077IVw.png" /><figcaption><em>training error as a summation</em></figcaption></figure><p><em>regularization error R</em>: expectation taken with respect to the underlying distribution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*r3vAJx2DlTjeTH6igUxYEQ.png" /><figcaption><em>regularization error as an integral</em></figcaption></figure><p>In practice, we can’t calculate the generalization error R exactly because:<br>1. Nobody tells us the precise form of P(x,y).<br>2. We can’t sample an infinite stream of data points.<br>So we estimate R.</p><h3>Model Complexity</h3><p>In theory, when we have simple models and abundant data Remp and R tend to be close, but with complex models, this is not the case as the generalization gap grows.</p><p>We can’t conclude that our model has discovered a generalizable pattern based on the fitting of the training data alone, but, if our model class was not capable of ﬁtting arbitrary labels, then it must have discovered a pattern.</p><p>When a model is capable of ﬁtting arbitrary labels, low training error<br>does not necessarily imply low generalization error. However, it does not necessarily imply high generalization error either!</p><h3><strong>Underfitting and Overfitting</strong></h3><p>Both of them are related to model complexity,</p><p><em>Underfitting: <br></em>1. generalization gap (Remp-R) is small<br>2. model is too simple to capture patterns in data<em><br>Overfitting:<br></em>1. training error is lower than validation error.<br>overfitting is not always a bad thing in deep learning.</p><h3>Polynomial curve fitting</h3><p>Given training data consisting of a single feature x to estimate the corresponding y we try to find a polynomial of degree d.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/201/1*0PyXuCKR4LRooj9lIOC2tQ.png" /></figure><blockquote>A higher-order polynomial function is more complex than a lower-order polynomial function since the higher-order polynomial has more parameters and the model function’s selection range is wider.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*VSV1BtzinKq5-9Dlsy0qYA.png" /><figcaption>Influence of model complexity on underfitting and overfitting</figcaption></figure><p>Another big consideration to bear in mind is the dataset size, the fewer samples we have in the dataset the chances of overfitting increase, as we increase the amount of training data, the <em>generalization error typically decreases.</em></p><h3>Model Selection</h3><p>We select our ﬁnal model, only after evaluating multiple models that diﬀer in various ways.</p><p><em>model selection: choosing a final model among many models.</em></p><p>Keep in mind that we should not touch our test set until after we have chosen all our hyperparameters as if we used the test set to select a model we are at the risk of overfitting on the test data.</p><h3>Cross Validation</h3><p>Incorporating a <em>validation set </em>in addition to a test and train set helps us address the above problem and select a better model.</p><p>When training data is not big enough we may not be able to afford to hold out a validation set we use <strong><em>K-fold cross-validation</em></strong></p><ol><li>The original training data is split into K non-overlapping subsets.</li><li>Model training and validation are executed K times, each time training on (K − 1) subsets and validating on a diﬀerent subset.</li><li>The training and validation errors are estimated by averaging the results from the K experiments.</li></ol><p>I hope you enjoyed this part as much as I did, please leave comments on any improvements or ideas you have.</p><h4><a href="https://mlearning.substack.com/about"><strong>BECOME a WRITER at MLearning.ai</strong></a></h4><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5cb23ca6d204" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dive Into Deep Learning — Part 2]]></title>
            <link>https://nouranalimohammed.medium.com/dive-into-deep-learning-part-2-c91fb72276fd?source=rss-d192b4414494------2</link>
            <guid isPermaLink="false">https://medium.com/p/c91fb72276fd</guid>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Nouran Ali]]></dc:creator>
            <pubDate>Sun, 26 Feb 2023 08:10:27 GMT</pubDate>
            <atom:updated>2023-02-26T08:14:50.511Z</atom:updated>
            <content:encoded><![CDATA[<h3>Dive Into Deep Learning — Part 2</h3><p>This is part 2 of my summary of the chapters I read from the dive into deep learning <a href="https://d2l.ai/index.html">book</a>.</p><p>In case you didn’t read it this is part 1 summarizing section 3.1 pages 82 to the beginning of 86.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*07XuItz1q80RtDz2Pmz1aA.jpeg" /></figure><p><a href="https://nouranalimohammed.medium.com/dive-into-deep-learning-part1-452295c1da32">Dive Into Deep Learning — Part1</a></p><p>The following sections of the analytic solution talk about optimizing the model and how to calculate the gradients.</p><p><strong>Minibatch stochastic gradient descent</strong></p><p>The idea behind the gradient descent algorithm is to <em>iteratively reduce the error by updating the parameters in a direction that incrementally lowers the loss function.</em></p><p>The authors then mention two extremes to apply the GD algorithm...</p><ol><li>The naive approach:<br>Take the derivative of the loss function which is an average of the losses calculated on every example in the dataset, a full update is powerful but it has some drawbacks…<br><strong>Drawbacks:<br></strong>. Can be extremely slow as we need to pass over the entire dataset to make a single update.<br>. If there is a lot of redundancy in the training data, the beneﬁt of a full update is very low</li><li>The extreme approach<br>Consider only a single example at a time and update steps based on one observation at a time, does that remind you of something??<br>Yes, it’s the <em>stochastic gradient descent </em>algorithm or SGD.<br>It can be effective even in large datasets but it also has some drawbacks…<br><strong>Drawbacks:<br>. </strong>It can take longer to process one sample at a time compared to a full batch<br><strong>.</strong> Some NN layers work well when we process more than one observation at a time (batch normalization layer).</li></ol><p>If the two methods have major drawbacks then what should we do? It’s simple we just pick a middle ground.</p><p><strong>Minibatch stochastic gradient descent</strong></p><p>Instead of taking the full dataset as a batch or only taking a single sample, we take a minibatch, the size of the minibatch depends on many variables:</p><ul><li>Memory size</li><li>number of accelerators/GPUs</li><li>number of layers</li><li>dataset size</li></ul><p>But the authors recommend a number between 32 and 256 (multiple of a large power of 2)</p><p>Algorithm steps:</p><ol><li>Randomly sample a minibatch Bi of a fixed number |B| of training samples and initialize the model parameters randomly</li><li>Compute the derivative of the avg loss on the minibatch</li><li>we multiply the gradient by a predetermined small positive value η, <em>the learning rate</em>, and subtract the resulting term from the current parameter values.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/357/1*OW6G2HeRp1hglNy7X3GlwQ.png" /><figcaption>process of updating w and b</figcaption></figure><p>We are then introduced to some important concepts:</p><p><em>hyperparameters</em>: tunable parameters that are not updated in the training loop such as batch size |B|.</p><p><em>generalization: </em>ﬁnd the parameters that lead to accurate predictions on previously unseen data.</p><p><strong>The Normal Distribution and Squared Loss</strong></p><p>The normal distribution and linear regression with squared loss share a deeper connection than common parentage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/357/1*ICrDjkKegfC8SDbP2pCpsA.png" /><figcaption>normal distribution formula</figcaption></figure><p>Let’s assume that observations arise from noisy measurements where the noise is normally distributed as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/336/1*unlqpyPn_WsLXv_NssQaVA.png" /><figcaption>E is a noise term</figcaption></figure><p>Then the likelihood of getting a particular y of a given x is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/335/1*3D3UUCeTSHCJPThremN0YQ.png" /></figure><p>According to the principle of maximum likelihood, the best values of parameters w and b are those that maximize the likelihood of the entire<br>dataset</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/205/1*iQg89HA3AP5pCq7a-JtU2Q.png" /></figure><p>The final formula for minimizing the negative likelihood is as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*LBOFwqBe8GOZzXyK-qdOoA.png" /></figure><p>Finally, Linear regression is considered a single-layer neural network</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*57opRXBqqv_Rg1Htn7_3Eg.png" /></figure><p>And that’s all for section 3.1, it’s kinda intense but once you break it into smaller pieces, it’s enjoyable and easy.</p><p>Thank you for making it to the end of this article and will meet you in part 3!</p><h4><a href="https://mlearning.substack.com/about"><strong>BECOME a WRITER at MLearning.ai</strong></a></h4><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c91fb72276fd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dive Into Deep Learning — Part1]]></title>
            <link>https://nouranalimohammed.medium.com/dive-into-deep-learning-part1-452295c1da32?source=rss-d192b4414494------2</link>
            <guid isPermaLink="false">https://medium.com/p/452295c1da32</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[linear-regression]]></category>
            <dc:creator><![CDATA[Nouran Ali]]></dc:creator>
            <pubDate>Tue, 21 Feb 2023 19:10:28 GMT</pubDate>
            <atom:updated>2023-02-24T14:28:37.564Z</atom:updated>
            <content:encoded><![CDATA[<h3>Dive Into Deep Learning — Part1</h3><p>In this series, I will be sharing with you my summarization of the Dive into deep learning book, I started reading it as a review of what I know about DL and also to explore new concepts I might have missed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l6sqzVdOeuqU3l3DofFvBQ.png" /></figure><p>The book is available for free here along with the code examples: <a href="https://d2l.ai/index.html">https://d2l.ai</a></p><p>I will start with chapter 3: Linear Regression as the first two chapters are about mathematics needed in order to understand the concepts in the book.<br>I recommend this <a href="https://www.coursera.org/specializations/mathematics-for-machine-learning-and-data-science">specialization</a> on Coursera it’s very well illustrated and covers all that you need in order to start learning about DL and ML.</p><p><em>In this piece, I will summarize section 3.1 from pg 82 till page 86, the content of the book is very rich so I don’t want to make the article very long.</em></p><h3>Linear Regression</h3><p><strong><em>1- When to use regression?</em></strong><em><br></em>When we want to predict a <strong>numerical value</strong> like the price of an item, or its length.</p><p>The book walks us through an example of predicting house prices.</p><p><strong><em>2-</em></strong> <strong><em>There are some terminologies that we need to be aware of while reading which are:</em></strong></p><ul><li><strong><em>training set</em></strong><em>:</em> dataset where each row corresponds to one observation</li><li><strong><em>an example/data point/instance/sample</em></strong>: a single observation or row</li><li><strong><em>label/target:</em></strong> the thing we are trying to predict</li><li><strong><em>features/covariants:</em></strong> the variables upon which predictions are made</li><li><strong><em>n</em></strong>: number of examples in the dataset and <strong><em>i </em></strong>is the ith sample</li></ul><p><strong><em>3- Linear Regression assumptions</em></strong></p><p>given a function y=wx+b</p><ul><li>The relationship between the target y and the features x is approximately linear meaning that the conditional mean E[y|x=x] can be represented as a weighted sum of all the features x.<br>This assumption means that the expected value of the target can be expressed as a weighted sum of features.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/232/1*9MSujqOxM_w2ptH4urorAw.png" /><figcaption>house prices formula</figcaption></figure><p>Any noise follows a Gaussian distribution</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/232/1*ubidaJJ5iu_BZnHpzcnVXQ.png" /><figcaption>normal (Gaussian) distribution formula</figcaption></figure><p><strong><em>4- What is a model?</em></strong></p><p>A model is a solution that describes how features can be transformed into an estimate of the target.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/232/1*9MSujqOxM_w2ptH4urorAw.png" /></figure><p>as we said the target is a weighted sum of the features, here w-area and w-age are called <em>weights </em>and b is the<em> bias/offset/intercept.</em></p><p><strong><em>weights</em></strong><em>: </em>determine the effect of each feature on the target.<br><strong><em>bias</em>:</strong> determines the value of the target when all features are zero.</p><p>The goal of regression is finding w and b that make our model’s predictions as close to the real values (prices) as possible and <strong>produce the lowest error</strong>.</p><p>We will use the vectorized formula, <em>refer to section 2.1.4 in the book to understand broadcasting.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/232/1*MIJ8AMeUsrV_CxKLE0Gmqg.png" /><figcaption>the vectorized formula</figcaption></figure><p><strong><em>5- Loss Function</em></strong></p><p>The loss function is a quality measure for the model meaning, it quantifies the distance between the real and predicted values of the target.<br>The loss value is a non-negative number where smaller values are better and perfect predictions give a loss of 0.</p><p>The most common loss function for regression is the squared error given by</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/232/1*HypFN7jZhZ8ZfgVewPXXAQ.png" /><figcaption>the squared error loss</figcaption></figure><ul><li><em>large differences between y-hat and y lead to excessive sensitivity to anomalies but also encourages the model to avoid large errors.</em></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/228/1*crG0BqqIo0-QuldHrWdC3A.png" /><figcaption>Fitting a linear regression model to one-dimensional data</figcaption></figure><p>Then the average of losses over the training set is denoted by</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/337/1*i-hqQUq2r_3wrpbNM4g7Nw.png" /></figure><p>As we said before the goal of the training process is to find w and b that minimize the total loss for all training sample</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/337/1*GjDgeapOHo4bCjDOfwErCQ.png" /></figure><p><strong><em>6- Analytic Solution</em></strong></p><p>We need a method to update the model to improve its quality and unlike most models linear regression is considered an easy optimization problem, we need to apply the following steps:</p><ol><li>Add b to w by appending a column for b in w matrix.</li><li>Then the prediction goal is to minimize ||y-Xw||² &gt;&gt; x must be of full rank meaning no feature is linearly dependent on other features.</li><li>There will be just one critical point on the loss surface and it corresponds to the minimum loss over the entire domain.</li><li>Take the derivative of the loss with respect to w and set it equal to zero</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*m3R1dqKR4HY6N3mcN6qEZw.png" /></figure><p>5. Solving for w gives the optimal solution</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*OYttBldrBnwZ0Gv5juZeKQ.png" /><figcaption>the solution is unique when the matrix X⊤ X is invertible</figcaption></figure><p>Note that the requirement of an analytic solution is so restrictive that it would exclude almost all the exciting aspects of deep learning.</p><p>That’s it I hope u enjoyed it! In the next part, I will explore with you pages 86–92.</p><p><a href="https://mlearning.substack.com/about"><strong>BECOME a WRITER at MLearning.ai</strong></a></p><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=452295c1da32" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can the data split method improve your model performance?]]></title>
            <link>https://nouranalimohammed.medium.com/can-the-data-split-method-improve-your-model-performance-b2571691c61d?source=rss-d192b4414494------2</link>
            <guid isPermaLink="false">https://medium.com/p/b2571691c61d</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[ml-techniques]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Nouran Ali]]></dc:creator>
            <pubDate>Sat, 04 Jun 2022 14:00:00 GMT</pubDate>
            <atom:updated>2022-06-06T06:15:52.758Z</atom:updated>
            <content:encoded><![CDATA[<h3>My first tech story in a long while…</h3><p>Hello again, in this article I want to talk about an interesting yet simple topic, it was a hidden problem in a project I’m working on and all of us missed out on that fact, we spent months trying out models and cleaning data trusting our data loading and splitting method yet we got 0 improvements so what was it??</p><p>Let’s get started by stating some rules</p><ol><li>I can’t state many details about the project scope so I will talk about the primary lines that may fit with other similar problems</li><li>I’m def will NOT be talking about train_test_split -obv-</li></ol><ul><li>Assuming the reader knows enough about ML models and testing them-</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/1*r73p1rxMZWnZLoYi5Odf4A.png" /><figcaption>the process of splitting the data</figcaption></figure><h3>Why do we need to split our data?</h3><p>Usually while conducting experiments while building an ML model we try several model architectures or learning methods so splitting our dataset into train/test/validation parts helps us :</p><ol><li>Choose the best model among many.</li><li>Choose the best model parameters or settings.</li><li>Having a holdout test set helps us measure the real performance aka accuracy by showing the model unseen data and not only judging by the validation accuracy -as it may be deceiving cuz of data leakage-</li></ol><h3>How do we split the data?</h3><p>There are many known methods, the most common method is to choose 70% for training and split the remaining 30% between validation and testing, you can use any tool to do so scikit learn, numpy, pandas or if your dataset contains several folders/files u can do it by hand if they’re not many.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z-LE3kis3SeeLWoiW2EzRw.png" /><figcaption>different split ratios</figcaption></figure><h3>My problem</h3><p>In my case, my training set consisted of ~250GB of raw satellite images plus 25 label files, the data goes through several processing steps till it reaches the final step, having about 25 large CSV files.</p><p>This is a:</p><p><strong>Multi-class pixel classification</strong> problem where also segmentation matters.</p><p><strong>The old method</strong> we used is that we split the 25 CSV files as follows: 13 whole files for training, 6 testing, and 6 validation.</p><p>and we kept training different models and always getting the same bad results even after performing all the techniques that should improve the results, whether augmentation, upsampling, downsampling, data cleaning (manual/automated), or changing the models into more simple/complex ones, changing the optimizers, loss functions and <strong>EVERYTHING.</strong></p><p>Yet the same bad results with about less than 1% improvement.</p><p><strong>Then by chance, </strong>I was doing an experiment on a small set of the data so I used sklearn’s train_test_split method, and<strong> surprise, +90% accuracy for all classes !!!!</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/1*VmM5uZiSJihB9hZfnCzzsQ.png" /><figcaption>a similar example to my problem — image segmentation/pixel classification</figcaption></figure><h3><strong>The thinking process</strong></h3><p>As someone who has been struggling in that project to get even 80% accuracy for all classes, I was shocked! and then I did 3 things that taught me different things about the problem scope.</p><p>1- Retrain all the models using the old method to compare the results.</p><p>2- Combining all the data into one huge CSV file, randomizing it, and split into train/test/validation.</p><p>3- Split each file of the 25 into Train/Test/Validation resulting in 25 files for training, 25 for Testing, and 25 for validation.</p><h3>Results and actions/experiment</h3><p><strong>Method #1:<br>While some thought it was the most logical one, neither the model learning curve nor performance reflected that</strong></p><ol><li>The learning process was very unstable the model NEVER reached a saturation point, The curve never smoothed or stabilized.</li><li>The numbers/results were always very very low.</li><li>The performance in production was terrible even on the data it saw before (used training data as input)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*ctp86mmjQE2U1TVzcK5HlA.png" /><figcaption>model instability</figcaption></figure><p><strong>Method #2:<br>It wasn’t as bad as #1, in fact, the model performed amazingly</strong></p><ol><li>The learning curve was very stable.</li><li>The accuracy was very high +90% for all classes.</li></ol><p><strong>BUT<br></strong>The huge problem was that randomizing pixels made it lose the ability to correctly segment the image, it performed well on single pixels but neighboring similar pixels were not located correctly</p><p>Also in Methods #1 and #2 <strong>the models DIDN’T SEE ALL CLASSES IN EVERY EPOCH.</strong></p><p><strong>Method #3<br>I call it a SUCCESS, it performed amazingly in everything, the models were able to locate segments correctly, learn smoothly, and produce good results, it was a huge surprise.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l_fatNDj9whEyuXlHPwMTg.png" /><figcaption>stable curve</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*f6Uhb8MI4REGcYKkYM9OUQ.png" /><figcaption>example stable output</figcaption></figure><h3>The Takeaway</h3><p>The biggest lesson I learned from all of these experiments is to never underestimate the power of a very simple, repetitive task we do when building every ML model, sometimes the smallest things do matter.</p><p>Another thing is to try and never doubt my guesses, I once guessed that this might be our problem and someone said noway it is but I tried it and it worked.</p><p>Working on ML/DL models can be very tedious sometimes and annoying while you think your problem was Architecture it might be just data, a software bug, or a logical bug while at other times you may think it’s data and it turns out to be model or another bug, there’re no clear rules yet which can make things a bit annoying but it also makes them fun and interesting opening an area to learn something new every day.</p><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b2571691c61d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Different metrics to evaluate classifiers — Explained]]></title>
            <link>https://medium.com/geekculture/different-metrics-to-evaluate-classifiers-explained-838063f015dc?source=rss-d192b4414494------2</link>
            <guid isPermaLink="false">https://medium.com/p/838063f015dc</guid>
            <category><![CDATA[evaluation]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[classification-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Nouran Ali]]></dc:creator>
            <pubDate>Sun, 23 May 2021 23:09:51 GMT</pubDate>
            <atom:updated>2021-06-20T17:56:50.961Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s been a while since I last posted an article so I wanted to get back with a topic I once made a whole session about for my colleagues and from their feedback I felt I should post about it to refresh my info and document it!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/421/1*LwqNzuVBJoJwdz_LlNPhPw.png" /><figcaption>classification decision boundary</figcaption></figure><p>So in this piece, I’ll talk about classifiers and how to evaluate them to make sure that your metric reflects the model&#39;s actual performance.</p><h3><strong>1- What is classification?</strong></h3><p>Classification is a supervised learning technique where the target variable is a discrete value, in other words, the model predicts a label, not a continuous value. For example, the target variable may have values 0 or 1 that may denote cat or dog. Classification is one of the most common ML tasks.</p><p>A classifier is a model that performs classification.</p><h3><strong>2- Imbalanced datasets</strong></h3><p>This is a problem that appears in many datasets where the number or ratio of classes are not equal or nearly equal, class 0 may have 90 instances and class 1 has only 10, this problem can be solved by many ways like resampling, collecting more data or choosing the right metric.</p><h3><strong>3- Classification Metrics</strong></h3><p>To evaluate any ML model you need to set a metric and according to its value on the test or the holdout set, you retrain the model and make adjustments.</p><p>Classification metrics are mainly divided into 3 types</p><p>1- Threshold metrics</p><p>2- Ranking metrics</p><p>3- Probability metrics</p><h3><strong>3.1- Threshold Metrics</strong></h3><p>These are the metrics that quantify the errors of the classifier’s predictions, they are used when we want the model to minimize the number of errors or misclassified items or <strong>when our dataset is balanced. Why?</strong> because your model may be performing poorly and accuracy or error tricks you to think that it’s performing well &gt;&gt; remember our 90 to 10 dataset, the model may achieve 90% accuracy while it’s only working for the majority class</p><p><strong>Accuracy</strong> = correct predictions/Total predictions</p><p><strong>Error</strong> = incorrect predictions/Total predictions</p><p>Error metric is the complement of the accuracy &gt;&gt; 1-accuracy</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/401/1*-eJuISNkV-VFA7Lz_6PtzA.png" /></figure><p><strong>Sensitivity-Specificity Metrics</strong></p><p>These metrics give you an intuition about how well the +ve class was predicted.</p><p><strong>Sensitivity/Recall= </strong>True positive/(True positive+false negative) it’s defined as the true positive rate and it measures how well true positives were predicted</p><p><strong>Specificity</strong> = True negative / (False positive + True negative) it’s the complement of the sensitivity (True -ve rate) it gives an intuition about how well the -ve class was predicted. and for imbalanced classification, it can be more helpful than sensitivity.</p><p><strong>Precision-Recall Metrics</strong></p><p>We knew that the recall is the same as sensitivity,</p><p>Precision = TP/(TP+FP) it’s defined as the accuracy of positive predictions or the proportion of +ve predictions that were actually correct</p><p>Recall and precision are often combined in one metric to find a balance between them this metric is called <strong>F-score.</strong></p><p>F1-Score = (2*precision+recall) / (precision+recall), F1 score gives equal weight for both precision and recall.</p><p>An abstraction to F1 score is F-beta where the balance of precision and recall in the calculation of the <a href="https://machinelearningmastery.com/arithmetic-geometric-and-harmonic-means-for-machine-learning/">harmonic mean</a> is controlled by a coefficient called <em>beta</em>.</p><p><strong>Fbeta-score</strong> = ((1 + beta²) * Precision * Recall) / (beta² * Precision + Recall)</p><p>There are other threshold metrics but these are most popular.</p><p>One last thing about these metrics is that they assume full knowledge of the conditions under which the classifier will be deployed. In particular, they assume that the class imbalance present in the training set is the one that will be encountered throughout the operating life of the classifier. which is not often the case so they can mislead you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*fUnVMuu91SR_Ehrfvne2iA.jpeg" /></figure><h3><strong>3.2 Ranking metrics</strong></h3><p>These metrics don’t make any assumptions about class distributions in the dataset and their concern is to evaluate how effective the classifiers separates data classes and they are only used with classifiers that can output a probability score of instance belonging to class, ie: point X belongs to class A with 0.9 probability.</p><p>The most widely used metric in this is the ROC curve and it analyzes how binary classifiers perform. It plots the Recall/TPR versus the FPR</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*WMyJPSPxSaS3aI5kwRhYaw.png" /><figcaption>TPR vs. FPR</figcaption></figure><p>TO evaluate the model we calculate the AUC (Area under the ROC curve) from point (0,0) to point (1,1)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/346/1*6EfGiFUS0SD4r2ZwUOSwIA.png" /><figcaption>AUC</figcaption></figure><p>AUC ranges from 0 to 1 , having auc=1 means that the model has 100% correct predictions</p><p>Advantages of AUC :</p><p>1- Scale-invariant : It measures how well predictions are ranked, rather than their absolute values.</p><p>2- Classification-threshold-invariant. It measures the quality of the model’s predictions irrespective of what classification threshold is chosen.</p><p>Limitations of AUC :</p><p>1- Scale invariance is not always desirable.</p><p>2- Classification-threshold invariance is not always desirable.</p><p><a href="https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc">Classification: ROC and AUC | Machine Learning | Google for Developers</a></p><h3><strong>3.3 Probabilistic Metrics</strong></h3><p>These metrics quantify the uncercetainty of a classifier’s predicitons</p><p>They are useful for problems where we are less interested in incorrect vs. correct class predictions and more interested in the uncertainty the model has in predictions and penalizing those predictions that are wrong but highly confident.</p><p>These metrics also require the probabilities to be calibrated &gt;&gt;logistic regression, other models that are not trained under a probalistic settings require their probabilities to be calibrated against a dataset prior to being evaluated via a probabilistic metric &gt;&gt; SVM and KNN</p><p>The most common metric is log loss for binary classification or known as cross-entropy</p><p>for binary classifiers:</p><ul><li><strong>LogLoss</strong> = -((1 — y) * log(1 — yhat) + y * log(yhat)) where yhat is the prediction of the model</li></ul><p>for generalization</p><ul><li><strong>LogLoss</strong> = -( sum c in C y_c * log(yhat_c)) where c is classes</li></ul><p>A perfect classifier has a log loss of 0.0, with worse values being positive up to infinity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/489/1*gMh5lLuzobFxvGFYCAedsw.jpeg" /><figcaption>log loss decreases when prob. increases</figcaption></figure><h3><strong>4- How to choose a metric?</strong></h3><p>Choosing a metric depends on the dataset, the problem, your assumptions and business requirements, so there’s no metric that fits all. Especially if you have class imbalance, there’s a great mindmap you can start with here from Machine learning mastery that summarizes things a bit</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/943/1*v_ob3BcaS_2s0KkrbVa0pg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=838063f015dc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/different-metrics-to-evaluate-classifiers-explained-838063f015dc">Different metrics to evaluate classifiers — Explained</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gender from Voice?]]></title>
            <link>https://medium.com/codex/gender-from-voice-b12c04a1a836?source=rss-d192b4414494------2</link>
            <guid isPermaLink="false">https://medium.com/p/b12c04a1a836</guid>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[visualization]]></category>
            <dc:creator><![CDATA[Nouran Ali]]></dc:creator>
            <pubDate>Tue, 06 Apr 2021 10:50:45 GMT</pubDate>
            <atom:updated>2021-04-11T11:32:41.160Z</atom:updated>
            <content:encoded><![CDATA[<h3>Detecting a person’s gender from voice features.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4HeezBk-JgiE1pFo" /><figcaption>Photo by <a href="https://unsplash.com/@dainisgraveris?utm_source=medium&amp;utm_medium=referral">Dainis Graveris</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>A new tutorial on building stories with data analysis and visualization using the <a href="https://www.kaggle.com/primaryobjects/voicegender">Gender Recognition by Voice</a> dataset from Kaggle.</p><p>By now you already know that I like asking questions to draw stories when performing analysis, I find this a really fun thing that people who don’t like to work with data a lot can do to and it also gives me more sense of the data and better ideas for feature engineering.</p><p>This was a task from my training with SureStart that I really enjoyed so I tried to do as much EDA as I could. In this article, I’ll walk you through some of the things that I did and for more you can always check the original notebook.</p><p>As usual, we’ll import the important libraries</p><pre>import numpy as np <br>import pandas as pd <br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>import warnings<br>from sklearn.linear_model import LogisticRegression<br>from sklearn.svm import LinearSVC<br>from sklearn.svm import SVC<br>from sklearn.neighbors import KNeighborsClassifier<br>from sklearn.ensemble import RandomForestClassifier<br>from sklearn.tree import DecisionTreeClassifier<br>from sklearn.ensemble import GradientBoostingClassifier<br>from sklearn.naive_bayes import GaussianNB<br><em>#model selection</em><br>from sklearn.model_selection import train_test_split<br>from sklearn.metrics import  accuracy_score,precision_score,recall_score,confusion_matrix,roc_curve,roc_auc_score<br>from sklearn.model_selection import GridSearchCV<br><em>#preprocess.</em><br>from sklearn.preprocessing import MinMaxScaler,StandardScaler,LabelEncoder,OneHotEncoder<br>from sklearn.impute import SimpleImputer<br>warnings.filterwarnings(&#39;ignore&#39;)</pre><p>Reading our dataset into a pandas data frame and display the shape</p><pre>df_train= pd.read_csv(&#39;/kaggle/input/voicegender/voice.csv&#39;)<br>df_train.shape</pre><h3>EDA</h3><p>Our dataset contains 21 columns and 3168 instances or rows, let’s check column names</p><pre>df_train.columns</pre><p>It’s a list of 21 column names, [‘meanfreq’, ‘sd’, ‘median’, ‘Q25’, ‘Q75’, ‘IQR’, ‘skew’, ‘kurt’, ‘sp.ent’, ‘sfm’, ‘mode’, ‘centroid’, ‘meanfun’, ‘minfun’, ‘maxfun’,<br> ‘meandom’, ‘mindom’, ‘maxdom’, ‘dfrange’, ‘modindx’, ‘label’]</p><p>From the <a href="https://www.kaggle.com/primaryobjects/voicegender">description</a> page, we know what each column corresponds to.</p><p>I then checked nulls and found none.</p><p>I created 2 functions that can ease my work so that I don’t repeat code, The first one is to check for outliers in a column (outliers are values that don’t fit in the distribution by being too large or too small and cause noise) I used the 1.5 IQR rule, you can check this article for better demonstration</p><p><a href="https://towardsdatascience.com/why-1-5-in-iqr-method-of-outlier-detection-5d07fdc82097">Why “1.5” in IQR Method of Outlier Detection?</a></p><pre>def check_outliers(col):<br>    q1,q3=df[col].quantile([0.25,0.75])<br>    iqr=q3-q1<br>    rang=1.5*iqr<br>    return(q1-rang,q3+rang)</pre><p>The other function is for creating a box plot and a distribution plot for a given column.</p><pre>def plot(col):<br>    fig,axes=plt.subplots(1,2)<br>    sns.boxplot(data=df,x=col,ax=axes[0])<br>    sns.distplot(a=df[col],ax=axes[1],color=&#39;#ff4125&#39;)<br>    fig.set_size_inches(15,5)<br>    lower,upper = check_outliers(col)<br>    l=[df[col] for i <strong>in</strong> df[col] if i&gt;lower <strong>and</strong> i&lt;upper] <br>    print(&quot;Number of data points remaining if outliers removed : &quot;,len(l))</pre><p><strong>Univariate analysis</strong></p><p>here I’m analyzing individual columns, I chose a box plot and a dist plot as all features are numeric.</p><pre>plot(&#39;meanfreq&#39;)<br>#Number of data points remaining if outliers removed :  3104</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/659/1*pvhM3eN5WDwzcvTmSSq3JQ.png" /></figure><ol><li>From the boxplot, we can see that we have some outliers less than approx 0.11 mean frequency, and also most outliers are to the left of the distribution.</li><li>From the distribution plot, we know that it’s a normal distribution with a little -ve skewness that we can normalize later.</li></ol><p>Then I’ll draw other interesting columns</p><pre>plot(&#39;sd&#39;)<br>#Number of data points remaining if outliers removed :</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*JEoJSMIByZ8xiBVnbz__ZA.png" /></figure><pre>plot(&#39;median&#39;)<br>#Number of data points remaining if outliers removed :  3059</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*OlEW3TytYpzfA0e2s7o3ow.png" /></figure><pre>plot(&#39;Q25&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*TPNmFNpnkPZvJp6GJ8L1ig.png" /></figure><p>For more plots, you can check the original notebook that I’ll share with you, but mainly most columns are not a perfect normal distribution which is pretty predictable and some have outliers which is also very normal as all instances can never fit in the bell curve especially in real-world data.</p><p>Also, one thing to note is that our data is balanced meaning that both classes contain an equal or semi-equal number of instances which is often not the case for real-world datasets.</p><pre>sns.countplot(data=df,x=&#39;label&#39;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*77Gid5qDjmfgcsXRlw4KoQ.png" /></figure><p><strong>Bivariate analysis</strong></p><p>Here we compare two columns against each other, first for easier prediction later I encoded the labels.</p><pre>df[&#39;label&#39;]=df[&#39;label&#39;].replace({&#39;male&#39;:1,&#39;female&#39;:0})</pre><p>I started by creating a heatmap to show the correlation between each attribute and the others.</p><pre><em># correlation heatmap </em><br>cor_mat= df[:].corr()<br>mask = np.array(cor_mat)<br>mask[np.tril_indices_from(mask)] = False<br>fig=plt.gcf()<br>fig.set_size_inches(30,12)<br>sns.heatmap(data=cor_mat,mask=mask,square=True,annot=True,cbar=True)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*aAQq17W3GDVkKlVlP4o_kA.png" /></figure><ol><li>IQR is the most correlated feature with the target label.</li><li>median and centroid, mean freq and centroid &gt;&gt; delete centroid as highly correlated features are often considered redundant</li><li>other observations, there are weak correlations and high ones, I’ll delete some columns in feature engineering.</li></ol><p>To plot some columns against the target, I created a new function</p><pre>def plot_against_target(feature):<br>    sns.factorplot(data=df,y=feature,x=&#39;label&#39;,kind=&#39;box&#39;)<br>    fig=plt.gcf()<br>    fig.set_size_inches(7,7)</pre><pre>plot_against_target(&#39;meanfreq&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/665/1*KZ5OuT-qTzbDzPUhUi7n2Q.png" /></figure><p>We can conclude that females have higher voice frequency compared to males which is a <strong>well-known</strong> fact!</p><p>I created other similar plots to compare both classes’ values and they’re very different in almost all attributes. The last plot I made here was another correlation plot which is a Scatter plot for some columns against each other that I’ll use later.</p><pre>g=sns.PairGrid(df[[&#39;meanfreq&#39;,&#39;sd&#39;,&#39;median&#39;,&#39;Q25&#39;,&#39;IQR&#39;,&#39;sp.ent&#39;,&#39;sfm&#39;,&#39;meanfun&#39;,&#39;label&#39;]], hue = &quot;label&quot;)<br>g = g.map(plt.scatter).add_legend()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*_6Fw9cLKKKwpxPaSvfwp5w.png" /></figure><p><strong>Feature Engineering</strong></p><p>1- removing outliers ^^</p><pre>for col <strong>in</strong> df.columns:<br>    l,u=check_outliers(col)<br>    df=df[(df[col]&gt;l)&amp;(df[col]&lt;u)]</pre><p>after removing outliers we lost nearly 50% of the dataset so I reversed this part and left them.</p><p>2- According to plots, we’ll delete some columns “skew,Kurt,mindom,maxdom,centroid” as they’re highly correlated with others.</p><pre>temp_df=df.copy()<br>temp_df.drop([&#39;skew&#39;,&#39;kurt&#39;,&#39;mindom&#39;,&#39;maxdom&#39;,&#39;centroid&#39;],axis=1,inplace=True)</pre><p>3- Create new Features</p><p>Combining already existing features into a stronger one is a common practice in FE. I created one new feature called pear_skew as follows</p><pre>temp_df[&#39;pear_skew&#39;]=temp_df[&#39;meanfreq&#39;]-temp_df[&#39;mode&#39;]<br>temp_df[&#39;pear_skew&#39;]=temp_df[&#39;pear_skew&#39;]/temp_df[&#39;sd&#39;]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/343/1*PkCgAucTaSLtqc0grbRkQg.png" /></figure><p>Then I changed the median column to be 1/3 (2*mean + mode) which is the <strong>empirical relationship</strong>.</p><pre>temp_df[&#39;meanfreq&#39;]=temp_df[&#39;meanfreq&#39;].apply(lambda x:x*2)<br>temp_df[&#39;median&#39;]=temp_df[&#39;meanfreq&#39;]+temp_df[&#39;mode&#39;]<br>temp_df[&#39;median&#39;]=temp_df[&#39;median&#39;].apply(lambda x:x/3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/357/1*dUTsnokblgvoo8LiwfgruQ.png" /></figure><p>4- Scaling</p><p>I used Sklearn’s StandardScaler to normalize the data, it works by subtracting mean and dividing by standard deviation.</p><pre>scaler=StandardScaler()<br>scaled_df=scaler.fit_transform(temp_df.drop(&#39;label&#39;,axis=1))</pre><p><strong>ML modeling</strong></p><p>Finally the easy part! creating ML models, I used the default parameters of the models.</p><p>1- Split the target from features and create train/test splits</p><pre>X=scaled_df<br>Y=df[&#39;label&#39;].values<br>x_train,x_test,y_train,y_test=train_test_split(X,Y,test_size=0.20,random_state=42)</pre><p>2- I created 8 different models to compare the results</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/252/1*mqLtL2IxPR12bevHIXF3uQ.png" /></figure><p>In conclusion, this work was about analysis more than ML modeling, you can check the original Notebook <a href="https://www.kaggle.com/nouranali/voice-gender-recognition-with-ml">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b12c04a1a836" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codex/gender-from-voice-b12c04a1a836">Gender from Voice?</a> was originally published in <a href="https://medium.com/codex">CodeX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using data analysis to draw the winning PUBG scenario]]></title>
            <link>https://medium.com/codex/using-data-analysis-to-draw-the-winning-pubg-scenario-1ed8df30e4ab?source=rss-d192b4414494------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ed8df30e4ab</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <dc:creator><![CDATA[Nouran Ali]]></dc:creator>
            <pubDate>Sun, 07 Mar 2021 20:22:06 GMT</pubDate>
            <atom:updated>2021-03-30T02:56:16.598Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tpmzxiXH0w34J4oN" /><figcaption>Photo by <a href="https://unsplash.com/@clintbustrillos?utm_source=medium&amp;utm_medium=referral">Clint Bustrillos</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4><a href="http://medium.com/codex">CODEX</a></h4><p>So this article will be a tutorial on how to make your data tell a story applied to PUBG mobile dataset, I love PUBG and play it every day with my nieces who actually taught me how to play! And when I found this dataset on <a href="https://www.kaggle.com/c/pubg-finish-placement-prediction"><strong>Kaggle</strong></a>, I was so happy because I had the domain knowledge that’d allow me to guess and check if these guesses are supported by the data provided.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oMtPPqMCBf_6wDU6dfz9AA.jpeg" /></figure><p>Let’s get started right away and build some hypotheses and check if they’re supported by data! If you’re not familiar with PUBG I suggest you check the data info page at <a href="https://www.kaggle.com/c/pubg-finish-placement-prediction/data">Kaggle</a></p><p>As usual, the first step is to import the libraries you’ll be using,</p><pre>import numpy as np <br>import pandas as pd<br>import os<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>%matplotlib inline</pre><p>Then we’ll read the data using the pandas read_csv function, the data is in a CSV file and display data shape</p><pre>train = pd.read_csv(&#39;/kaggle/input/pubg-finish-placement-prediction/train_V2.csv&#39;)<br>train.shape</pre><p>The dataset is 4446966 rows with 29 columns, then I’ll display the dataset head, which is the first 5 rows, all the 29 columns didn’t fit so I split them ;)</p><pre>train.iloc[:,:15].head()<br>train.iloc[:,14:].head()</pre><p>I then went ahead to check the datatypes</p><pre>train.info()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/433/1*h-5o44GU8Q9dxbO2FdKQMA.png" /></figure><p>We can see that the data types are not in the wrong format (no strings represented as integers or integers as characters) we can also see that the data frame takes a little less than 1GB of memory which is expected as the data contains more than 4 million observation.</p><p>The next thing is to check the missing values, luckily there was only 1 so I dropped it.</p><pre>train.isnull().sum()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/214/1*5vEhTUizTBgLy_KZEuD6Ng.png" /></figure><pre>#drop null values<br>train.dropna(inplace=True)</pre><p>then I went ahead to check a summary statistics about each numerical column to preview some insights.</p><pre>train.describe().T</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*ISGexQel6Pn5FolW3iIWCA.png" /></figure><p>We observe that some columns have outliers so later we need to decide how we’ll deal with them( drop, normalize, or leave them as they can contain important information)</p><p>In this tutorial, I’ll focus on EDA rather than ML modeling.</p><h3><strong>EDA</strong></h3><p>I’ll divide my analysis into a hierarchy depending on players activities and winning.</p><ol><li><strong>Killer Players</strong></li></ol><p>the 99th quantile of the kills &gt;&gt; 99% of values fall behind this value.</p><pre>train[&#39;kills&#39;].quantile(0.99)<br>#it&#39;s equal to 7</pre><pre>temp= train.copy()<br>temp.loc[temp[&#39;kills&#39;] &gt; temp[&#39;kills&#39;].quantile(0.99)] = &#39;8+&#39;<br>plt.figure(figsize=(15,10))<br>sns.countplot(temp[&#39;kills&#39;].astype(&#39;str&#39;).sort_values())<br>plt.title(&#39;No. of Kills&#39;);temp= train.copy()<br>temp.loc[temp[&#39;kills&#39;] &gt; temp[&#39;kills&#39;].quantile(0.99)] = &#39;8+&#39;<br>plt.figure(figsize=(15,10))<br>sns.countplot(temp[&#39;kills&#39;].astype(&#39;str&#39;).sort_values())<br>plt.title(&#39;No. of Kills&#39;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*_SAXf5uIm9HqjlbQOdglhQ.png" /></figure><p>This is a count plot, I replaced all kills above the 99th percentile with +8, from the plot we see that most players don’t kill any of their opponents,</p><p>Do they damage them at least?</p><pre>temp= train.copy()<br>temp =temp[temp[&#39;kills&#39;]==0]<br>plt.figure(figsize=(15,10))<br>sns.distplot(temp[&#39;damageDealt&#39;])<br>plt.title(&#39;Damage dealt by non killers&#39;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/526/1*TQAKh4ACSBOWzwyTMPoYUQ.png" /></figure><p>Most non-killer players don’t deal damage to their enemies too, Moving on to the next question what team size occurs most Squads, Duos, or Solos?</p><pre>plt.figure(figsize=(15,10))<br>plt.xticks(rotation=45)<br>sns.countplot(train[&#39;matchType&#39;].astype(&#39;str&#39;));</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*9z3x0udh1RwdGvDNIAJUiw.png" /></figure><p>FPP and TPP are playing modes that denote player view of himself during the game, usually TPP is easier. We observe from the plot that Squads (4 players) that play in FPP are more than other match types.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*nOLwzIU3TF3lAAD031yXkg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*BT2SaNlY9EYb5sXR8TjNVw.png" /><figcaption>TPP vs FPP mode</figcaption></figure><p>Then I combined all squad, solos, duos types in one column for easier analysis.</p><pre>train.matchType.replace([&#39;squad-fpp&#39;,&#39;squad&#39;,&#39;normal-squad-fpp&#39;,&#39;normal-squad&#39;],&#39;Squad&#39;,inplace=True)<br>train.matchType.replace([&#39;duo-fpp&#39;,&#39;duo&#39;,&#39;normal-duo-fpp&#39;,&#39;normal-duo&#39;],&#39;Duo&#39;,inplace=True)<br>train.matchType.replace([&#39;solo-fpp&#39;,&#39;solo&#39;,&#39;normal-solo-fpp&#39;,&#39;normal-solo&#39;],&#39;Solo&#39;,inplace=True)<br>train.matchType.replace([&#39;crashfpp&#39;,&#39;flaretpp&#39;,&#39;flarefpp&#39;,&#39;crashtpp&#39;],&#39;Othertypes&#39;,inplace=True)<br>#display countplot of new columns values<br>sns.countplot(train.matchType);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/449/1*jcwTGSoUDTaZfeWpOFZ4tw.png" /></figure><p>We know from this that the majority of players are Squads! exactly <strong>“53.978%”</strong></p><p><strong>2. Movement type.</strong></p><p>A PUBG player travels about 1154.218118648076 meters, let’s check the walking distance and how winning and walking may correlate, as a PUBG player I think that walking gives me the ability to hide better than riding a vehicle or swimming, is it true?</p><pre>plt.figure(figsize=(15,10))<br>sns.scatterplot(x=&#39;winPlacePerc&#39;,y=&#39;walkDistance&#39;,data=train)<br>plt.title(&#39;The relationship between winning and running&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/1*qitvXlMJbOdz86_-kq5HAA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/317/1*ozwaYDO0iJdCwP5Gx3Sbag.png" /></figure><p>looks like walking and winning have a +ve correlation, that says that my assumption was kinda true!</p><p>Let’s check vehicles..</p><pre>plt.figure(figsize=(15,10))<br>sns.scatterplot(x=&#39;winPlacePerc&#39;,y=&#39;rideDistance&#39;,data=train)<br>plt.title(&#39;The relationship between winning and driving&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/762/1*aZgcpYEZoFMvEHgTPAvuVA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/309/1*HHthokucNlVohWlCN-LDSQ.png" /></figure><p>So vehicles and winning are less correlated, but there’s a pubg trick that u can kill another player if you shoot at his car or throw a bomb at it to explode. let’s check that.</p><pre>f,ax1 = plt.subplots(figsize =(20,10))<br>sns.pointplot(x=&#39;vehicleDestroys&#39;,y=&#39;winPlacePerc&#39;,data=train,color=&#39;#606060&#39;,alpha=0.8)<br>plt.xlabel(&#39;Number of Vehicle Destroys&#39;,fontsize = 15,color=&#39;blue&#39;)<br>plt.ylabel(&#39;Win Percentage&#39;,fontsize = 15,color=&#39;blue&#39;)<br>plt.title(&#39;Vehicle Destroys/ Win Ratio&#39;,fontsize = 20,color=&#39;blue&#39;)<br>plt.grid();</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/677/1*Uj8IwoOCHYhpTgjhYTFSbA.png" /></figure><p>The point plot shows that destroying at least one vehicle increases the chance of winning by ~35%, AWESOME!!</p><p>PUBG is a team-based game, when a member of your team is knocked down, you can revive him and bring him back in the game as long as he’s not dead..let’s check if that affects the winning.</p><pre>f,ax1 = plt.subplots(figsize =(20,10))<br>sns.pointplot(x=&#39;revives&#39;,y=&#39;winPlacePerc&#39;,data=train,alpha=0.8)<br>plt.xlabel(&#39;Number of Revives&#39;,fontsize = 15,color=&#39;blue&#39;)<br>plt.ylabel(&#39;Win Percentage&#39;,fontsize = 15,color=&#39;blue&#39;)<br>plt.title(&#39;Revives/ Win Ratio&#39;,fontsize = 20,color=&#39;blue&#39;)<br>plt.grid();</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/671/1*MKo8dyc606U2LVZvS1wmnA.png" /></figure><p>I’m not sure about the plot, I think it’s not determined,</p><p><strong>3. Boosts and healing elements</strong></p><p>In PUBG you can take health boosts or healing elements to help if you’re shot or injured, let’s check if they can make you win!</p><pre>plt.figure(figsize=(15,10))<br>sns.scatterplot(x=&#39;winPlacePerc&#39;,y=&#39;heals&#39;,data=train)<br>plt.title(&#39;The relationship between winning and healing elements&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*a2YXfadewR9u04yMmKU_0w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/315/1*YUszPeeQfxAwok2tw-isCw.png" /></figure><p>So healing elements are ~0.43 correlated with winning</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/313/1*So3SqBs2M9J4-dwrdRN2Ow.png" /></figure><p>And not a surprise, health boosters correlate by 0.634 with winning! which is expected.</p><h3><strong>Feature Engineering</strong></h3><p>Now I’ll drop some features, and create new ones out of existing features.</p><p>plotting a heatmap to show all features and their correlation with the target was not very successful as the plot was very crowded, so I plotted the top 5 attributes.</p><pre>f,ax = plt.subplots(figsize=(11, 11))<br>cols = train.corr().nlargest(5, &#39;winPlacePerc&#39;)[&#39;winPlacePerc&#39;].index<br>cm = np.corrcoef(train[cols].values.T)<br>sns.set(font_scale=1.25)<br>hm = sns.heatmap(cm, cbar=True, annot=True, square=True, fmt=&#39;.2f&#39;, annot_kws={&#39;size&#39;: 10}, yticklabels=cols.values, xticklabels=cols.values)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*GvAZV12dsUWZVDik-7HjoQ.png" /></figure><p>We notice that player’s weapons affect winning, which is true some weapons in PUBG are more powerful than others (I love M416 and Groza)!</p><p>let’s create a feature that indicates the number of players in a PUBG match,</p><pre>train[&#39;playersJoined&#39;]=train.groupby(&#39;matchId&#39;[&#39;matchId&#39;].transform(&#39;count&#39;)</pre><p>In EDA we knew that bosts and healing elements can increase the chance of winning a game so let’s put them in one feature also the walking, swimming, and riding distance.</p><pre>train[&#39;healsAndBoosts&#39;] = train[&#39;heals&#39;]+train[&#39;boosts&#39;]<br>train[&#39;totalDistance&#39;] = train[&#39;walkDistance&#39;]+train[&#39;rideDistance&#39;]+train[&#39;swimDistance&#39;]</pre><p>We can also substitute solos, duos, squads with a number of players in the team.</p><pre>train[&#39;team&#39;] = [1 if i&gt;50 else 2 if (i&gt;25 &amp; i&lt;=50) else 4 for i <strong>in</strong> train[&#39;numGroups&#39;]]</pre><p>Then we can drop unused columns,</p><p>Some columns had skewness in their values &gt;&gt; distribution is not normal</p><pre>sns.distplot(X[&#39;damageDealt&#39;]);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*9qIZyIw58kKZg5PJjT2pjw.png" /></figure><pre>sns.distplot(X[&#39;totalDistance&#39;]);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*4BLddsxnsNUweXD2RJmjiA.png" /></figure><p>I used cube root transformation to keep 0 values, they’re both +ve skewed</p><pre>X[&#39;damageDealt&#39;]=X[&#39;damageDealt&#39;]**(1/3)<br>sns.distplot(X[&#39;damageDealt&#39;]);</pre><pre>X[&#39;damageDealt&#39;]=X[&#39;damageDealt&#39;]**(1/3)<br>sns.distplot(X[&#39;damageDealt&#39;]);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*kHUGeOOaMmeSKEOvMoZa4Q.png" /></figure><pre>X[&#39;totalDistance&#39;]=X[&#39;totalDistance&#39;]**(1/3)<br>sns.distplot(X[&#39;totalDistance&#39;]);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*VTxLC507Qk_T4RwOpwrdCg.png" /></figure><p>So from my point of view and analysis, to increase your chances of winning a PUBG game you should consider.</p><ol><li>Play in a team (Squad).</li><li>Use healings and health-boosting elements.</li><li>Destroy your enemies vehicles.</li><li>Kill as many enemies as you can.</li><li>Move and walk a lot and collect powerful weapons.</li><li>You should also hide a lot.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yF3TJR1rpFBoTPBMjWy-zQ.jpeg" /></figure><p>you can find the full notebook <a href="https://www.kaggle.com/nouranali/pubg-winning-scenario">here</a> for more details.</p><p>That’s if for now, comment below with your best PUBG strategies 👇</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ed8df30e4ab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codex/using-data-analysis-to-draw-the-winning-pubg-scenario-1ed8df30e4ab">Using data analysis to draw the winning PUBG scenario</a> was originally published in <a href="https://medium.com/codex">CodeX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating Datasets for Machine Learning. My Own Lessons and Experience.]]></title>
            <link>https://medium.com/swlh/creating-datasets-for-machine-learning-my-own-lessons-and-experience-44e118b15d6a?source=rss-d192b4414494------2</link>
            <guid isPermaLink="false">https://medium.com/p/44e118b15d6a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[skills]]></category>
            <category><![CDATA[self-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Nouran Ali]]></dc:creator>
            <pubDate>Sat, 13 Feb 2021 14:49:28 GMT</pubDate>
            <atom:updated>2021-02-17T12:57:58.415Z</atom:updated>
            <content:encoded><![CDATA[<p>Today I wanted to be more on the technical side in the core of Machine learning and data science which is <strong>Data</strong>.</p><p>Without data, Machine learning, data science, and all these fields wouldn’t exist, in fact, these fields are popular now because we have huge amounts of data being generated daily, a fun fact is for some use cases there’s still no enough data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*ZmFkM9eejH4F7MhtkhrpEw.png" /></figure><p><strong>So what is a dataset?</strong></p><blockquote>A data set is a collection of data, which can be in the form of a table consisting of columns and rows where columns are features and rows are observations, Image datasets can be in the form of file structures stored on disks, they can be stored in dedicated databases, Text datasets can have multiple forms CSV, excel, text files or also in databases.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*yEkXZqLYGqJ8VRXOtMhYVQ.jpeg" /><figcaption>The famous IMAGENET dataset which contains 14,197,122 image</figcaption></figure><p><strong>The need for datasets</strong></p><p>Without data, there’s no machine learning nor analytics. We need data to gain business insights, understand our problem, make hypothesis, guess what model architectures to use, and train our ML system.</p><p><strong>Diving into our subject, as a data scientist why do I need to learn how to create my own dataset?</strong></p><p>I’m speaking here from the point of a pure personal experience and I’m not a working professional, yet.</p><p><strong><em>1- Simply the problem you’re solving or the business may not have an available ready to use dataset that you’d just download and get to work.</em></strong></p><p>When working on a project and before getting excited about the idea make sure that you have access to training data or at least know how will you create a data set. For me sometimes I just LOVE to get excited about projects where I’ll create my own data set.</p><p>This faced me during working on my graduation project, the problem we’re tackling is specific to Delta Egypt, there were no ready to use data sets, so we went ahead to read research papers on how to create a dataset with the specifications required in an automated manner ;)</p><p><strong><em>2- It’s a very useful skill for every data scientist or ML engineer and not limited to data engineers.</em></strong></p><p>Learning these skills will boost your skills and resume and will also help you with personal projects or if you’re working with a small team.</p><p><strong><em>3- It’s fun -at least I think so-</em></strong></p><p>I had much fun creating a data set for images and label it on my own, a music features dataset, and a music genres dataset. I learned each time and gained domain knowledge every time as well as learning to use new tools and dealing with different APIs.</p><p><strong>Formulating the problem</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oDwQYaG-RiVsrKOMAdyq_w.jpeg" /></figure><p>As I said, before getting too excited about the project’s flashy idea, pose these questions. These 3 questions are everything and are <strong>EQUALLY IMPORTANT</strong> once answered everything else is easy.</p><p><strong><em>1- What is the solution I’m building?</em></strong></p><p>Am I building a chatbot, a web app, a computer vision app, or what?</p><p><strong><em>2- Where can I get training data?</em></strong></p><p>Is it available open-source, will I pay for it, is it stored in a database and need preprocessing, will I collect it myself, will I collect and label it? <strong><em>Know everything.</em></strong></p><p><strong><em>3- How much domain knowledge I have?</em></strong></p><p>Building a medical diagnosis system? What do I know about medical diagnosis or will I contact a domain expert “in this case a doctor”, Building a sign language translation system? Do I even know sign language? and so on.</p><p><strong>Invest in this stage as much as you can</strong> so that you don’t lose time and resources. Try to document as many insights as you can to know exactly what you’re searching for.</p><h3><strong>My advice for you is</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*nz7BMKNyeUiqzo41BeTyWw.png" /></figure><p><strong>1- If you’re new to ML/data science</strong>, use data sets from Kaggle or any other resource to learn more about data distributions and hierarchies and so on, don’t just hop into this as it’s kinda advanced and needs more experience and sense of the problem, the solution, the available products and reading research papers or using other tools like Apache stuff, AWS, SQL.</p><p><strong>2- If your problem can be solved without this hazard </strong>just download the data set and don’t waste your time.</p><p><strong>3- Ask an expert when you get stuck,</strong> actually this is the most important thing to do, experience cuts down more than 90% of the time you could spend just searching and being lost -also a personal experience-</p><p><strong>4- Don’t be grumpy about it</strong>, ML is not about building models with a few lines of Keras, it’s about learning from DATA so if your team decided to go with that option and you’re against it either don’t complain about how hard and time consuming it is or just do something else.</p><p><strong>5- SHARE what you learned and how did you create it, </strong>oh my god this would’ve saved me months of searching, the research community is not that kind tbh I read nearly +30 research papers just to know how they created a change detection dataset and no one bothered to illustrate or share the code, until I encountered someone who published a paper for his own dataset and guess what he wrote only 4 lines on how he did it, luckily I understood those 4 lines.</p><p><strong>6- Finally have fun and focus on the end goal </strong>which is expanding your skillset. It’s all about growing and creating cool apps that solve human problems, isn’t it?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/260/1*Nzmp0eHzvCGcSpCyEjRCuA.gif" /><figcaption>happy learning!</figcaption></figure><p>Thank you, please feel free to share your thoughts with me and to request any topics fill in this <a href="https://forms.gle/hGNgd2ozatU34oiTA">form</a> ❤</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=44e118b15d6a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/creating-datasets-for-machine-learning-my-own-lessons-and-experience-44e118b15d6a">Creating Datasets for Machine Learning. My Own Lessons and Experience.</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What no one wi tell you about self-learning -in tech-]]></title>
            <link>https://nouranalimohammed.medium.com/what-no-one-wi-tell-you-about-self-learning-in-tech-eb255dfce066?source=rss-d192b4414494------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb255dfce066</guid>
            <category><![CDATA[time-management]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[selflearningai]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[selfcare-tips]]></category>
            <dc:creator><![CDATA[Nouran Ali]]></dc:creator>
            <pubDate>Sun, 07 Feb 2021 18:58:35 GMT</pubDate>
            <atom:updated>2021-02-09T09:35:02.539Z</atom:updated>
            <content:encoded><![CDATA[<h3>What no one will tell you about self-learning -in tech-</h3><p>Hello everyone in my first ever medium article, It’s not my first time to blog but it’s the first time to blog with my name ;), I wanted the first article to be a kick-off, not technically heavy, just a chat.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/1*DLB0-QTHctQUNBIMtj7O8Q.gif" /></figure><p><strong>First I want to introduce myself as well as my background,</strong></p><p>I’m a senior computer engineering student who’s currently into Machine learning engineering and data science, I like writing, learning, music and volunteering but my LOVE is to know everything about everything, I also like cultures and thinking about human behaviors, opinions, and actions.</p><p>When I was in the 9th grade I was first introduced to programming and it felt awesome, that power to create and translate your ideas so that a machine can understand was pretty impressive to me, that’s when I knew what I wanted to be. Sadly life didn’t go as planned I didn’t become a 14-year-old nerd, but 3 years later I managed to make it to engineering school and start the actual journey….</p><p>In college, I was first introduced to self-learning, MOOCS, programming competitions, and all these cool things and honestly, I felt like everyone knew everything about everything -Later I was proven wrong-</p><p><strong>But no one really told me that:</strong></p><p><strong>1-</strong> <strong>It’s ok to feel lost</strong>, we all feel that 99% of the time, you question urself and ur skills, and you don’t think or feel that this is the right way to do things. But it’s totally normal.</p><p><strong>2-</strong> <strong>CUT-OUT people who make you feel bad about urself </strong>or who make you feel like you’re not good enough, oh how I wish someone had told me that earlier, I was friends with the wrong people, it happens but what I regret is that letting them get into my head and believe that I wasn’t good or skilled enough.</p><p><strong>3- There’s always time to learn </strong>and surprisingly you’ll always be learning so stop feeling like everyone is 1000 miles ahead of you ’cause this is gonna drain your energy instead of pushing you.</p><p><strong>4- Remember to have fun and breathe</strong>, we sometimes bury ourselves in studying and school work without stopping to have a breath and just rest, this makes us wear out even faster, <strong>just</strong> <strong>breathe</strong> you’ll gonna make it before the deadline.</p><ul><li><strong>One of the major things that helped me balance</strong> is splitting my goals into long term and short term goals, this is helping me take control of my time especially when I have many projects and exams.</li><li>Also having a to-do list for the day is great to help you estimate the time needed to do ur tasks or study.</li></ul><p><strong>5- Volunteering and sharing</strong> what you know with everyone will make you a better person, hiding your resources won’t make you the best, the more you give the more you’ll gain, I intend to talk about this in detail in another blog.</p><p><strong>Finally</strong>, it’s ok to feel anxious just don’t drown in that feeling, Be kind we are all trying, You’re doing enough.</p><p>And don’t forget to start your day with a good cup of coffee, it changes everything ❤</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gzQ5xdJ62650KPlOomm2cA.jpeg" /></figure><p>Thank you ❤</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb255dfce066" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>