<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Himanshu S on Medium]]></title>
        <description><![CDATA[Stories by Himanshu S on Medium]]></description>
        <link>https://medium.com/@himanshuxd?source=rss-8f8658588de9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*tVx7ej2GRElh1IDFjaWMCw.jpeg</url>
            <title>Stories by Himanshu S on Medium</title>
            <link>https://medium.com/@himanshuxd?source=rss-8f8658588de9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 12:11:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@himanshuxd/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Guide to Rigorous Descriptive Statistics for Machine Learning and Data Science :]]></title>
            <link>https://himanshuxd.medium.com/the-guide-to-rigorous-descriptive-statistics-for-machine-learning-and-data-science-9209f88e4363?source=rss-8f8658588de9------2</link>
            <guid isPermaLink="false">https://medium.com/p/9209f88e4363</guid>
            <category><![CDATA[guides-and-tutorials]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Himanshu S]]></dc:creator>
            <pubDate>Sat, 09 Feb 2019 10:04:00 GMT</pubDate>
            <atom:updated>2021-08-11T05:55:47.149Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Guide to Rigorous Descriptive Statistics : For Machine Learning and Data Science</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fJj6UC9ZmAhsaDmCJlSI7Q.jpeg" /></figure><p>Components of Uni-variate Analysis, Bi-variate Analysis and Multi-variate Analysis alongside Kurtosis, Skewness, Correlation explained mathematically and visually in a simple way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*GkjmpGnnljpSi7JFZP6Fcg.gif" /><figcaption>[1] Inferential Vs Descriptive Statistics.</figcaption></figure><p>A simple summary of what separates <strong>Inferential</strong> and <strong>Descriptive</strong> Statistics according to <strong><em>Wikipedia</em></strong> &gt;</p><blockquote>A <strong>descriptive statistic</strong> (in the <a href="https://en.wikipedia.org/wiki/Count_noun">count noun</a> sense) is a <a href="https://en.wikipedia.org/wiki/Summary_statistic">summary statistic</a> that quantitatively describes or summarizes features of a collection of <a href="https://en.wikipedia.org/wiki/Information">information</a>, while <strong>descriptive statistics</strong> in the <a href="https://en.wikipedia.org/wiki/Mass_noun">mass noun</a> sense is the process of using and analyzing those statistics. Descriptive statistics is distinguished from <a href="https://en.wikipedia.org/wiki/Statistical_inference">inferential statistics</a> (or inductive statistics), in that descriptive statistics aims to summarize a <a href="https://en.wikipedia.org/wiki/Sample_(statistics)">sample</a>, rather than use the data to learn about the <a href="https://en.wikipedia.org/wiki/Statistical_population">population</a> that the sample of data is thought to represent. This generally means that descriptive statistics, unlike inferential statistics, is not developed on the basis of <a href="https://en.wikipedia.org/wiki/Probability_theory">probability theory</a>, and are frequently <a href="https://en.wikipedia.org/wiki/Nonparametric_statistics">nonparametric statistics</a>. Even when a data analysis draws its main conclusions using inferential statistics, descriptive statistics are generally also presented. For example, in papers reporting on human subjects, typically a table is included giving the overall <a href="https://en.wikipedia.org/wiki/Sample_size">sample size</a>, sample sizes in important subgroups (e.g., for each treatment or exposure group), and <a href="https://en.wikipedia.org/wiki/Demographic">demographic</a> or clinical characteristics such as the <a href="https://en.wikipedia.org/wiki/Average">average</a> age, the proportion of subjects of each sex, the proportion of subjects with related <a href="https://en.wikipedia.org/wiki/Comorbidity">comorbidities</a>, etc. [p1]</blockquote><p>Therefore in layman terms, <strong>Descriptive Statistics</strong> is a form of <em>representation of Data</em> where we aim to <em>summarize</em>, <em>label</em>, <em>organize</em> and <em>clean up</em> the data given in various formats, but <strong>without drawing conclusions <em>or </em>inferences</strong> based off on the data we are using.</p><h3><strong>Why Descriptive Statistics, Why not just go for Inferential Statistics ?</strong></h3><p>That is because we want to best <strong>describe data</strong> (Descriptive Statistics) and then feed it into our Machine Learning model making process through usage of features and sample sets, we <strong>don’t seek to infer</strong> the data ourselves (Inferential Statistics) since that is not our aim. That is before drawing conclusions from our data, we need to see how rich our <em>data is in both quantity and quality</em> by describing it as best as possible.</p><p><strong>Descriptive Statistical Analysis includes two sub division of analysis methods depending of number of variables we have data on :&gt;</strong></p><h3><strong>1 &gt; <em>Uni-Variate Analysis :</em></strong></h3><p><a href="https://en.wikipedia.org/wiki/Univariate_analysis">Univariate analysis</a> involves describing the <a href="https://en.wikipedia.org/wiki/Frequency_distribution">distribution</a> of a single variable, including its central tendency (including the <a href="https://en.wikipedia.org/wiki/Mean">mean</a>, <a href="https://en.wikipedia.org/wiki/Median">median</a>, and <a href="https://en.wikipedia.org/wiki/Mode_(statistics)">mode</a>) and dispersion (including the <a href="https://en.wikipedia.org/wiki/Range_(statistics)">range</a> and <a href="https://en.wikipedia.org/wiki/Quartiles">quartiles</a> of the data-set, and measures of spread such as the <a href="https://en.wikipedia.org/wiki/Variance">variance</a> and <a href="https://en.wikipedia.org/wiki/Standard_deviation">standard deviation</a>). The shape of the distribution may also be described via indices such as <a href="https://en.wikipedia.org/wiki/Skewness">skewness</a> and <a href="https://en.wikipedia.org/wiki/Kurtosis">kurtosis</a>. Characteristics of a variable’s distribution may also be depicted in graphical or tabular format, including <a href="https://en.wikipedia.org/wiki/Histograms">histograms</a> and <a href="https://en.wikipedia.org/wiki/Stem-and-leaf_display">stem-and-leaf display</a>. [p1]</p><h4><strong>Measure’s of central tendency &gt;</strong></h4><p>These are methods to get a single value that somehow is <em>central</em> and a <em>descriptor of the entire</em> uni variate data set. There are three <em>main</em> measure’s of central tendency : Mean, Median and Mode.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*HO8STTKaPuDHcYCu7TgDNg.png" /><figcaption>[2] Mode, Median and Mean against X variable sample</figcaption></figure><p><strong>Mean</strong> : Also known as <strong>average</strong>, it is simply all the values summed up divided by the number of values in our dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/252/1*JBnuM7dROAjHjFLyCupweA.png" /><figcaption>[3] Mathematical Formula for Mean</figcaption></figure><p><strong>Median</strong> : Median is the value <strong><em>separating the higher half </em>from<em> the lower half</em></strong> of the data, when arranged in ascending or descending order. For an <em>odd numbered data sample</em>, it is the <strong>central number</strong> whereas for an <em>even numbered data sample</em>, it is the <strong>average of the middle two</strong> values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*9bCuEmGh6lDnrla2twVddQ.png" /><figcaption>[4] Median for odd number sample and even number sample</figcaption></figure><p><strong><em>Mean Vs Median &gt;</em></strong></p><p>The main <strong>advantage</strong> of Median over Mean is that it is <em>not affected by the presence of extremely large values or extremely small values </em>(<strong><em>outliers</em></strong>) which may <strong><em>skew</em></strong> the mean. Median is a better reflection of a “<strong><em>typical</em></strong>” value of the set. Median can also be found using the <em>Stem and Leaf Plot </em>method.</p><p><strong>Mode</strong> : The mode of a set of values is the <strong><em>term that appears most frequently</em></strong> thorough the sample, if there are two terms that appear the same number of times the data is <em>bimodal</em>, for three frequent values appearing same maximum number of times the data is <em>trimodal</em>, and for even more modes appearing in a data set, we call it <em>Multimodal</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/422/1*p2ip2QLOO5IoyF-nZqejGw.gif" /><figcaption>[5] Mode for a Sample Data Set</figcaption></figure><h4><strong>Measure’s of Dispersion &gt;</strong></h4><p><em>Central Tendency Measure’s</em> are typical representations of a Frequency Distribution but they <strong>fail to give a complete picture of the distribution</strong>, they don’t tell us about the <strong><em>scatter</em></strong>-ness of the values within the distribution. For that we need Measure’s of Spread and Dispersion, which are a better indicator of <strong>variability within the dataset</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*lEyHOG4r3x827AnB39TyrA.png" /><figcaption>[6] Dispersion and Spread between A, B and C samples which aren’t reflected through Central Tendency Measure’s</figcaption></figure><ul><li><strong>Range</strong> :</li></ul><p>The range is the <strong><em>difference between the smallest value and the largest value</em></strong> in a dataset. That is <em>X(maximum)-X(minimum), </em>therefore range is a big indicator to how vast the data spread is and how close the minimum and maximum values in the whole dataset are.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*Mxp0NqZAF3Bg6qnljJNLjg.png" /><figcaption>[7] Range for a sample dataset as : x(max) — x(min)</figcaption></figure><p>For <em>above</em> dataset, we have <strong><em>Range</em></strong> = 8–4 = 4</p><ul><li><strong>Percentile </strong>:</li></ul><p>There is no universally accepted definition of a percentile. Although a satisfactory explanation can help you understand what it means and how it can be used to rank data values. A percentile is <strong><em>not</em></strong><em> </em>a percent and should not be confused with percentage; a percentile is a <strong><em>value (or the average of two values ) in the data set that marks a certain percentage of the way through the data</em></strong>. If you are at 80th percentile that means that 80% values were below you and 20% values are above yours.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/1*kXH1tMBoTrnOZ4tLA796qA.png" /><figcaption>[8] Percentile for height data sample</figcaption></figure><p>For <strong>grouped percentiles</strong> we add up the <strong><em>all the percentages for the lower groups</em></strong> and add up <strong><em>half the percentages at the group</em></strong> we wish to calculate the percentile of.</p><p><strong><em>Deciles</em></strong> are similar to Percentiles where they <strong><em>split up percentile</em></strong>s to better indicate rank of a value compared to the rest of the dataset values. The <strong><em>1st decile is the 10th percentile</em></strong> (value dividing data that is 10% below it), <strong><em>2nd decile is 20th percentile</em></strong> (value dividing data that is 20% below it)and <strong><em>so on …</em></strong></p><ul><li><strong>Quartile </strong>:</li></ul><p>Quartile is like decile but here we have Quarter splits of percentiles.</p><p>The first quartile, denoted <strong>Q1</strong>, is the value in the data set that holds 25% of the values <em>below</em> it. The third quartile, denoted <strong>Q3</strong>, is the value in the data set that holds 25% of the values <em>above</em> it.</p><p>Q1 -&gt; 25 percentile (holds <em>25% values</em> below it), <em>median</em> of <em>lower half</em></p><p>Q2 -&gt; 50 percentile (holds <em>50% values</em> below it) also the <em>median of whole data</em></p><p>Q3 -&gt; 75 percentile (holds <em>75% values</em> below it), <em>median</em> of <em>upper half</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*gV5r1dUfmaPxoSMsL7h5rA.png" /><figcaption>[9] IQR and Quartiles</figcaption></figure><p>We have IQR defined such that, <strong>Inter Quartile Range (IQR) = Q3-Q1 = 77–64 = 13 </strong>for the dataset in figure.</p><h4><strong>Measure’s of Spread &gt;</strong></h4><p>Both <strong><em>Variance</em></strong> and <strong><em>Standard Deviation</em></strong> are <em>measures of the spread of data around the mean</em>. <em>Larger</em> Standard Deviance and Variance values indicate a more <strong><em>dispersed</em></strong> dataset, while <strong><em>Zero</em></strong> Standard Deviance and Variance means all dataset <strong><em>values are the same</em></strong>.</p><h4><strong>Standard Deviation :</strong></h4><p><strong><em>Standard Deviation</em></strong> (<strong>σ</strong>) also called <strong><em>SD </em></strong>is a measure that is used to <em>quantify</em> the amount of <strong><em>variation</em></strong> or <strong><em>dispersion</em></strong> of a dataset. A <em>smaller</em> SD value indicates that more d<em>ata values are closer to mean</em> and <em>larger</em> SD values mean the dataset is <em>widely dispersed</em>. The standard deviation of a <a href="https://en.wikipedia.org/wiki/Random_variable">random variable</a>, <a href="https://en.wikipedia.org/wiki/Statistical_population">statistical population</a>, <a href="https://en.wikipedia.org/wiki/Data_set">data set</a>, or <a href="https://en.wikipedia.org/wiki/Probability_distribution">probability distribution</a> is the <a href="https://en.wikipedia.org/wiki/Square_root">square root</a> of its <a href="https://en.wikipedia.org/wiki/Variance">variance</a>. It is <a href="https://en.wikipedia.org/wiki/Algebra">algebraically</a> simpler, though in practice less <a href="https://en.wikipedia.org/wiki/Robust_statistics">robust</a>, than the <a href="https://en.wikipedia.org/wiki/Average_absolute_deviation">average absolute deviation</a>. A useful property of the standard deviation is that, unlike the variance, it is expressed in the same units as the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*xWzRbxFsN_9suIbZQE3hNQ.png" /><figcaption>[20] <strong>σ </strong>for Metabolic Rate Vs Sex of Fulmars</figcaption></figure><ul><li>Mathematically for <strong>non-grouped data</strong>, which is just basically a list of values, the standard deviation is given by :</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*huzReg2CFf67WbfywwZYIg.png" /><figcaption>[21] SD for non-grouped data</figcaption></figure><p>Working out the mean for below data sample we have :</p><p>mean = (4 + 9 + 11 + 12 + 17 + 5 + 8 + 12 + 14) / 9 = 10.222</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/789/1*gyCGBtUGs09OUy648q1lkA.png" /><figcaption>[22] Sample Data Set and the mean difference squared values</figcaption></figure><p>Summing up the above mean differences we get :</p><p>38.7 + 1.49 + 0.60 + 3.16 + 45.9 + 27.3 + 4.94 + 3.16 + 14.3 = 139.55</p><p>dividing above by n which is 9 we get 15.51 which is <strong>σ squared</strong>, now for <strong>σ :</strong></p><p>we take square root of 15.51 which gives <strong>σ = 3.94</strong></p><ul><li>For <strong>grouped data</strong> :</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/361/1*cVvuPgUV3_-YB5H0hv6vjQ.png" /><figcaption>[23] SD for grouped data</figcaption></figure><p>where :</p><ul><li><strong><em>x</em></strong> mean = summation (<strong><em>f</em></strong> * <strong><em>x</em></strong>) / summation (<strong><em>f</em></strong>)</li></ul><p>The following is an example dataset of <em>grouped data</em>, <strong><em>x</em></strong> given along with it’s frequency <strong><em>f</em></strong> :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/84/1*t4ujDn-jIuZM8LtKQyoTcw.png" /><figcaption>[24] grouped data (<strong><em>x</em></strong>,<strong><em>f</em></strong>)</figcaption></figure><p>For above we can calculate :</p><p><strong><em>mean</em></strong> = (4x9 + 5x14 + 6x22 + 7x11 + 8x17) / (9 + 14 + 22 + 11 + 17)</p><p>Therefore <strong><em>mean</em></strong> = 451 / 73 = 6.178</p><p><em>summation</em> of <em>mean difference squared</em> :</p><p>= 9*(-2.178)² + 14*(-1.178)² + 22*(-0.178)² + 11*(0.822)² + 17*(1.822)²</p><p><em>root</em> (<em>summation mean diff sq</em> / summation <strong><em>f</em></strong>) = 126.685 / 73 = 1.784 = 1.33</p><p>So <strong><em>standard deviation</em></strong> for above sample is : 1.33</p><p>Also it is to be noted that Standard Deviation has two form based off on the population size of the sample data and whether you want to generalize it or not:</p><ul><li><strong>Sample Standard Deviation </strong>:</li></ul><p>Usually where we <em>don’t have</em> <em>complete Population sample data</em>, Sample Standard Deviation is used and we <strong><em>generalize</em></strong> our findings for the entire population, <em>mathematically</em> &gt;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/449/1*djMup9MFbK6_QPMUB9gisg.png" /><figcaption>[25] Sample Standard Deviation</figcaption></figure><ul><li><strong>Population Standard Deviation </strong>:</li></ul><p>In cases where <strong><em>complete</em></strong><em> Population Sample data is </em><strong><em>present</em></strong>, Population standard deviation is used, here there is no need to generalize the result and <strong><em>data samples</em></strong> can be further used to <em>find Sample Standard Deviation</em> for <strong><em>specific</em></strong> population ranges, <em>mathematically</em> we use&gt;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*lE65nT6k-BD91DJ9OrHqjg.png" /><figcaption>[26] Population Standard Deviation</figcaption></figure><h4><strong>Variance :</strong></h4><p><em>Variance (σ squared)</em> <strong><em>measures how far a dataset is spread out</em></strong>. The technical definition is “The <em>average of the squared differences from the mean</em>”. We use <strong><em>variance</em></strong> to see how individual numbers relate to each other within a data set, <em>rather than using broader mathematical techniques</em> such as arranging numbers into Quartiles. A drawback to variance is that it gives added weight to numbers far from the mean (<strong><em>outliers</em></strong>), since squaring these numbers can <strong><em>skew</em></strong> interpretations of the data. The <strong>advantage</strong> of variance is that it <strong><em>treats all deviations from the mean the same regardless of direction</em></strong>- as a result, the squared deviations cannot sum to zero and give the appearance of no variability at all in the data. The <strong><em>drawback</em></strong> of variance is that it is <em>not easily interpreted</em>, and the <em>square root of its value is usually taken to get the standard deviation of the data set</em> in question.</p><p>Mathematically :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b-GB8ukVzf8BtYfgKPRRYw.jpeg" /><figcaption>[19] Mathematical Formula for Variance</figcaption></figure><ul><li><strong>σ squared</strong> is variance</li><li>χ is the value of an <strong>individual</strong> data point</li><li>μ is the <strong>mean</strong> of data points</li><li>N is the total <strong># of data points</strong></li></ul><p>The variance for a <strong>population</strong> (a <em>parameter</em> not a statistic)is calculated by:</p><ol><li>Finding the mean (the average).</li><li>Subtracting the mean from each number in the data set and then squaring the result. The results are squared to make the negatives positive. Otherwise negative numbers would cancel out the positives in the next step. It’s the distance from the mean that’s important, not positive or negative numbers.</li><li>Averaging the squared differences.</li></ol><h4><strong>Other Measure’s that can describe shape of Distribution</strong> &gt;</h4><h3><strong>Skewness</strong> &gt;</h3><p>It is the <em>degree of distortion of Normal Distribution </em>also called ‘<em>tapering</em>’ of data distribution, <strong>Skewness</strong> is a measure of symmetry <em>or</em> more precisely <strong><em>lack of symmetry</em></strong> in our Data Distribution. A distribution, or data set, is symmetric if it looks the same to the left and right of the center point, and skewness is a measure of how <em>asymmetric</em> the distribution is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fWvQFQZrA4LijlMPSjf97A.png" /><figcaption>[10] Skew both +ve and -ve</figcaption></figure><ol><li><strong>negative skew</strong>: The left tail is longer; the mass of the distribution is concentrated on the right of the figure. It has a few relatively low values. The distribution is said to be <em>left-skewed</em>. In such a distribution, the <a href="https://en.wikiversity.org/wiki/Mean">mean</a> is lower than <a href="https://en.wikiversity.org/wiki/Median">median</a> (or equivalently, the <a href="https://en.wikiversity.org/wiki/Mean">mean</a> is lower than the <a href="https://en.wikiversity.org/w/index.php?title=Mod&amp;action=edit&amp;redlink=1">mode</a>; in which case the skewness is lower than zero.</li><li><strong>positive skew</strong>: The right tail is longer; the <em>mass</em> of the distribution is concentrated on the left of the figure. It has a few relatively high values. The distribution is said to be <em>right-skewed</em>. In such a distribution, the <a href="https://en.wikiversity.org/wiki/Mean">mean</a> is greater than <a href="https://en.wikiversity.org/wiki/Median">median</a>, or equivalently, the <a href="https://en.wikiversity.org/wiki/Mean">mean</a> is greater than the <a href="https://en.wikiversity.org/w/index.php?title=Mod&amp;action=edit&amp;redlink=1">mode</a>; in which case the skewness is greater than zero.</li></ol><p>In a skewed (unbalanced, lopsided) distribution, the <a href="https://en.wikiversity.org/wiki/Mean">mean</a> is farther out in the long tail than is the <a href="https://en.wikiversity.org/wiki/Median">median</a>. If there is no skewness or the distribution is symmetric like the <a href="https://en.wikiversity.org/wiki/Normality">bell-shaped normal curve</a> then the mean = median = mode.</p><p><strong>Gauging -&gt; Excessive Skewness or Moderate Skewness :</strong></p><ul><li>For skewness between -0.5 and 0.5 our data is quite symmetrical and not skewed too much.</li><li>For skewness between -1.0 and -0.5 or 0.5 and 1.0 our data is moderately skewed.</li><li>For skewness exceeding 1.0 or below -1.0 our data is excessively skewed.</li></ul><p>We have many formulas that can be used to derive a Skew coefficient from data values instead of relying on visual plots to understand if it’s positive or negative. Some of them are :</p><ul><li>Fisher-Pearson coefficient of skewness :</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*kvBHYPSW_6FmJ0-zoPLEhg.png" /><figcaption>[11] Fisher-Pearson, s is standard deviation</figcaption></figure><ul><li>Galton skewness (also known as Bowley’s skewness) :</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/333/1*S9odyTOFZeo66n3dbQ9pxg.png" /><figcaption>[12] Galton Skewness</figcaption></figure><ul><li>Pearson 2 skewness :</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/307/1*wTCMK4QJDHJ40e3YtSTbNw.png" /><figcaption>[13] Pearson 2, Y~ is sample median</figcaption></figure><h3>Kurtosis &gt;</h3><p>Kurtosis is a <strong><em>measure of outliers</em></strong> present in the distribution. It is a measure that indicates how much data is contained in the tails of your data. Kurtosis can also said to be a measure of the “<em>tailedness</em>” of the <a href="https://en.wikipedia.org/wiki/Probability_distribution">probability distribution</a> of a <a href="https://en.wikipedia.org/wiki/Real_number">real</a>-valued <a href="https://en.wikipedia.org/wiki/Random_variable">random variable</a>.</p><ul><li><strong>positive </strong>value tells us that our dataset has <em>heavy-tails</em>. (lot of data in tails)</li><li><strong>negative </strong>value means that dataset has <em>light-tails</em>. (little data in tails)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*sGnM0G200MmHD0iHfMwcoQ.png" /><figcaption>[14] Kurtosis, Normal Distribution in Black</figcaption></figure><p><strong>Excess Kurtosis &gt;</strong></p><p>The <em>excess</em> Kurtosis is defined as <strong><em>Kurtosis minus 3</em></strong>, it is a measure of how the distribution’s tails compare to normal (see Aldrich, E, 2014). Three different Kurtosis possibilities are :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*w-rfKNo03ha9wLdvU4Y31A.png" /><figcaption>[15] Mesokurtic, Leptokurtic and Platykurtic Distributions</figcaption></figure><ul><li><strong><em>Mesokurtic </em></strong>: Distribution with zero excess Kurtosis are called <strong><em>mesokurtic</em></strong>, or mesokurtotic, the most common example would be Normal Distribution and it’s family, as standard Normal Distribution has kurtosis of three.</li><li><strong><em>Leptokurtic (Kutosis &gt; 3)</em></strong> : A distribution with <a href="https://en.wikipedia.org/wiki/Positive_number">positive</a> excess kurtosis is called <strong>leptokurtic</strong>, or leptokurtotic. “Lepto-” means “slender”. In terms of shape, a leptokurtic distribution has <a href="https://en.wikipedia.org/wiki/Fat-tailed_distribution"><em>fatter tails</em></a>. Examples of leptokurtic distributions include the <a href="https://en.wikipedia.org/wiki/Student%27s_t-distribution">Student’s t-distribution</a>, <a href="https://en.wikipedia.org/wiki/Rayleigh_distribution">Rayleigh distribution</a>, <a href="https://en.wikipedia.org/wiki/Laplace_distribution">Laplace distribution</a>, <a href="https://en.wikipedia.org/wiki/Exponential_distribution">exponential distribution</a>, <a href="https://en.wikipedia.org/wiki/Poisson_distribution">Poisson distribution</a> and the <a href="https://en.wikipedia.org/wiki/Logistic_distribution">logistic distribution</a>. Such distributions are sometimes termed <em>super-Gaussian</em>.</li><li><strong><em>Platykurtic (Kurtosis &lt; 3)</em></strong>: A distribution with <a href="https://en.wikipedia.org/wiki/Negative_number">negative</a> excess kurtosis is called <strong>platykurtic</strong>, or platykurtotic. “Platy-” means “broad”. In terms of shape, a platykurtic distribution has <em>thinner tails</em>. Examples of platykurtic distributions include the <a href="https://en.wikipedia.org/wiki/Continuous_uniform_distribution">continuous</a> and <a href="https://en.wikipedia.org/wiki/Discrete_uniform_distribution">discrete uniform distributions</a>, and the <a href="https://en.wikipedia.org/wiki/Raised_cosine_distribution">raised cosine distribution</a>. The most platykurtic distribution of all is the <a href="https://en.wikipedia.org/wiki/Bernoulli_distribution">Bernoulli distribution</a> with <em>p</em> = 1/2 (for example the number of times one obtains “heads” when flipping a coin once, a <a href="https://en.wikipedia.org/wiki/Coin_toss">coin toss</a>), for which the excess kurtosis is −2. Such distributions are sometimes termed <em>sub-Gaussian</em>.</li></ul><h3><strong>2 &gt; <em>Bi-Variate Analysis and Multi-Variate Analysis :</em></strong></h3><p>When a sample consists of more than one variable, descriptive statistics may be used to describe the relationship between pairs of variables. In this case, descriptive statistics include:</p><ul><li>Cross-tabulations and contingency tables</li><li>Graphical representation via <a href="https://en.wikipedia.org/wiki/Scatterplot">scatterplots</a></li><li>Quantitative measures of <a href="https://en.wikipedia.org/wiki/Correlation_and_dependence">dependence</a></li><li>Descriptions of <a href="https://en.wikipedia.org/wiki/Conditional_distribution">conditional distributions</a></li></ul><p>The main reason for differentiating univariate and bivariate analysis is that bivariate analysis is not only simple descriptive analysis, but also it describes the relationship between two different variables. Quantitative measures of dependence include correlation (such as <a href="https://en.wikipedia.org/wiki/Pearson%27s_r">Pearson’s r</a> when both variables are continuous, or <a href="https://en.wikipedia.org/wiki/Spearman%27s_rho">Spearman’s rho</a> if one or both are not) and <a href="https://en.wikipedia.org/wiki/Covariance">covariance</a> (which reflects the scale variables are measured on). The slope, in regression analysis, also reflects the relationship between variables. The unstandardized slope indicates the unit change in the criterion variable for a one unit change in the <a href="https://en.wikipedia.org/wiki/Prediction">predictor</a>. The standardized slope indicates this change in standardized (<a href="https://en.wikipedia.org/wiki/Z-score">z-score</a>) units. Highly skewed data are often transformed by taking logarithms. Use of logarithms makes graphs more symmetrical and look more similar to the <a href="https://en.wikipedia.org/wiki/Normal_distribution">normal distribution</a>, making them easier to interpret intuitively.[p1]</p><h3><strong>Correlation &gt;</strong></h3><p>Correlation is any <strong><em>statistical association</em></strong> (statistical <em>relationship</em>) between <em>two </em><strong><em>random variables</em></strong> or <strong><em>bi-variate data</em></strong>. In common cases though, it refers to how close of a <em>linear relationship two variables have</em> with each other.</p><p>Some examples of data that have a <strong>high correlation:</strong></p><ul><li>Hype for Machine Learning Vs people doing Machine Learning work.</li><li>Distance you run each day and your fitness level.</li><li>The amount of typing you do and your average typing wpm.</li></ul><p>Some examples of data that have a <strong>low correlation </strong>(or none at all):</p><ul><li>How much TV you watch vs how much food your dog eats.</li><li>Your Uncle’s favorite color and your name.</li><li>Possible release dates for Avenger’s : Endgame (2019) and possible release dates for Joker (2019) [not too sure though]</li></ul><p>Mathematically, it depends on <em>correlation coefficient</em> ‘<strong>r</strong>’ it ranges from -1.0 to +1.0 and the closer <strong>r</strong> is to -1.0 or +1.0, the more closely related the two variables are. For correlation due to value being closer to -1.0, it is also called ‘<em>inverse</em>’ correlation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LyOPZ4QKuN0JCl1myLJ7KQ.png" /><figcaption>[18] Several Sets of (<strong><em>x</em></strong>,<strong><em>y</em></strong>) points with Pearson Coefficient ‘<strong>r’ </strong>for each set.</figcaption></figure><p>Usually, in statistics, we measure four types of correlations: <strong><em>Pearson correlation, Kendall rank correlation, Spearman correlation, and the Point-Biserial correlation</em></strong>. But <strong><em>Pearson r correlation</em></strong> is the most widely used technique for linear relationship between variables :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*i9DZtys8fNryYEVF9tn6vw.png" /><figcaption>[16] Formula for Pearson <strong>r</strong> correlation</figcaption></figure><p>There are <em>five</em> assumptions that are made with respect to <em>Pearson’s correlation</em>:</p><ol><li>The variables must be either <em>interval</em> or <em>ratio measurements</em>.</li><li>The variables must be <strong><em>approximately normally distributed</em></strong>.</li><li>There is a <strong><em>linear relationship</em></strong> between the two variables.</li><li><strong><em>Outliers</em></strong> are either kept to a <em>minimum</em> or are <em>removed</em> entirely. (This can be done by removing the extreme outliers after plotting the data)</li><li>There is <em>homoscedasticity</em> of the data. In <a href="https://en.wikipedia.org/wiki/Statistics">statistics</a>, a <a href="https://en.wikipedia.org/wiki/Sequence">sequence</a> or a vector of <a href="https://en.wikipedia.org/wiki/Random_variable">random variables</a> is <strong>homoscedastic</strong> if all random variables in the sequence or vector have the same finite <a href="https://en.wikipedia.org/wiki/Variance">variance</a>. This is also known as <strong>homogeneity of variance</strong>. [p2]</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*kH4q99St_Yx1XlPkzVsZxw.png" /><figcaption>[17] Graphical Test for evaluation of Data Linearity</figcaption></figure><p>Therefore it is <em>inappropriate</em> to use <em>Pearson’s r coefficient method</em> on data that doesn’t fulfill the above requirements such as the <strong><em>non-linear dataset</em></strong> in above plots, which will give <strong><em>erroneous values</em></strong> of <strong>r</strong>. To avoid this data can be plotted through usage of <strong><em>scatter plot</em></strong> and verified for linearity.</p><p>For citing do use the format :</p><pre>@article{himanshuxd,<br>    author       = {Himanshu S},<br>    title        = { The Guide to Rigorous Descriptive Statistics : For Machine Learning and Data Science },<br>    howpublished = {\url{https://himanshuxd.medium.com/the-guide-to-rigorous-descriptive-statistics-for-machine-learning-and-data-science-9209f88e4363}},<br>    year         = {2019}<br>}</pre><p>[p1] : <a href="https://en.wikipedia.org/wiki/Descriptive_statistics">https://en.wikipedia.org/wiki/Descriptive_statistics</a></p><p>[p2] : <a href="https://en.wikipedia.org/wiki/Homoscedasticity">https://en.wikipedia.org/wiki/Homoscedasticity</a></p><p>[1] : <a href="http://doingstats.com/img/stats_process.gif">http://doingstats.com/img/stats_process.gif</a></p><p>[2] : <a href="https://www.researchgate.net/figure/Measures-of-central-tendency_fig10_258397249">https://www.researchgate.net/figure/Measures-of-central-tendency_fig10_258397249</a></p><p>[3] : <a href="http://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_SummarizingData/">http://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_SummarizingData/</a></p><p>[4] : <a href="https://en.wikipedia.org/wiki/Median">https://en.wikipedia.org/wiki/Median</a></p><p>[5] : <a href="http://www.mathsmutt.co.uk/files/statbook_files/Eqn2.gif">http://www.mathsmutt.co.uk/files/statbook_files/Eqn2.gif</a></p><p>[6] : <a href="http://statisticalconcepts.blogspot.com/2017/01/measures-of-dispersion-statistics.html">http://statisticalconcepts.blogspot.com/2017/01/measures-of-dispersion-statistics.html</a></p><p>[7] : <a href="http://www.abs.gov.au/websitedbs/a3121120.nsf/home/statistical+language+-+measures+of+spread">http://www.abs.gov.au/websitedbs/a3121120.nsf/home/statistical+language+-+measures+of+spread</a></p><p>[8] : <a href="https://www.mathsisfun.com/data/percentiles.html">https://www.mathsisfun.com/data/percentiles.html</a></p><p>[9] : <a href="http://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_summarizingdata/bs704_summarizingdata7.html">http://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_summarizingdata/bs704_summarizingdata7.html</a></p><p>[10] : <a href="https://en.wikiversity.org/wiki/Skewness">https://en.wikiversity.org/wiki/Skewness</a></p><p>[11], [12], [13] : <a href="https://www.itl.nist.gov/div898/handbook/eda/section3/eda35b.htm">https://www.itl.nist.gov/div898/handbook/eda/section3/eda35b.htm</a></p><p>[14] : <a href="https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/statistics-definitions/kurtosis-leptokurtic-platykurtic/">https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/statistics-definitions/kurtosis-leptokurtic-platykurtic/</a></p><p>[15] : <a href="https://imgur.com/gallery/15YSVEm">https://imgur.com/gallery/15YSVEm</a></p><p>[16] : <a href="https://www.statisticssolutions.com/correlation-pearson-kendall-spearman/">https://www.statisticssolutions.com/correlation-pearson-kendall-spearman/</a></p><p>[17] : <a href="https://statistics.laerd.com/statistical-guides/pearson-correlation-coefficient-statistical-guide.php">https://statistics.laerd.com/statistical-guides/pearson-correlation-coefficient-statistical-guide.php</a></p><p>[18] : <a href="https://en.wikipedia.org/wiki/Correlation_and_dependence">https://en.wikipedia.org/wiki/Correlation_and_dependence</a></p><p>[19] : <a href="https://www.investopedia.com/terms/v/variance.asp">https://www.investopedia.com/terms/v/variance.asp</a></p><p>[20] : <a href="https://en.wikipedia.org/wiki/Standard_deviation">https://en.wikipedia.org/wiki/Standard_deviation</a></p><p>[21], [22], [23], [24] : <a href="https://revisionmaths.com/gcse-maths-revision/statistics-handling-data/standard-deviation">https://revisionmaths.com/gcse-maths-revision/statistics-handling-data/standard-deviation</a></p><p>[25], [26] : <a href="https://statistics.laerd.com/statistical-guides/measures-of-spread-standard-deviation.php">https://statistics.laerd.com/statistical-guides/measures-of-spread-standard-deviation.php</a></p><p><strong>See you on my next article !</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RUS7yXb1uHtpJPv8y2AkcA.png" /></figure><pre><em>If you found this useful and informative, please let me know by clapping or commenting ! Also for any queries you may have in regard to the above, ask me by commenting </em><strong><em>or</em></strong><em> tweeting @himanshuxd</em></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9209f88e4363" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Activation Functions : Sigmoid, ReLU, Leaky ReLU and Softmax basics  for Neural Networks and Deep…]]></title>
            <link>https://himanshuxd.medium.com/activation-functions-sigmoid-relu-leaky-relu-and-softmax-basics-for-neural-networks-and-deep-8d9c70eed91e?source=rss-8f8658588de9------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d9c70eed91e</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[tutorial]]></category>
            <dc:creator><![CDATA[Himanshu S]]></dc:creator>
            <pubDate>Sat, 19 Jan 2019 03:29:52 GMT</pubDate>
            <atom:updated>2024-05-19T23:23:30.089Z</atom:updated>
            <content:encoded><![CDATA[<h3>Activation Functions : Sigmoid, tanh, ReLU, Leaky ReLU, PReLU, ELU, Threshold ReLU and Softmax basics for Neural Networks and Deep Learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y-VSlI1cW0rhiuXLagLkpA.png" /><figcaption>Activation Functions in Kryptonese ?</figcaption></figure><p>In the fascinating realm of artificial intelligence, <em>Neural Networks</em> stand as a cornerstone, revolutionizing technologies from speech recognition to real-time translation. The journey of Neural Networks began in 1944, thanks to the pioneering work of <strong><em>Warren McCullough</em></strong> and <strong><em>Walter Pitts</em></strong>. These networks, <em>inspired by the intricate workings of the human brain</em>, consist of numerous interconnected nodes that <strong><em>mimic the firing of biological neurons</em></strong>.</p><p>Imagine a vast network where each node, akin to a <em>neuron</em>, processes <strong><em>incoming signals</em></strong><em> by multiplying them with specific weights</em>. These signals, akin to synaptic inputs, are then <strong><em>summed</em></strong> to produce an <strong><em>output</em></strong>. This process, known as <em>Forward Propagation</em>, is fundamental to how Neural Networks learn and make decisions. (Note: The gradient calculation and <em>backpropagation</em>, which come into play as we add more layers, will be discussed in a subsequent post.)</p><p>But what truly breathes life into these networks are <strong><em>Activation Functions</em></strong>. These functions <strong><em>determine</em></strong> whether a <em>neuron</em> should be <strong><em>activated or not</em></strong> more or less like how our own Biological brains work, essentially deciding the neuron’s output based on the input it receives. Activation Functions are crucial for introducing <strong>non-linearity</strong> into the network, <strong>enabling it to learn complex patterns</strong> and perform sophisticated tasks.</p><p>Below is a sample code snippet illustrating how a simple <strong><em>digital neuron</em></strong>, mimicking a biological neuron, might be defined with a <em>Python</em> class :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4402b841367d3b6053edb46cc489f15d/href">https://medium.com/media/4402b841367d3b6053edb46cc489f15d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*Jx97VUn1i4GFBbD6G11eyg.png" /><figcaption>Synapses and Neurons in Neural Networks both Biological and Computational</figcaption></figure><p>For example (see D in above figure), if the <em>weights</em> are <strong><em>w1, w2, w3 … wN</em></strong> and <em>inputs are </em><strong><em>i1, i2, i3 … iN</em></strong> we get a <em>summation</em> of <strong><em>S = w1*i1 + w2*i2 + w3*i3 … wN*iN</em></strong></p><p>For several layers of <em>Neural Networks</em> and Connections we can have varied values of <strong><em>wX</em></strong> and <strong><em>iX</em></strong> and the summation <strong><em>S</em></strong> which varies according to whether the particular <em>Neuron</em> is <strong><em>activated</em></strong> or <strong><em>not</em></strong>, so to <strong><em>normalize</em></strong> this and prevent drastically different range of values, we use what is called a <strong><em>Activation Function</em></strong> for <em>Neural networks</em> that turns these values into something equivalent between <strong>0, 1</strong> <em>or</em> <strong>-1, 1</strong> to make the whole process <strong><em>statistically balanced</em></strong>. This process is <strong>not</strong> just to preserve sanity of the code but also to <strong>reduce complexity</strong> and <strong>computing power</strong> required which would be more difficult on inactivated inputs. It also adds stability within the network by <strong>mitigating the impact of vastly divergent input</strong> values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*Ah2uXX7FVN3gHoy_6UobKA.gif" /><figcaption>Input, Weights and Output</figcaption></figure><h3>Introduction &gt;</h3><p><strong><em>Activation functions</em></strong> that are commonly used based on few desirable properties like :</p><ul><li><em>Nonlinear</em> — The hallmark of a potent activation function lies in its nonlinearity. By introducing nonlinearities, even a two-layer neural network can morph into a universal function approximator. Notably, the identity activation function falls short in this regard, rendering networks employing it akin to mere single-layer models.</li><li><em>Range </em>— The range of an activation function plays a critical role in shaping the stability and efficiency of gradient-based training methods. Finite ranges foster stability, as pattern presentations exert significant influence over only a subset of weights. Conversely, infinite ranges enhance efficiency, amplifying the impact of pattern presentations across most weights. However, in the latter scenario, smaller learning rates are typically warranted to navigate the heightened variability.</li><li><em>Continuously differentiable</em> — For seamless integration with gradient-based optimization methods, activation functions must exhibit continuity and differentiability. While the ReLU function deviates from this criterion by lacking continuity at certain points, it remains viable for use, albeit with some optimization challenges. In contrast, the binary step activation function presents insurmountable hurdles, as its lack of differentiability at crucial junctures impedes any meaningful progress with gradient-based methods.</li></ul><blockquote><strong>Derivative or Differential or Slope: </strong>Change in y-axis according to change in x-axis.</blockquote><ul><li><em>Monotonic</em> — A crucial characteristic of an activation function lies in its monotonicity. A monotonic activation function ensures that the error surface associated with a single-layer model remains convex, facilitating more predictable and efficient optimization processes.</li></ul><blockquote><strong>Monotonic function:</strong> A function which is either entirely non-increasing or non-decreasing.</blockquote><ul><li><em>Smooth functions with a monotonic derivative</em> — Activation functions that exhibit smoothness with monotonic derivatives have demonstrated superior generalization abilities in certain scenarios. This property enhances the network’s capacity to generalize patterns beyond the training data, leading to more robust performance in real-world applications.</li><li><em>Approximates identity near the origin</em> — Activation functions that approximate the identity function near the origin offer a distinct advantage during the training phase. In such cases, neural networks can efficiently learn when initialized with small random weights. Conversely, when activation functions deviate from approximating identity near the origin, careful consideration is required during weight initialization to ensure optimal learning dynamics.</li></ul><h3><strong>Table of Activation Functions</strong> &gt;</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F9-nc6ez5GOJ1mdB3TLlow.png" /><figcaption>Activation Function for Neural Networks</figcaption></figure><p>Breaking down some Activation functions :</p><p>There are some linear and simple straightforward functions such as the Binary Step Function or Linear Function <em>(f = ax)</em> but since they are not widely used and are undesirable as activation functions, we won’t be discussing them.</p><h3><strong>1. The Sigmoid Function</strong> &gt;</h3><p>Sigmoid Function (Logistic Function) is a classic choice characterized by its smooth, S-shaped curve. It maps input values to a range between 0 and 1, making it particularly suitable for binary classification tasks where outputs need to be interpreted as probabilities. However, it suffers from the vanishing gradient problem, limiting its efficacy in deep neural networks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rPf0P4QBV4wD8dlfs45OHg.png" /><figcaption>Sigmoid Function</figcaption></figure><p>Although sigmoid function and it’s derivative is <strong><em>simple</em></strong> and helps in reducing time required for making models, there is a major drawback of <strong><em>info loss</em></strong> due to the derivative having a short range.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/1*Oty_OoM9o02EmbwEqeXEgw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/1*HaAT5CmVq2io8PdPKIgVHQ.png" /><figcaption>Sigmoid and it’s derivative function</figcaption></figure><p>As our neural network grows deeper, information compression increases, leading to potential data loss known as the vanishing and exploding gradient problem. Sigmoid functions, biased towards positive outputs, are less ideal for early layers due to this positive skew. However, they find utility in the final layer for output interpretation. Understanding these nuances aids in optimizing neural network performance.</p><blockquote><em>Besides the </em><a href="https://en.wikipedia.org/wiki/Logistic_function"><em>logistic function</em></a><em>, sigmoid functions include the ordinary </em><a href="https://en.wikipedia.org/wiki/Inverse_trigonometric_function"><em>arctangent</em></a><em>, the </em><a href="https://en.wikipedia.org/wiki/Hyperbolic_function"><em>hyperbolic tangent</em></a><em>, the </em><a href="https://en.wikipedia.org/wiki/Gudermannian_function"><em>Gudermannian function</em></a><em>, and the </em><a href="https://en.wikipedia.org/wiki/Error_function"><em>error function</em></a><em>, but also the </em><a href="https://en.wikipedia.org/wiki/Generalised_logistic_function"><em>generalised logistic function</em></a><em> and </em><a href="https://en.wikipedia.org/wiki/Algebraic_function"><em>algebraic functions</em></a></blockquote><h3>2. <strong>Tanh Function </strong>&gt;</h3><p>In <strong><em>tanh</em></strong> function the drawback we saw in sigmoid function is addressed (not entirely), here the only difference with sigmoid function is the curve is symetric across the origin with values ranging from -1 to 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*3OerqzdzmcDWbo6XC_zG_g.png" /><figcaption>blue dotted line shows tanh function</figcaption></figure><p>The formula for hyperbolic tangent (tanh) can be given as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/109/1*KoMk9IyQpjlD4G3Zuzn3kQ.png" /><figcaption>tanh(x)</figcaption></figure><p>While tanh mitigates some of the issues present in the sigmoid function, such as asymmetry, it doesn’t completely eliminate the vanishing or exploding gradient problem. However, being centered at zero, it offers advantages over the sigmoid function. Hence, other activation functions are often preferred in machine learning, as we’ll explore below.</p><h3>3. ReLU <strong>(Rectified Linear Units) </strong>and Leaky ReLU &gt;</h3><blockquote>The rectifier is, as of 2018, the most popular activation function for <a href="https://en.wikipedia.org/wiki/Deep_learning">deep neural networks</a>.</blockquote><p>Most contemporary Deep Learning applications, spanning Computer Vision, Speech Recognition, Natural Language Processing, and Deep Neural Networks, predominantly utilize ReLU instead of logistic activation functions. ReLU not only exhibits a <em>superior convergence rate</em> compared to tanh or sigmoid functions but also offers manifold <em>advantages</em> in various applications.</p><p>Additionally, ReLU boasts several variants, such as Softplus (SmoothReLU), Noisy ReLU, Leaky ReLU, Parametric ReLU, and ExponentialReLU (ELU), each tailored to specific requirements. We’ll delve into some of these variants in the subsequent discussion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/1*j_S4-AMQgduuX6ahg3o7eA.png" /><figcaption>ReLU</figcaption></figure><p>ReLU : short for <em>Rectified Linear Unit</em>, functions by setting its output to <strong><em>0 if the input is less than 0</em></strong>, and retaining the input value otherwise. In other words, if the input is greater than 0, the output equals the <em>raw</em> input itself. This operation closely mirrors the behavior of <strong><em>biological neurons</em></strong>, making ReLU a popular choice in neural network architectures.</p><p>3.5</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/282/1*_M8-WXjWWRzR-_f-Ex72pQ.png" /><figcaption>ReLU f(x)</figcaption></figure><p>ReLU is <strong><em>non-linear</em></strong> and has the advantage of not having any <em>backpropagation errors unlike</em> the <em>sigmoid function</em>, also for larger Neural Networks, the <strong><em>speed</em></strong> of building models based off on ReLU is very fast opposed to using Sigmoids :</p><ul><li><em>Biological plausibility</em>: One-sided, compared to the <a href="https://en.wikipedia.org/wiki/Antisymmetric_relation">antisymmetry</a> of <a href="https://en.wikipedia.org/wiki/Tanh">tanh</a>.</li><li><em>Sparse activation</em>: For example, in a randomly initialized network, only about 50% of hidden units are activated (having a non-zero output).</li><li><em>Better gradient propagation</em>: Fewer <a href="https://en.wikipedia.org/wiki/Vanishing_gradient_problem">vanishing gradient</a> problems compared to sigmoidal activation functions that saturate in both directions.</li><li><em>Efficient computation</em>: Only comparison, addition and multiplication.</li><li><em>Scale-invariant</em>: max ( 0, a x ) = a max ( 0 , x ) for a ≥ 0</li></ul><p>While ReLUs offer numerous benefits, they are not without <strong>drawbacks</strong>. Notably, ReLU is <strong>non-zero centered</strong> and <strong>non-differentiable at 0</strong>, although it is differentiable everywhere else. Another limitation of ReLU is its restriction to hidden layers</p><p>It cannot be used elsewhere due to the potential issue which is mentioned below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/726/1*29VH_NiSdoLJ1jUMLrURCA.png" /><figcaption>Sigmoid Vs ReLU</figcaption></figure><p>Another issue encountered with ReLU is the <strong>Dying ReLU problem</strong>, wherein certain ReLU <strong><em>neurons become inactive for all inputs</em></strong>, resulting in a complete cessation of gradient flow. When a large number of neurons suffer from this condition within a neural network, it significantly impacts the network’s performance.</p><p>To address this challenge, Leaky ReLU comes into play. <strong><em>Leaky ReLU</em></strong> introduces a small slope for inputs left of <em>x = 0</em>, as depicted in the figure below, effectively creating a “<strong><em>leak</em></strong>” and extending the range of ReLU. This alteration ensures that <strong><em>even neurons with negative inputs contribute to the gradient flow</em></strong>, mitigating the risk of neuron death and enhancing the robustness of the network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*siH_yCvYJ9rqWSUYeDBiRA.png" /><figcaption>Leaky ReLU and Parametric ReLU</figcaption></figure><p>With Leaky ReLU there is a small negative slope, so instead of not firing at all for large gradients, our neurons do output some value and that makes our layer much more optimized too.</p><h3>4. PReLU (Parametric ReLU) Function &gt;</h3><p>In Parametric ReLU, depicted in the figure above, the slope for <em>x &lt; 0</em> is not fixed, as in Leaky ReLU with a fixed slope of 0.01. Instead, a parameter ′<em>a</em>′ is introduced, which varies depending on the model.</p><p>By employing weights and biases, we adjust this parameter through backpropagation across multiple layers, allowing the network to learn the optimal slope for negative inputs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/330/1*bY6fh1DJxlJz9QE09LjJgg.png" /><figcaption>PReLU</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/1*9oS14TK0zJr0NOcUnMhZVw.png" /><figcaption>for a ≤ 1</figcaption></figure><p>Hence, due to its relationship with the maximum value, PReLU is also employed in architectures known as “maxout” networks.</p><h3>5. ELU (Exponential LU) Function &gt;</h3><p>Exponential Linear Units (ELUs) are utilized to expedite the deep learning process by shifting the mean activations closer to zero. This is achieved by introducing an alpha constant, which must be a positive number.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*mWEL2mKKC_y8les1hBd9VA.png" /><figcaption>ELU blue ReLU brown</figcaption></figure><p>ELU has been demonstrated to yield more accurate results and faster convergence compared to ReLU. While both ELU and ReLU produce similar outputs for positive inputs, ELU smoothly smooths negative inputs (to -alpha) at a slower rate, whereas ReLU sharpens this smoothing abruptly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/396/1*muLdWA3kzgamzzXtKL62gg.png" /><figcaption>alpha is a hyper parameter, with positive value constraint</figcaption></figure><h3>6. Threshold ReLU Function &gt;</h3><p>Through the amalgamation of ReLU and FTSwish, Threshold ReLU (TReLU) was introduced. TReLU resembles ReLU with two significant modifications: it permits negative values while capping them, thereby substantially enhancing accuracy. Its formulation follows as: f(x) = x for x &gt; theta, f(x) = 0 otherwise, where theta is a float &gt;= 0 (the threshold location of activation).</p><h3>7. <strong>Softmax </strong>Function &gt;</h3><p>Softmax is a fascinating activation function because it not only scales the output to a range between 0 and 1 but also ensures that the sum of all outputs equals 1. Consequently, the output of Softmax forms a probability distribution.</p><blockquote>The softmax function is often used in the final layer of a neural network-based classifier. Such networks are commonly trained under a <a href="https://en.wikipedia.org/wiki/Cross_entropy">log loss</a> (or <a href="https://en.wikipedia.org/wiki/Cross-entropy">cross-entropy</a>) regime, giving a non-linear variant of multinomial logistic regression.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*ZeythBexsRfY97H0D0LETA.png" /><figcaption>Softmax Graphed</figcaption></figure><p>Mathematically Softmax is the following function where <strong><em>z</em></strong> is vector of inputs to output layer and <strong><em>j</em></strong> indexes the the output units from <strong><em>1,2, 3 …. k</em></strong> :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/282/1*OC_DUasjBVuWJcbIhUcd4g.png" /><figcaption>Softmax Function</figcaption></figure><p>In conclusion, activation functions are crucial components in neural network architectures, each serving specific purposes. <strong>Softmax activation excels in multi-class classification tasks, while Sigmoid activation is fundamental for binary classification</strong>. Tanh provides symmetric outputs, while <strong>ReLU and its variants ensure robustness and efficiency</strong>. ELU facilitates faster convergence, and Threshold ReLU enables dynamic threshold adjustments. Experimenting with these functions at different layers is essential for optimizing neural networks, empowering practitioners to effectively tackle diverse challenges in machine learning.</p><p>For citing do use the format :</p><pre>@article{himanshuxd,<br>    author       = {Himanshu S},<br>    title        = { Activation  Functions : Sigmoid, tanh, ReLU, Leaky ReLU, PReLU, ELU, Threshold ReLU  and Softmax basics for Neural Networks and Deep Learning },<br>    howpublished = {\url{https://himanshuxd.medium.com/activation-functions-sigmoid-relu-leaky-relu-and-softmax-basics-for-neural-networks-and-deep-8d9c70eed91e}},<br>    year         = {2019}<br>}</pre><p>[1] : <a href="https://en.wikipedia.org/wiki/Artificial_neural_network">https://en.wikipedia.org/wiki/Artificial_neural_network</a></p><p>[2] : <a href="https://en.wikipedia.org/wiki/Activation_function">https://en.wikipedia.org/wiki/Activation_function</a></p><p>[3] : <a href="https://en.wikipedia.org/wiki/Rectifier_(neural_networks)">https://en.wikipedia.org/wiki/Rectifier_(neural_networks)</a></p><p>[4] : <a href="http://cs231n.github.io/neural-networks-1/">http://cs231n.github.io/neural-networks-1/</a></p><p>[5] : <a href="https://en.wikipedia.org/wiki/Softmax_function">https://en.wikipedia.org/wiki/Softmax_function</a></p><p>[6] : <a href="https://github.com/Kulbear/deep-learning-nano-foundation/wiki/ReLU-and-Softmax-Activation-Functions">https://github.com/Kulbear/deep-learning-nano-foundation/wiki/ReLU-and-Softmax-Activation-Functions</a></p><p>[7] : <a href="https://www.kaggle.com/dansbecker/rectified-linear-units-relu-in-deep-learning">https://www.kaggle.com/dansbecker/rectified-linear-units-relu-in-deep-learning</a></p><p>[8] : <a href="http://dataaspirant.com/2017/03/07/difference-between-softmax-function-and-sigmoid-function/">http://dataaspirant.com/2017/03/07/difference-between-softmax-function-and-sigmoid-function/</a></p><p><strong>See you on my next article !</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*aw9icJtk234Boiz-QV-VZA.gif" /></figure><pre><em>If you found this useful and informative, please let me know by clapping or commenting ! Also for any queries you may have in regard to the above, ask me by commenting </em><strong><em>or</em></strong><em> tweeting @himanshuxd</em></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d9c70eed91e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Installing Octave : On Ubuntu and Arch Linux for Stanford Machine Learning Course (the Andrew Ng…]]></title>
            <link>https://himanshuxd.medium.com/installing-octave-on-ubuntu-through-ppa-octave-stable-channel-1eb92779ee26?source=rss-8f8658588de9------2</link>
            <guid isPermaLink="false">https://medium.com/p/1eb92779ee26</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[octave]]></category>
            <category><![CDATA[terminal]]></category>
            <category><![CDATA[ubuntu]]></category>
            <dc:creator><![CDATA[Himanshu S]]></dc:creator>
            <pubDate>Thu, 16 Aug 2018 23:44:47 GMT</pubDate>
            <atom:updated>2021-06-21T18:59:15.548Z</atom:updated>
            <content:encoded><![CDATA[<h3>Installing Octave : On Ubuntu and Arch Linux for Stanford Machine Learning Course (the Andrew Ng Course)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SfIkWHnFAZTX9fCqwf-otA.png" /></figure><p>For Windows Users it might be simpler (or expensive depending on how you look at it) to just use MATLAB which works way faster and has better execution time, but for people who use Linux as a daily driver, here is what the Octave GUI looks like &gt;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JJ8-WPWOvflIOgpPfgNB8Q.png" /><figcaption>Octave GUI</figcaption></figure><p>Even though <a href="https://www.gnu.org/software/octave/#install">octave website</a> doesn’t go into details on how GNU Octave can be installed for various distro of Linux, a quick ppa search of Octave on Google gives the stable release channel, to install Octave execute following commands on your terminal :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a7e13132f29d63a9bd4bead3702427f0/href">https://medium.com/media/a7e13132f29d63a9bd4bead3702427f0/href</a></iframe><p>When above installs, Octave can simply be launched by either entering : octave on your terminal (or) by Octave Desktop launcher found through the Application key.</p><p>For Arch Linux which I have switched to from Ubuntu, you could use the octave community package, to install it issue the following command :</p><pre>sudo pacman -S octave</pre><p>thereafter just use the following command :</p><pre><em>#for the GUI<br>octave — gui</em></pre><pre>#for the CLI<br>octave-cli</pre><p><strong>See you on my next article !</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*2Q2Qu0O1KmfyHanr5CXSfg.gif" /></figure><pre><em>If you found this useful and informative, please let me know by clapping or commenting ! Also for any queries you may have in regard to the above, ask me by commenting </em><strong><em>or</em></strong><em> tweeting @himanshuxd</em></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1eb92779ee26" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Create -> Registration + Login Web App with Python & Django (works for various versions)]]></title>
            <link>https://himanshuxd.medium.com/how-to-create-registration-login-webapp-with-django-2-0-fd33dc7a6c67?source=rss-8f8658588de9------2</link>
            <guid isPermaLink="false">https://medium.com/p/fd33dc7a6c67</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[django]]></category>
            <category><![CDATA[web]]></category>
            <category><![CDATA[tutorial]]></category>
            <dc:creator><![CDATA[Himanshu S]]></dc:creator>
            <pubDate>Tue, 17 Jul 2018 22:48:38 GMT</pubDate>
            <atom:updated>2024-06-03T13:01:57.892Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HQLXyzLanzgDAnHglpVO5w.jpeg" /><figcaption>a snek</figcaption></figure><h3><strong>Why ?</strong></h3><p>I have been using Python (along with bash et al) for a long time now for my Automation and Web Crawling &amp; Scraping endeavors, I was confused between opting for Flask or extensive Django for a mock Web App but decided to use the <strong>more robust Django</strong> used for back-end programming in Python due to it’s wider functionality compared to the simplistic Flask. It seemed like a perfect fit for a mini hobby project. I used Ubuntu for the Whole project and the same has worked perfectly in Windows too, but the process itself is essentially the same for any Operating System since Python and Django both are cross-platform compatible. Just follow the install instructions carefully, advanced users could even try some compilation. What Django uses is the Model-Template-View (MTV) pattern which is quite similar to the widely used<a href="https://www.tutorialspoint.com/mvc_framework/mvc_framework_introduction.htm"> MVC</a> pattern of making Web Apps. While many would argue that Python is not a good fit for Web Development and I agree with them, it’s better to learn JavaScript if you are looking for a future as a Web Developer, but as an experiment or for those who work closely with Python, Django and Flask are a boon as they are both simple and highly functional even for production quality apps.</p><h3><strong>Where ?</strong></h3><p>Start with downloading and installing Python : <a href="https://www.python.org/downloads/windows/">https://www.python.org/downloads/</a> (and we will get Django package later on)</p><p>A Django Project consists of four major Python files within the project tree namely settings.py, views.py, urls.py, models.py</p><p>The settings file deals with all the Django project settings at system level , the urls file stores all the <a href="https://en.wikipedia.org/wiki/URL">URL</a> patterns for our project that stem out and redirect from our main page. We store our data models in Python Django format in the models file which is then translated into <a href="https://en.wikipedia.org/wiki/SQLite">SQLite</a> operations by default in the process without any explicit need to code. Finally the views file is where the request and response functions are handled and the page serving is done. For other files and their usage go ahead and read the Django documentation and its glossary, as any and every good developer would say : <strong>RTFM</strong> (<em>Read the F*&amp;^ing Manual</em>) :</p><p><a href="https://docs.djangoproject.com/en/2.1/glossary/">https://docs.djangoproject.com/en/2.1/glossary/</a> : old</p><p><a href="https://django.readthedocs.io/en/latest/intro/overview.html">https://django.readthedocs.io/en/latest/intro/overview.html</a> : latest</p><p>Optionally check out the Github repository located at <a href="https://github.com/django/django">https://github.com/django/django</a></p><h3>How ?</h3><p>The whole process from scratch to creating a Registration &amp; Login page in short can be broken down into three simple steps :</p><h3><strong>1 &gt; Creating the Project structure</strong></h3><pre>Be sure to include Python executive in your PATH environment variable and check if pip command works on your terminal. Also substitute Python with whatever executive call works in the terminal such as Py or Python3 etc in case calling Python has no effect.</pre><pre>We make use of something called a virtual environment to make our project locally sustainable without bloating the main Python installation and encapsulate it.</pre><pre>Since `virtualenv` was deprecated I removed the lines present here on it, we will be making use of `venv` : <a href="https://docs.python.org/3/library/venv.html">https://docs.python.org/3/library/venv.html</a></pre><pre>We make a project folder and change our current directory to it by using</pre><pre>$ mkdir djangox</pre><pre>$ cd djangox</pre><pre>Next we set up a virtual environment (a mini python package store if you may) named `denvx` by executing</pre><pre>$ python -m venv denvx</pre><pre>also activate (source) denvx</pre><pre>$ source /denvx/bin/activate</pre><pre>For installing Django framework into our virtual environment(denvx) use pip (pip is the python package manager it comes installed by default in most cases)</pre><pre>$ pip install django</pre><pre>We now start our django-project using</pre><pre>$ django-admin startproject dprojx</pre><pre>also make an django App for Registration and Login page with</pre><pre>$ cd dprojx &amp;&amp; django-admin startapp dappx</pre><pre>Next we create additional folders such as &#39;<em>/templates</em>&#39;, &#39;<em>/media</em>&#39; and &#39;<em>/static</em>&#39; ... for our resources</pre><pre>$ mkdir templates media static media/profile_pics templates/dappx</pre><pre>Django now has models and views created for our app &#39;<em>dappx</em>&#39; which should me migrated for which we use</pre><pre>$ python manage.py migrate<br>$ python manage.py makemigrations dappx<br>$ python manage.py migrate</pre><pre>This migrates the changes for our created project.</pre><p>Your Project Structure should now look something like this :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/306/1*OLfOGshv5gw70-p7YKjv4w.png" /><figcaption>Project Structure</figcaption></figure><h3>2 &gt; Modifying the Project Files</h3><p>Open up <em>dprojx/settings.py</em> and add the lines for other _DIR to look like :</p><pre># dprojx/settings.py</pre><pre># Build paths inside the project like this: os.path.join(BASE_DIR, …)<br>BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))<br>TEMPLATE_DIR = os.path.join(BASE_DIR,&#39;templates&#39;)<br>STATIC_DIR = os.path.join(BASE_DIR,&#39;static&#39;)<br>MEDIA_DIR = os.path.join(BASE_DIR,&#39;media&#39;)</pre><p>Also in the installed apps section you need to add <em>dappx </em>for it to be recognized and registered :</p><pre>INSTALLED_APPS = [<br>    &#39;django.contrib.admin&#39;,<br>    &#39;django.contrib.auth&#39;,<br>    &#39;django.contrib.contenttypes&#39;,<br>    &#39;django.contrib.sessions&#39;,<br>    &#39;django.contrib.messages&#39;,<br>    &#39;django.contrib.staticfiles&#39;,<br>    &#39;dappx&#39;,<br>]</pre><p>Add <em>TEMPLATE_DIR</em> to <em>‘DIRS’: []</em> section too :</p><pre>TEMPLATES = [<br>    {<br>        &#39;BACKEND&#39;: &#39;django.template.backends.django.DjangoTemplates&#39;,<br>        &#39;DIRS&#39;: [TEMPLATE_DIR,],<br>        &#39;APP_DIRS&#39;: True,<br>        &#39;OPTIONS&#39;: {<br>            &#39;context_processors&#39;: [<br>                &#39;django.template.context_processors.debug&#39;,<br>                &#39;django.template.context_processors.request&#39;,<br>                &#39;django.contrib.auth.context_processors.auth&#39;,<br>                &#39;django.contrib.messages.context_processors.messages&#39;,<br>            ],<br>        },<br>    },<br>]</pre><p>Add these lines at the end of our <em>dprojx/settings.py</em> for <em>static</em>, <em>templates</em> and <em>media</em> folder :</p><pre>STATIC_URL = &#39;/static/&#39;<br>STATICFILES_DIRS = [STATIC_DIR,]</pre><pre>MEDIA_ROOT = MEDIA_DIR<br>MEDIA_URL = ‘/media/’</pre><pre>LOGIN_URL = ‘/dappx/user_login/’</pre><p>Next we create the <em>dappx/models.py</em> to be used that will be the basis of our <em>forms.py :</em></p><pre># dappx/models.py</pre><pre>from django.db import models<br>from django.contrib.auth.models import User</pre><pre># Create your models here.</pre><pre>class UserProfileInfo(models.Model):</pre><pre>user = models.OneToOneField(User,on_delete=models.CASCADE)</pre><pre>portfolio_site = models.URLField(blank=True)</pre><pre>profile_pic = models.ImageField(upload_to=&#39;profile_pics&#39;,blank=True)</pre><pre>def __str__(self):<br>  return self.user.username</pre><p>from <em>models.py </em>we now switch to <em>dappx/forms.py </em>:</p><pre># dappx/forms.py</pre><pre>from django import forms<br>from dappx.models import UserProfileInfo<br>from django.contrib.auth.models import User</pre><pre>class UserForm(forms.ModelForm):<br>    password = forms.CharField(widget=forms.PasswordInput())<br>    class Meta():<br>        model = User<br>        fields = (&#39;username&#39;,&#39;password&#39;,&#39;email&#39;)</pre><pre>class UserProfileInfoForm(forms.ModelForm):<br>     class Meta():<br>         model = UserProfileInfo<br>         fields = (&#39;portfolio_site&#39;,&#39;profile_pic&#39;)</pre><p>Register your models in <em>dappx/admin.py</em> too :</p><pre># dappx/admin.py</pre><pre>from django.contrib import admin<br>from dappx.models import UserProfileInfo, User<br># Register your models here.</pre><pre>admin.site.register(UserProfileInfo)</pre><p>The most important bit comes into play now where our View is generated for our <em>.html </em>pages in <em>dappx/views.py</em> :</p><pre># dappx/views.py</pre><pre>from django.shortcuts import render<br>from dappx.forms import UserForm,UserProfileInfoForm<br>from django.contrib.auth import authenticate, login, logout<br>from django.http import HttpResponseRedirect, HttpResponse<br>from django.urls import reverse<br>from django.contrib.auth.decorators import login_required</pre><pre>def index(request):<br>    return render(request,&#39;dappx/index.html&#39;)</pre><pre><a href="http://twitter.com/login_required">@login_required</a><br>def special(request):<br>    return HttpResponse(&quot;You are logged in !&quot;)</pre><pre><a href="http://twitter.com/login_required">@login_required</a><br>def user_logout(request):<br>    logout(request)<br>    return HttpResponseRedirect(reverse(&#39;index&#39;))</pre><pre>def register(request):<br>    registered = False<br>    if request.method == &#39;POST&#39;:<br>        user_form = UserForm(data=request.POST)<br>        profile_form = UserProfileInfoForm(data=request.POST)<br>        if user_form.is_valid() and profile_form.is_valid():<br>            user = user_form.save()<br>            user.set_password(user.password)<br>            user.save()<br>            profile = profile_form.save(commit=False)<br>            profile.user = user<br>            if &#39;profile_pic&#39; in request.FILES:<br>                print(&#39;found it&#39;)<br>                profile.profile_pic = request.FILES[&#39;profile_pic&#39;]<br>            profile.save()<br>            registered = True<br>        else:<br>            print(user_form.errors,profile_form.errors)<br>    else:<br>        user_form = UserForm()<br>        profile_form = UserProfileInfoForm()<br>    return render(request,&#39;dappx/registration.html&#39;,<br>                          {&#39;user_form&#39;:user_form,<br>                           &#39;profile_form&#39;:profile_form,<br>                           &#39;registered&#39;:registered})</pre><pre>def user_login(request):<br>    if request.method == &#39;POST&#39;:<br>        username = request.POST.get(&#39;username&#39;)<br>        password = request.POST.get(&#39;password&#39;)<br>        user = authenticate(username=username, password=password)<br>        if user:<br>            if user.is_active:<br>                login(request,user)<br>                return HttpResponseRedirect(reverse(&#39;index&#39;))<br>            else:<br>                return HttpResponse(&quot;Your account was inactive.&quot;)<br>        else:<br>            print(&quot;Someone tried to login and failed.&quot;)<br>            print(&quot;They used username: {} and password: {}&quot;.format(username,password))<br>            return HttpResponse(&quot;Invalid login details given&quot;)<br>    else:<br>        return render(request, &#39;dappx/login.html&#39;, {})</pre><p>The templates can be arranged now for generating the views, we use four templates for this : <em>‘base.html’, ‘registration.html’, ‘login.html’, ‘index.html’ </em>:</p><p><strong><em>base.html</em></strong> :</p><pre>&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>  &lt;head&gt;<br>    &lt;meta charset=&quot;utf-8&quot;&gt;<br>    &lt;title&gt;Base&lt;/title&gt;<br>    &lt;!-- Latest compiled and minified CSS --&gt;<br>    &lt;link rel=&quot;stylesheet&quot; href=&quot;<a href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css</a>&quot; integrity=&quot;sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u&quot; crossorigin=&quot;anonymous&quot;&gt;</pre><pre>&lt;/head&gt;<br>  &lt;body&gt;<br>    &lt;nav class=&quot;navbar navbar-default navbar-static-top&quot;&gt;<br>      &lt;div class=&quot;container&quot;&gt;<br>        &lt;ul class=&quot;nav navbar-nav&quot;&gt;</pre><pre>{# Django Home Link / Admin Link / Register Link#}<br>          &lt;li&gt;&lt;a class=&quot;navbar-brand&quot; href=&quot;{% url &#39;index&#39; %}&quot;&gt;DJANGO&lt;/a&gt;&lt;/li&gt;<br>          &lt;li&gt;&lt;a class=&quot;navbar-link&quot; href=&quot;{% url &#39;admin:index&#39; %}&quot;&gt;Admin&lt;/a&gt;&lt;/li&gt;<br>          &lt;li&gt;&lt;a class=&quot;navbar-link&quot; href=&quot;{% url &#39;dappx:register&#39; %}&quot;&gt;Register&lt;/a&gt;&lt;/li&gt;     <br>          {# Some logic on what to display for last item#}<br>          {% if user.is_authenticated %}<br>            &lt;li&gt;&lt;a href=&quot;{% url &#39;logout&#39; %}&quot;&gt;Logout&lt;/a&gt;&lt;/li&gt;<br>          {% else %}<br>            &lt;li&gt;&lt;a class=&quot;navbar-link&quot; href=&quot;{% url &#39;dappx:user_login&#39; %}&quot;&gt;Login&lt;/a&gt;&lt;/li&gt;<br>          {% endif %}</pre><pre>&lt;/ul&gt;<br>      &lt;/div&gt;<br>    &lt;/nav&gt;<br>    &lt;div class=&quot;container&quot;&gt;<br>    {% block body_block %}<br>    {% endblock %}<br>    &lt;/div&gt;<br>  &lt;/body&gt;<br>&lt;/html&gt;</pre><p><strong><em>index.html</em></strong> :</p><pre>{% extends &quot;dappx/base.html&quot; %}<br>{% block body_block %}</pre><pre>&lt;div class=&quot;container&quot;&gt;<br>  &lt;div class=&quot;jumbotron&quot;&gt;<br>    &lt;h1&gt;Welcome to the Djungle !&lt;/h1&gt;<br>    {% if user.is_authenticated %}<br>        &lt;h2&gt;Hello {{ user.username }}&lt;/h2&gt;<br>    {% else %}<br>        &lt;h2&gt;Register or Login if you&#39;d like to&lt;/h2&gt;<br>    {% endif %}<br>  &lt;/div&gt;<br>&lt;/div&gt;</pre><pre>{% endblock %}</pre><p><strong><em>login.html</em></strong> :</p><pre>{% extends &#39;dappx/base.html&#39; %}<br>{% block body_block %}<br>  &lt;div class=&quot;container&quot;&gt;<br>    &lt;div class=&quot;jumbotron&quot;&gt;<br>      &lt;h1&gt;Login here :&lt;/h1&gt;</pre><pre>&lt;form method=&quot;post&quot; action=&quot;{% url &#39;dappx:user_login&#39; %}&quot;&gt;<br>          {% csrf_token %}<br>          {# A more &quot;HTML&quot; way of creating the login form#}<br>          &lt;label for=&quot;username&quot;&gt;Username:&lt;/label&gt;<br>          &lt;input type=&quot;text&quot; name=&quot;username&quot; placeholder=&quot;Username&quot;&gt;</pre><pre>&lt;label for=&quot;password&quot;&gt;Password:&lt;/label&gt;<br>          &lt;input type=&quot;password&quot; name=&quot;password&quot; placeholder=&quot;Password&quot;&gt;</pre><pre>&lt;input type=&quot;submit&quot; name=&quot;&quot; value=&quot;Login&quot;&gt;</pre><pre>&lt;/form&gt;</pre><pre>&lt;/div&gt;<br>  &lt;/div&gt;<br>{% endblock %}</pre><p><strong><em>registration.html</em></strong> :</p><pre>{% extends &quot;dappx/base.html&quot; %}<br>{% load static %}</pre><pre>{% block body_block %}<br>  &lt;div class=&quot;container&quot;&gt;<br>    &lt;div class=&quot;jumbotron&quot;&gt;<br>      {% if registered %}<br>        &lt;h1&gt;Thank you for registering!&lt;/h1&gt;<br>      {% else %}<br>        &lt;h1&gt;Register Here&lt;/h1&gt;<br>        &lt;h3&gt;Just fill out the form.&lt;/h3&gt;</pre><pre>&lt;form enctype=&quot;multipart/form-data&quot; method=&quot;POST&quot;&gt;<br>          {% csrf_token %}<br>          {{ user_form.as_p }}<br>          {{ profile_form.as_p }}<br>          &lt;input type=&quot;submit&quot; name=&quot;&quot; value=&quot;Register&quot;&gt;<br>        &lt;/form&gt;<br>        <br>      {% endif %}<br>    &lt;/div&gt;<br>  &lt;/div&gt;<br>{% endblock %}</pre><p>We then register the above urls into our project <em>urls.py</em> files for this create a file <em>dappx/urls.py </em>:</p><pre># dappx/urls.py</pre><pre>from django.conf.urls import url<br>from dappx import views</pre><pre># SET THE NAMESPACE!<br>app_name = &#39;dappx&#39;</pre><pre># Be careful setting the name to just /login use userlogin instead!<br>urlpatterns=[<br>    url(https://medium.com/r&#39;^register/r&#39;^register/$&#39;,views.register,name=&#39;register&#39;#39;,views.register,name=&#39;register&#39;),<br>    url(https://medium.com/r&#39;^user_login/r&#39;^user_login/$&#39;,views.user_login,name=&#39;user_login&#39;#39;,views.user_login,name=&#39;user_login&#39;),<br>]</pre><p>In the main <em>urls.py</em> file the rest of the pattern is specified :</p><pre># dprojx/urls.py</pre><pre>from django.contrib import admin<br>from django.urls import path<br>from django.conf.urls import url,include<br>from dappx import views</pre><pre>urlpatterns = [<br>    path(&#39;admin/&#39;, admin.site.urls),<br>    url(https://medium.com/r&#39;^r&#39;^$&#39;,views.index,name=&#39;index&#39;#39;,views.index,name=&#39;index&#39;),<br>    url(https://medium.com/r&#39;^special/&#39;,views.special,name=&#39;special&#39;),<br>    url(https://medium.com/r&#39;^dappx/&#39;,include(&#39;dappx.urls&#39;)),<br>    url(https://medium.com/r&#39;^logout/r&#39;^logout/$&#39;, views.user_logout, name=&#39;logout&#39;#39;,%20views.user_logout,%20name=&#39;logout&#39;),<br>]</pre><p>By now your project structure should look somewhat like :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/357/1*0WtI9YLjsMpHya_ta1toyw.png" /><figcaption>Project tree</figcaption></figure><p>Again apply the migrations using :</p><pre>$ python manage.py migrate<br>$ python manage.py makemigrations dappx<br>$ python manage.py migrate</pre><p>and create a <a href="https://docs.djangoproject.com/en/1.8/intro/tutorial02/">superuser</a> (an admin for the project basically) using the command :</p><pre>$ python manage.py createsuperuser</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/210/1*NaVbua2uDsbft1j4tQTRHw.png" /><figcaption>Sample of what appears</figcaption></figure><p>Enter your desired <em>username</em>, <em>email</em> and <em>password.</em></p><h3>3 &gt; Firing Up the Server !</h3><p>Now you are ready to start your server :</p><pre>$ python manage.py runserver</pre><pre>Performing system checks...</pre><pre>System check identified no issues (0 silenced).<br>July 18, 2018 - 04:03:03<br>Django version 2.0.7, using settings &#39;dprojx.settings&#39;<br>Starting development server at <a href="http://127.0.0.1:8000/">http://127.0.0.1:8000/</a><br>Quit the server with CONTROL-C.</pre><p>There we have it</p><p>Navigate to <a href="http://127.0.0.1:8000/">http://127.0.0.1:8000/</a> to see your Django powered site in action.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3pYumQ2bEAMLKT83bhkEwQ.png" /><figcaption>Welcome Page</figcaption></figure><p>Here are the Login and Registration pages :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gf5B6ib3ePdEM0Ke_wtWkQ.png" /><figcaption>Registration Page</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AIUbvDFB1XChsxwXc2-dDQ.png" /><figcaption>Login Page</figcaption></figure><h3><strong>Edit (01|04|19) :</strong></h3><p>For some of you who has been asking about an error like ModelForm has no Model specified. That error was due to a indentation error while copy-pasting the code to your editor and can be solved by going into the indentation settings and replacing all Tabs to four spaces or vice-versa. Also remember Django and Python is case-sensitive so Model and modelare different. If the error persists try to look for indentation error in the specific file and line and correct it manually as <a href="https://medium.com/u/e5f26da088b0">YooYoung Ko</a> pointed out. Cheers</p><h3><strong>Edit (09|04|20) :</strong></h3><p>I had time to check this article out due to the Corona pandemic going on and me randomly checking old stuff, hope you all and your families are safe, thanks to <a href="https://medium.com/u/f13c0a1629a6">George Ofonedu</a> <a href="https://medium.com/@georgeofonedu/the-tutorial-is-based-on-django-2-0-d21b0d851756">this</a> guide is compatible with Django 3.0 with minor changes that is `staticfiles` being renamed to `static` in Django templates with version `3.0` and I have edited it to reflect that change. Also many many thanks for the 1k+ claps I never expected this write up to go that far since it was just a hobby project I did.</p><h3>Edit (22|06|21) :</h3><p>The Corona Pandemic is still raging on, hope everyone is safe, my family just recovered from the disease so I know how tough it can be. I have tested the code to be compatible since I had some free time. Point out if there are any errors., do check out my other articles too if you found this useful.</p><h3><strong>Edit (03|06|24) :</strong></h3><p>This page is quite useful if you want to learn how to make parts of the code and process reusable : <a href="https://django.readthedocs.io/en/stable/intro/reusable-apps.html">https://django.readthedocs.io/en/stable/intro/reusable-apps.html</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd33dc7a6c67" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>