<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Alekhyo Banerjee on Medium]]></title>
        <description><![CDATA[Stories by Alekhyo Banerjee on Medium]]></description>
        <link>https://medium.com/@alekhyo?source=rss-f09f736c263c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*g7nkjv5ddj6FPmCMKQrOLA.jpeg</url>
            <title>Stories by Alekhyo Banerjee on Medium</title>
            <link>https://medium.com/@alekhyo?source=rss-f09f736c263c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 10:34:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@alekhyo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Computational Complexity of PCA]]></title>
            <link>https://alekhyo.medium.com/computational-complexity-of-pca-4cb61143b7e5?source=rss-f09f736c263c------2</link>
            <guid isPermaLink="false">https://medium.com/p/4cb61143b7e5</guid>
            <dc:creator><![CDATA[Alekhyo Banerjee]]></dc:creator>
            <pubDate>Mon, 31 Aug 2020 18:23:59 GMT</pubDate>
            <atom:updated>2020-10-25T15:21:36.628Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/348/0*_aSh36wo_gVQ9Z3X" /></figure><p>If you know the basics PCA you probably know that PCA is computationally extensive. This article gives an intuition behind all the extensive mathematical calculation behind PCA. Let’s dive deep.</p><p>A simple way of computing PCA of a matrix X is to compute the eigenvalue decomposition of its covariance matrix. Given N ×D matrix X, a target dimensionality d, the algorithm computes an N ×d matrix V such that the columns of V are the principal components of X.</p><p>The algorithm for computing the eigenvalue decomposition of the covariance matrix can be summarized in the following steps:</p><ul><li><strong>Step1: Compute the mean-centred matrix Xc: </strong>The mean-centred matrix Xc is computed by subtracting the vector of all the column means of matrix X from each row of X.</li><li><strong>Step 2: Compute the covariance matrix Cv = X′ c ∗Xc: </strong>The covariance matrix is computed by multiplying the mean-centred matrix Xc with its transpose. This is a computationally intensive step because it requires multiplying two matrices where typically none of them can fit in memory.</li><li><strong>Step 3: Compute the eigenvalue decomposition of Cv: </strong>Eigenvalues λi and eigenvectors vi satisfy the relation: Cv ∗ λi = λi ∗ vi, where λi is i th eigenvalue and vi is its corresponding eigenvector. Putting the above formula in the matrix form: Cv ∗Λ = Λ∗V, where Λ is a diagonal matrix whose elements are the eigenvalues of the covariance matrix Cv, and V is the matrix whose columns are the corresponding eigenvectors. The eigenvalue decomposition of the matrix Cv is given by Cv = Λ∗V ∗Λ −1.</li><li><strong>Step 4: Get the principal components: </strong>The principal components of the input matrix X are the eigenvectors associated with the largest eigenvalues. Since the eigenvalues in the diagonal matrix Λ are sorted in decreasing order, then the first d vectors in the matrix V are the principal components of X: V = (v1, v2,…, vd).</li></ul><h3>Time Complexity:</h3><p>The algorithm has two computationally intensive steps:</p><ol><li>Computing the covariance matrix</li><li>Computing the eigenvalue decomposition of the covariance matrix</li></ol><p>3. The computational complexity of the covariance matrix computations is O(ND×min(N, D)) which is a result of multiplying two matrices of size D×N and N ×D, respectively. The other computationally intensive computation is the eigenvalue decomposition. The worst-case complexity of such algorithms is O(D³) for a matrix of size D×D. Therefore the overall complexity is O(ND×min(N, D) +D³).</p><p>Libraries like scikit-learn use<a href="https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html"> Singular Value Decomposition (SVD)</a> for Eigenvalue decomposition. SVD is more optimized, numerically accurate and holds for every matrix, unlike eigen decomposition.</p><p>Singular Value Decomposition can be described as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/0*98axR0PSdU_xc1zN" /></figure><p>How to connect SVD with eigenvalue decomposition?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/0*K7YHtFQI9J29ePRp" /></figure><p>The column U gives the eigenvectors. Finding the top k singular values of X is equivalent to finding the <strong>top k eigenvectors</strong> of the covariance matrix.</p><p>Reference:</p><ol><li><a href="https://www.slideshare.net/YounesCharfaoui/principal-component-analysis-code-and-time-complexity-127431697">https://www.slideshare.net/YounesCharfaoui/principal-component-analysis-code-and-time-complexity-127431697</a></li><li><a href="https://stackoverflow.com/questions/20507646/how-is-the-complexity-of-pca-ominp3-n3">https://stackoverflow.com/questions/20507646/how-is-the-complexity-of-pca-ominp3-n3</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4cb61143b7e5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interview Questions on PCA]]></title>
            <link>https://alekhyo.medium.com/interview-questions-on-pca-9cdc96ddaa9f?source=rss-f09f736c263c------2</link>
            <guid isPermaLink="false">https://medium.com/p/9cdc96ddaa9f</guid>
            <dc:creator><![CDATA[Alekhyo Banerjee]]></dc:creator>
            <pubDate>Mon, 31 Aug 2020 18:22:21 GMT</pubDate>
            <atom:updated>2020-08-31T18:22:21.586Z</atom:updated>
            <content:encoded><![CDATA[<p>Here’s a bunch of Interview Questions asked on Principal Component Analysis</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/461/0*CesR-qiclyzNkhjk" /></figure><ol><li><strong>Explain the Curse of Dimensionality?</strong></li></ol><p>The curse of dimensionality refers to all the problems that arise working with data in the higher dimensions. As the number of features increase, the number of samples increases, hence, the model becomes more complex. The more the number of features, the more the chances of overfitting. A machine learning model that is trained on a large number of features, gets increasingly dependent on the data it was trained on and in turn overfitted, resulting in poor performance on real data, beating the purpose. The fewer features our training data has, the lesser assumptions our model makes and the simpler it will be.</p><p><strong>2. Why do we need dimensionality reduction? What are its drawbacks?</strong></p><p>In Machine Learning, dimension refers to the number of features. Dimensionality reduction is simply, the process of reducing the dimension of your feature set.</p><p>Advantages of Dimensionality Reduction:</p><ol><li>Less misleading data means model accuracy improves.</li><li>Fewer dimensions mean less computing. Less data means that algorithms train faster.</li><li>Less data means less storage space required.</li><li>Removes redundant features and noise.</li><li>Dimensionality Reduction helps us visualize the data on 2D plots or 3D plots.</li></ol><p>Drawbacks of Dimensionality Reduction are:</p><ol><li>Some information is lost, possibly degrading the performance of subsequent training algorithms.</li><li>It can be computationally intensive.</li><li>Transformed features are often hard to interpret.</li><li>It makes the independent variables less interpretable.</li></ol><p><strong>3. Explain Principal Component Analysis, assumptions, equations.</strong></p><p>Principal Component Analysis (PCA) is an unsupervised dimensionality reduction algorithm. It identifies the hyperplane that lies closest to the data, and then it projects the data onto it preserving the variance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/0*qKESl4pUx0p6vqrv" /></figure><p>Here is a result of the projection of a dataset onto each of these axes. As you can see, the projection onto the solid line preserves the maximum variance, while the projection onto the dotted line preserves very little variance, and the projection onto the dashed line preserves an intermediate amount of variance.</p><p>PCA selects the axis which preserves maximum variance in the training set. PCA finds as many axes as the number of dimensions such that every axis is orthogonal to each other.</p><p><em>Equations:</em></p><ol><li>First, we calculate the covariance matrix</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/289/0*MHm2ShJHsz1Cicj3" /></figure><p>2. Decompose the covariance matrix and find the eigenvalues and corresponding eigenvectors</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/0*TzgRrnwnbovbZbWi" /></figure><p>Eigen Value and Eigen Vector</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/0*eJboHTKKY7vDj-b2" /></figure><p>Decompose the Covariance Matrix</p><p>3. We arrange the eigenvalues as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/0*SB3UV1O2qH5KkbaZ" /></figure><p>Each Eigenvalue corresponds to a principal component. The highest eigenvalue corresponds to the 1st Principal Component and so on.</p><p>Assumptions:</p><ul><li>There needs to be a <strong>linear relationship between all variables</strong>. The reason for this assumption is that a PCA is based on Pearson correlation coefficients, and as such, there needs to be a linear relationship between the variables</li><li>You should have <strong>sampling adequacy</strong>, which simply means that for PCA to produce a reliable result, large enough sample sizes are required.</li><li>Your data should be <strong>suitable for data reduction</strong>. Effectively, you need to have adequate correlations between the variables to be reduced to a smaller number of components.</li><li>There should be <strong>no significant outliers</strong>.</li></ul><p><strong>4. Can PCA be used to reduce the dimensionality of a highly nonlinear dataset?</strong></p><p>PCA can be used to significantly reduce the dimensionality of most datasets, even if they are highly nonlinear because it can at least get rid of useless dimensions. However, if there are no useless dimensions, reducing dimensionality with PCA will lose too much information.</p><p><strong>5. Limitations of PCA?</strong></p><ol><li>Doesn’t work well for non linearly correlated data.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/0*LGEeHI2pkqla4xEp" /></figure><p>Here the data points are not linearly correlated. PCA won’t work well.</p><p>2. PCA always finds orthogonal principal components. Sometimes, our data demands non-orthogonal principal components to represent the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/437/0*dia0hCU8HeiAH9D8" /></figure><p>The green colour vectors are principal components. But, the actual maximum variance directions are blue colour vectors. Standard PCA fails to find that vectors.</p><p>3. PCA always considered the low variance components in the data as noise and recommend us to throw away those components. But, sometimes those components play a major role in a supervised learning task.</p><p>4. If the variables are correlated, PCA can achieve dimension reduction. If not, PCA just orders them according to their variances</p><p><strong>6. Is rotation necessary in PCA? If yes, Why? What will happen if you don’t rotate the components?</strong></p><p>Yes, rotation (orthogonal) is necessary to account the maximum variance of the training set. If we don’t rotate the components, the effect of PCA will diminish and we’ll have to select more number of components to explain variance in the training set.</p><p><strong>7. Is it important to standardize before applying PCA?</strong></p><p>PCA finds new directions based on the covariance matrix of original variables. Since the covariance matrix is sensitive to the standardization of variables. Usually, we do standardization to assign equal weights to all the variables. If we use features of different scales, we get misleading directions. But, it is not necessary to standardize the variables, if all the variables are on the same scale.</p><p><strong>8. Should one remove highly correlated variables before doing PCA?</strong></p><p>No, PCA loads out all highly correlated variables on the same Principal Component(Eigenvector), not different ones.</p><p><strong>9. What will happen when eigenvalues are roughly equal?</strong></p><p>If all eigenvectors are same then PCA won’t be able to select the principal components because in that case, all principal components are equal.</p><p><strong>10. How can you evaluate the performance of a dimensionality reduction algorithm on your dataset?</strong></p><p>Intuitively, a dimensionality reduction algorithm performs well if it eliminates a lot of dimensions from the dataset without losing too much information. Alternatively, if you are using dimensionality reduction as a preprocessing step before another Machine Learning algorithm (e.g., a Random Forest classifier), then you can simply measure the performance of that second algorithm; if dimensionality reduction did not lose too much information, then the algorithm should perform just as well as when using the original dataset.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9cdc96ddaa9f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Computational Complexity of SVM]]></title>
            <link>https://alekhyo.medium.com/computational-complexity-of-svm-4d3cacf2f952?source=rss-f09f736c263c------2</link>
            <guid isPermaLink="false">https://medium.com/p/4d3cacf2f952</guid>
            <dc:creator><![CDATA[Alekhyo Banerjee]]></dc:creator>
            <pubDate>Mon, 31 Aug 2020 18:20:16 GMT</pubDate>
            <atom:updated>2020-08-31T18:20:16.491Z</atom:updated>
            <content:encoded><![CDATA[<p>If you are pursuing Computer Science, probably you must have come across the notations of time complexity.</p><p>There’s not a single technical interview round that’s not going to question you on running time complexity of an algorithm.</p><p><strong>The time complexity of an algorithm is the total amount of time required by an algorithm to complete its execution.</strong> In simple words, every piece of code we write takes time to execute. The time taken by any piece of code to run is known as the time complexity of that code. The lesser the time complexity, the faster the execution.</p><p><em>WHY SHOULD WE CARE ABOUT TIME?</em></p><p>One might argue that why is it even required when we have powerful processors today. Well, it’s a necessity for a programmer to optimise the code. In the real world, you are dealing with thousands or even millions of data. Writing algorithm which has poor running time might hurt you or the organisation you are working for.</p><p>Time Complexity is often measured in terms of :</p><ul><li><strong>Big Oh (O): worst-case running time</strong></li><li><strong>Big Omega (Ω): best-case running time</strong></li><li><strong>Big Theta (Θ): both best and worst-case running time</strong></li></ul><h4>Usually, we are concerned about the upper bound of a computer program i.e the worst-case running time.</h4><h4>The same argument applies to Machine Learning Algorithms. Selecting the best ML algorithm in terms of computational complexity for a given problem is an important part of an ML practitioner. This article will be very specific in context to SVM.</h4><p>Both, Soft Margin and hard Margin formulation of SVM is a <a href="https://mathworld.wolfram.com/ConvexFunction.html">convex</a> <a href="https://optimization.mccormick.northwestern.edu/index.php/Quadratic_programming">quadratic programming</a> problem with linear constriants.</p><p>Generally, used techniques for quadratic programming are very slow. Even using gradient descent is computationally expensive, here Stochastic Gradient comes into play.</p><p>Quadratic Programming problems are optimised using Stochastic Gradient Descent. Though it takes more steps each individual step is less computationally expensive than gradient descent.</p><p><strong>Stochastic Gradient Descent (SGD)</strong> is a simple yet very efficient approach to fitting linear classifiers and regressors under convex loss functions such as (linear)<a href="https://en.wikipedia.org/wiki/Support_vector_machine"> Support Vector Machines</a> and<a href="https://en.wikipedia.org/wiki/Logistic_regression"> Logistic Regression</a>. Even though SGD has been around in the machine learning community for a long time, it has received a considerable amount of attention just recently in the context of large-scale learning.</p><p>SGD has been successfully applied to large-scale and sparse machine learning problems often encountered in text classification and natural language processing. Given that the data is sparse, the classifiers in this module easily scale to problems with more than 1⁰⁵ training examples and more than 1⁰⁵ features.</p><p>Support Vector Machine internally <a href="http://cs229.stanford.edu/materials/smo.pdf">SMO(Sequential Minimal Optimisation)</a> is used. libsvm is the best library used for optimising SVMs which implements SMO. Even <a href="https://scikit-learn.org/stable/modules/svm.html">Scikit-learn</a> uses libsvm for training SVM models.</p><p>Training time for SVM can be given as O(nd²) if d&lt;n</p><p><strong>Where d is the dimension</strong></p><p><strong>n is the no. of data points</strong></p><p>When you are dealing with a large dataset, the training time will be high. SVM is not used in such scenario typically.</p><p>The Run time (predicting time) can be given as O(kd)</p><p><strong>where k is the no.of support vectors(0&lt;k&lt;=d)</strong></p><p><strong>d is the no of data points.</strong></p><p>Reference:<a href="http://cs229.stanford.edu/materials/smo.pdf"> http://cs229.stanford.edu/materials/smo.pdf</a></p><p><a href="https://www.cs.utah.edu/~zhe/pdf/lec-19-2-svm-sgd-upload.pdf">https://www.cs.utah.edu/~zhe/pdf/lec-19-2-svm-sgd-upload.pdf</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4d3cacf2f952" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interview Questions on SVM]]></title>
            <link>https://alekhyo.medium.com/interview-questions-on-svm-bf13e5fbcca8?source=rss-f09f736c263c------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf13e5fbcca8</guid>
            <dc:creator><![CDATA[Alekhyo Banerjee]]></dc:creator>
            <pubDate>Mon, 31 Aug 2020 18:18:28 GMT</pubDate>
            <atom:updated>2020-08-31T18:18:28.432Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/719/0*ub920ECzauIwZUaV" /></figure><p>Support Vector Machine” (SVM) is a supervised machine learning algorithm which can be used for both classification or regression problems.</p><p>SVMs are particularly well suited for classification of complex but small- or medium-sized datasets.</p><p>We’ll talk about some Interview questions related to SVMs.</p><ol><li><strong>Explain SVM to a non-technical person.</strong></li></ol><p>Explanation: Suppose you have to construct a bidirectional road. Now you have to make a dividing line. The optimal approach would be to make margins on the sides and draw an equidistant line from both the margins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*15FzFJjgaaf6iznO" /></figure><p>This is exactly how SVM tries to classify points by finding an optimal centre line (technically called as hyperplane).</p><p><strong>2. Can you explain SVM?</strong></p><p>Explanation: Support vector machines is a supervised machine learning algorithm which works both on classification and regression problems. It tries to classify data by finding a hyperplane that maximizes the margin between the classes in the training data. Hence, SVM is an example of a large margin classifier.</p><p>The basic idea of support vector machines:</p><ul><li>Optimal hyperplane for linearly separable patterns</li><li>Extend to patterns that are not linearly separable by transformations of original data to map into new space(i.e the kernel trick)</li></ul><p><strong>3. What is the geometric intuition behind SVM?</strong></p><p>Explanation: If you are asked to classify two different classes. There can be multiple hyperplanes which can be drawn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/0*j6b6qNc-E0RfBxFj" /></figure><p>SVM chooses the hyperplane which separates the data points as widely as possible. SVM draws a hyperplane parallel to the actual hyperplane intersecting with the first point of class A (also known as Support Vectors) and another hyperplane parallel to the actual hyperplane intersecting with the first point of class B. SVM tries to maximize these margins. Eventually, this margin maximization improves the model’s accuracy on unseen data.</p><p><strong>4. How would explain Convex Hull in light of SVMs?</strong></p><p>Explanation: We simply build a convex hull for class A and class B and draw a perpendicular on the shortest distance between the closest points of both these hulls.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/0*0f3JsP3NsoWhYwKs" /></figure><p><strong>5. What do know about Hard Margin SVM and Soft Margin SVM?</strong></p><p>Explanation: If a point Xi satisfies the equation <strong><em>Yi(WT*Xi +b) ≥ 1, </em></strong>then Xi is correctly classified else incorrectly classified. So we can see that if the points are linearly separable then only our hyperplane is able to distinguish between them and if any outlier is introduced then it is not able to separate them. So these type of SVM is called<strong><em> hard margin SVM </em></strong><em>(since we have very strict constraints to correctly classify each and every data point).</em></p><p>To overcome this, we introduce a term<strong><em> ( ξ ) </em></strong>(pronounced as Zeta)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/183/0*Mimj5tNTj0RoCpRv" /></figure><p><em>if ξi= 0, the points can be considered as correctly classified.</em></p><p><em>if ξi&gt; 0 , Incorrectly classified points.</em></p><p><strong>6. What is Hinge Loss?</strong></p><p>Explanation: Hinge Loss is a loss function which penalises the SVM model for inaccurate predictions.</p><p>If<strong> <em>Yi(WT*Xi +b) ≥ 1</em></strong>, hinge loss is ‘<strong>0</strong>’ i.e the points are correctly classified. When</p><p><strong><em>Yi(WT*Xi +b) </em>&lt; 1</strong>, then hinge loss increases massively.</p><p>As <strong><em>Yi(WT*Xi +b) </em></strong>increases with every misclassified point, the upper bound of hinge loss {<strong>1- <em>Yi(WT*Xi +b)</em></strong>} also increases exponentially.</p><p>Hence, the points that are farther away from the decision margins have a greater loss value, thus penalising those points.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/273/0*ZnoTexvPg-6hGTHB" /></figure><p>We can formulate hinge loss as <strong>max[0, 1- <em>Yi(WT*Xi +b)</em>]</strong></p><p><strong>7. Explain the Dual form of SVM formulation?</strong></p><p>Explanation: The aim of the Soft Margin formulation is to minimize</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/131/0*ijBJ9cN63ydw0Nsv" /></figure><p>subject to</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/431/0*W1bCbvkRYDZjtoMU" /></figure><p>This is also known as the primal form of SVM.</p><p>The duality theory provides a convenient way to deal with the constraints. The dual optimization problem can be written in terms of dot products, thereby making it possible to use kernel functions.</p><p>It is possible to express a different but closely related problem, called its dual problem. The solution to the dual problem typically gives a lower bound to the solution of the primal problem, but under some conditions, it can even have the same solutions as the primal problem. Luckily, the SVM problem happens to meet these conditions, so you can choose to solve the primal problem or the dual problem; both will have the same solution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*d7YVJ-Pb3dj2m-8f" /></figure><p><strong>8. What’s the “kernel trick” and how is it useful?</strong></p><p>Explanation: Earlier we have discussed applying SVM on linearly separable data but it is very rare to get such data. Here, kernel trick plays a huge role. The idea is to map the non-linear separable data-set into a higher dimensional space where we can find a hyperplane that can separate the samples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/0*ZnINGVLyQZfrcZYG" /></figure><p>It reduces the complexity of finding the mapping function. So, <strong>Kernel function defines the inner product in the transformed space. </strong>Application of the kernel trick is not limited to the SVM algorithm. Any computations involving the dot products (x, y) can utilize the kernel trick.</p><p><strong>9. What is Polynomial kernel?</strong></p><p>Explanation:<strong> Polynomial kernel</strong> is a<a href="https://en.wikipedia.org/wiki/Kernel_function"> kernel function</a> commonly used with<a href="https://en.wikipedia.org/wiki/Support_vector_machine"> support vector machines</a> (SVMs) and other<a href="https://en.wikipedia.org/wiki/Kernel_trick"> kernelized</a> models, that represents the similarity of vectors (training samples) in a feature space over polynomials of the original variables, allowing learning of non-linear models.</p><p>For <em>d-degree</em> polynomials, the polynomial kernel is defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/0*WJ62UWXadGyU2r2I" /></figure><p><strong>10. What is RBF-Kernel?</strong></p><p>Explanation:</p><p>The RBF kernel on two samples <strong>x</strong> and <strong>x’</strong>, represented as feature vectors in some <em>input space</em>, is defined as</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/0*OjKNCngZPHvDtkws" /></figure><p>||x-x’||² recognized as the<a href="https://en.wikipedia.org/wiki/Euclidean_distance#Squared_Euclidean_distance"> squared Euclidean distance</a> between the two feature vectors. sigma is a free parameter.</p><p><strong>11. Should you use the primal or the dual form of the SVM problem to train a model on a training set with millions of instances and hundreds of features?</strong></p><p>Explanation: This question applies only to linear SVMs since kernelized can only use the dual form. The computational complexity of the primal form of the SVM problem is proportional to the number of training instances m, while the computational complexity of the dual form is proportional to a number between m² and m³. So, if there are millions of instances, you should use the primal form, because the dual form will be much too slow.</p><p><strong>12. Explain about SVM Regression?</strong></p><p>Explanation: The Support Vector Regression (SVR) uses the same principles as the SVM for classification, with only a few minor differences. First of all, because the output is a real number it becomes very difficult to predict the information at hand, which has infinite possibilities. In the case of regression, a margin of tolerance (epsilon) is set in approximation to the SVM</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/0*eXs0bEy10djpwu7d" /></figure><p><strong>13. Give some situations where you will use an SVM over a RandomForest Machine Learning algorithm.</strong></p><p>Explanation:</p><ul><li>The main reason to use an SVM instead is that the problem might not be linearly separable. In that case, we will have to use an SVM with a non-linear kernel (e.g. RBF).</li><li>Another related reason to use SVMs is if you are in a higher-dimensional space. For example, SVMs have been reported to work better for text classification.</li></ul><p><strong>14. What is the role of C in SVM? How does it affect the bias/variance trade-off?</strong></p><p>Explanation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/131/0*i6T4zRe2s5qN_Y4a" /></figure><p>In the given Soft Margin Formulation of SVM, C is a hyperparameter.</p><p><strong>C hyperparameter </strong>adds a penalty for each misclassified data point.</p><p>Large Value of parameter C implies a small margin, there is a tendency to overfit the training model.</p><p>Small Value of parameter C implies a large margin which might lead to underfitting of the model.</p><p><strong>15. SVM being a large margin classifier, is it influenced by outliers?</strong></p><p>Explanation: Yes, if C is large, otherwise not.</p><p><strong>16. In SVM, what is the angle between the decision boundary and theta?</strong></p><p>Explanation: Decision boundary is a plane having equation Theta1*x1+Theta2*x2+……+c = 0, so as per the property of a plane, it’s coefficients vector is normal to the plane. Hence, the Theta vector is perpendicular to the decision boundary.</p><p><strong>17. Can we apply the kernel trick to logistic regression? Why is it not used in practice then?</strong></p><p>Explanation:</p><ol><li>Logistic Regression is computationally more expensive than SVM — O(N³) vs O(N²k) where k is the number of support vectors.</li><li>The classifier in SVM is designed such that it is defined only in terms of the support vectors, whereas in Logistic Regression, the classifier is defined over all the points and not just the support vectors. This allows SVMs to enjoy some natural speed-ups (in terms of efficient code-writing) that is hard to achieve for Logistic Regression.</li></ol><p><strong>18. What is the difference between logistic regression and SVM without a kernel?</strong></p><p>Explanation: They differ only in the implementation . SVM is much more efficient and has good optimization packages.</p><p><strong>19. Can any similarity function be used for SVM?</strong></p><p>Explanation: No. It has to have to satisfy <a href="https://towardsdatascience.com/understanding-support-vector-machine-part-2-kernel-trick-mercers-theorem-e1e6848c6c4d">Mercer’s theorem</a>.</p><p><strong>20. Does SVM give any probabilistic output?</strong></p><p>Explanation: SVMs do not directly provide probability estimates, these are calculated using an expensive five-fold cross-validation</p><p>Reference:</p><ol><li><a href="https://www.kdnuggets.com/2016/07/support-vector-machines-simple-explanation.html">https://www.kdnuggets.com/2016/07/support-vector-machines-simple-explanation.html</a></li><li><a href="http://web.mit.edu/6.034/wwwbob/svm-notes-long-08.pdf">http://web.mit.edu/6.034/wwwbob/svm-notes-long-08.pdf</a></li><li><a href="https://en.wikipedia.org/wiki/Support_vector_machine">https://en.wikipedia.org/wiki/Support_vector_machine</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf13e5fbcca8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linear Regression from Scratch]]></title>
            <link>https://medium.com/analytics-vidhya/linear-regression-from-scratch-98688848bd4e?source=rss-f09f736c263c------2</link>
            <guid isPermaLink="false">https://medium.com/p/98688848bd4e</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ordinary-least-square]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[gradient-descent]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Alekhyo Banerjee]]></dc:creator>
            <pubDate>Mon, 27 Jul 2020 12:58:43 GMT</pubDate>
            <atom:updated>2020-07-27T12:58:43.238Z</atom:updated>
            <content:encoded><![CDATA[<p>Linear Regression needs no introduction. It’s the ‘Hello World’ of Machine Learning World. Let’s deep dive into the use cases of Linear Regression and mathematical intuition behind it.</p><p>We are in a world driven by data. As commonly said,<strong> </strong><a href="https://www.economist.com/leaders/2017/05/06/the-worlds-most-valuable-resource-is-no-longer-oil-but-data"><strong><em>‘Data is the new oil’</em></strong></a>. One of the most important types of data analysis is regression. Regression aims to build a mathematical model that can be used to predict the values of a <em>dependent variable</em> based upon the values of an <em>independent variable</em>. If you have ever taken an introductory statistics course in college, likely the final topic you covered was linear regression.</p><p>Regression is a fundamental problem in machine learning and regression problems appear in a diverse range of research areas and application including time-series analysis, control and robotics, optimization, and deep learning applications.</p><p>Linear regression is one of the simplest supervised learning algorithms in our toolkit. Linear models make a prediction using a linear function of the input features. It is a common and useful method of making predictions when the target vector is a quantitative value (e.g., home price, Salary, etc).</p><p>There are two types of linear regression, simple linear regression and multiple linear regression:</p><ol><li><strong>Univariate linear regression:</strong> A single independent variable is used to predict the value of a dependent variable.</li><li><strong>Multivariate linear regression:</strong> Two or more independent variables are used to predict the value of a dependent variable.</li></ol><p>The only difference lies in the number of independent variables. In both cases, there is only a single dependent variable. We will focus on univariate linear regression for the sake of understanding and simplicity.</p><p><strong>Which columns to identify as Dependent or Independent?</strong></p><p>The independent variable of a dataset cannot be predicted based on a mathematical model or so. The dependent variable, also known as ‘Target Values’ are the observations we get. It is <em>dependent</em> on the independent variables. Let’s understand using an example</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*sm0nHxjExbflLdcdN9nOAg.png" /></figure><p>Here, <strong>Experience</strong> column is <em>Independent variable</em> and <strong>Salary</strong> column is <em>Dependent variable</em></p><p>Let’s visualise the data by plotting them on a graph</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8I6awma1nmpyx49p6meagg.png" /></figure><p>If we can find a best-fitted line y=f(x) we can predict a value of y corresponding to a value of x.</p><p>Equation of a straight line is given by;</p><p>y= m*x + b,</p><p>where m is the slope</p><p>b is the intercept,</p><p>You might be wondering various lines can be fitted for various values of slope and intercept. So how to choose the <strong>‘One’</strong> curve among all?</p><p>To tackle this uncertainty, we introduce the concept of Sum of Squared errors or sum of squared residuals which is given by the formula</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*jtyzyeRl_p0lfVAE9n19xw.png" /></figure><p>Residual of a point is the observed value of the dependent variable (<em>y</em>) and the predicted value (<em>ŷ). </em>An observed value above the predicted value is positive, below the predicted is negative so we take the sum of the square of residual of each point such that large residuals are penalised.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*AH11fnsdgf8cdB_NB4ti5g.png" /></figure><p>The line with a minimum sum of the squared residual is the <strong><em>BEST FIT LINE.</em></strong></p><blockquote>Is it really necessary to fit a straight line? Why can’t we plot a polynomial curve with various degrees which fits all the data points? Isn’t it preferable?</blockquote><p>No, our model suffers from overfitting. The term <a href="https://medium.com/@martinezbielosdaniel/bias-variance-tradeoff-overfitting-and-underfitting-c63799cb4851"><strong><em>overfitting</em></strong></a> refers to a model that fits very well to the data with which it is trained, but it poorly generalizes them, meaning that when faced with new values, the model yields poor results.</p><p>Our objective is to find a value for slope and intercept such that the sum of the squared residual is minimum. The simplest and most common approach is, using Ordinary Least Square method</p><p>The formula for slope and intercept using <strong>Ordinary Least Square;</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/187/1*O_SlcKJkThbGJsiRdLiswA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/282/1*3nnh8AKMyzYoNm53ANAu5g.png" /></figure><p>Let’s get into the code section:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e03kysqYW9dlDXiKwFqnTg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FlzhUWlwbKY-UTmtY8MIyQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JwPu2mVkDC0ZvT1Yslildw.png" /></figure><p>Now, our model is trained. Let’s predict</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*01Ypng0F3JPv0T3kkgQ74A.png" /></figure><p>Tada !! We are through the basics of linear regression.</p><p>CONCLUSION:</p><p>Ordinary Least Square method is easy to understand and implement but it takes its own sweet time to train the model. It is computationally extensive. It is sensitive to unusual data points. Outliers can sometimes skew the results. <a href="https://youtu.be/sDv4f4s2SB8"><strong>Gradient descent</strong></a> is another algorithm for linear regression which is computationally faster and saves a lot of time on calculations which we shall cover in an upcoming article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98688848bd4e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/linear-regression-from-scratch-98688848bd4e">Linear Regression from Scratch</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Importance of Distance Metrics in Machine Learning]]></title>
            <link>https://alekhyo.medium.com/importance-of-distance-metrics-in-machine-learning-79518a7db822?source=rss-f09f736c263c------2</link>
            <guid isPermaLink="false">https://medium.com/p/79518a7db822</guid>
            <category><![CDATA[hyperparameter-tuning]]></category>
            <category><![CDATA[unsupervised-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[k-nearest-neighbours]]></category>
            <dc:creator><![CDATA[Alekhyo Banerjee]]></dc:creator>
            <pubDate>Sat, 30 May 2020 18:47:55 GMT</pubDate>
            <atom:updated>2020-06-07T16:33:17.714Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u8l4Zeo3x3bKGSTyBQgP5w.jpeg" /></figure><p>You read the title right !!! You might be wondering how ‘Distance’ and ‘Machine Learning’ are related? Am I going out of context?</p><p>No, if you have hands-on experience with Machine Learning algorithms, undoubtedly you came across ‘Distance’ as a parameter. This blog focuses on the need for distance metrics in Machine learning and it’s use cases so that you appreciate the concept more.</p><h3><strong>Distance Metrics</strong></h3><p>The metric is a function that defines a concept of <em>distance</em> between any two members of the set, which are usually called points.</p><p>The metric satisfies a few simple properties.</p><h4>* Non-negativity:</h4><p>d(i,j)≥0: Distance is a non-negative number.</p><h4>* Identity of indiscernible:</h4><p>d(i, i)=0: The distance of an object to itself is 0.</p><h4>* Symmetry:</h4><p>d(i,j)=d(j,i): Distance is a symmetric function.</p><h4>* Triangle Inequality:</h4><p>d(i,j)≤d(i,k)+d(k,j)</p><p>Intuitively, distance can be thought of as a measure of similarity, so two closest observations can be categorized similarly.</p><p>A supervised algorithm, K-Nearest Neighbors, and Unsupervised algorithms like K-mean clustering/Hierarchical clustering use distance metrics to understand patterns between data points.</p><p>Various distance metrics have their use cases, but it is important to be aware of them while considering the best solution for a given situation to avoid accuracy issues and interpretation issues.</p><p>This next section of the blog discusses some commonly used distance metrics in Machine Learning.</p><h3>Euclidean Distance</h3><p>You all have must be knowing Pythagoras Theorem from your junior school.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/310/1*OUPJfxhmksfCkMF9RhPHUg.png" /><figcaption>Pythagoras Theorem</figcaption></figure><p>Pythagoras formula can be extended to get the euclidean distance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/1*11IIV9BBUmJznGzyjM7Xmg.jpeg" /><figcaption>Euclid Distance between two points in two dimension</figcaption></figure><p>Euclidean distance(also known as the Pythagorean metric) is an ordinary straight line between two points in a coordinate axis.</p><p>The general formula for Euclidea distance in n-dimension is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*EMEvv2ayjBo6c7zmufqcPA.png" /><figcaption>Euclidean Distance for n-dimensional space</figcaption></figure><p>A real-life use case of Euclidean Distance will be calculating the distance of a flight between two countries</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*5mSiP-vFF-XPOUCQJB_OVw.png" /><figcaption>Distance between two countries can be measured using Euclidean Distance</figcaption></figure><h4>Applications in Machine Learning</h4><ul><li><strong>Cluster Analysis</strong>: This metric is commonly used in clustering algorithms such as K-means</li><li><strong>Data Science</strong>: It is used as a simple metric to measure the similarity between two data points</li></ul><h3>Manhattan Distance</h3><p>Also known as Taxicab geometry, it calculates the distance between two points as the sum of the absolute differences of their Cartesian coordinates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*LlJuGVWywBoU2MGwoSGnPA.jpeg" /><figcaption>Manhattan Distance Formula</figcaption></figure><p>Where,</p><ul><li>n is the number of dimensions</li><li>xi, yi is the data points</li></ul><p>A real-life use case of Manhattan Distance will be calculating cab fare between two locations in a city.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*b4qspARVoq4z2wzgDtux4A.png" /><figcaption>Manhattan Distance</figcaption></figure><h4>Euclidean Distance Vs Manhattan Distance</h4><ol><li>Euclidean distance is commonly used as it is more intuitive. Other distance metrics may be more appropriate in special circumstances.</li><li>Manhattan Distance is used in linear regression with L1 regularisation(also known as ridge regression)</li><li>For high dimensional vectors, you might find Manhattan Distance works better than Euclidean Distance. At high dimension, Euclidean Distance loses pretty much all meaning. The main issue is something commonly referred to as the <a href="https://youtu.be/OyPcbeiwps8"><em>Curse of Dimensionality</em></a><em>.</em></li></ol><p>An illustration of the difference between Euclidean and Manhattan Distance</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*VWHn7UAZha-fxFMUTpu4aA.png" /><figcaption>An illustration of Manhattan vs Euclidean distance</figcaption></figure><h4>Applications in Machine Learning</h4><ul><li><strong>Regression analysis</strong>: It is used in the linear regression to find a straight line that fits a given set of points</li><li><strong>Face Recognition: </strong>Manhattan Distance along with Image Segmentation can be used for facial recognition. For further details, you can <a href="https://www.ijcsmc.com/docs/papers/July2014/V3I7201408.pdf">read this</a>.</li></ul><h3>Minkowski Distance</h3><p>Minkowski distance is a similarity measurement between two points in the normed vector space (N-dimensional real space). The definition sounds very mathematical. Let me break it down for you, normed vector space means a space where distances can be represented as a vector that has a length.</p><p>Let’s assume that a map is a vector space. If we take a map, we see that distances between cities are normed vector space because we can draw a vector that connects two cities on the map. We can combine multiple vectors to create a route that connects more than two cities. Now, the adjective “normed.” It means that the vector has its length and no vector has a negative length. That constraint is met too because if we draw a line between cities on the map, we can measure its length.</p><p>The formula for Minkowski Distance is given by</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*JVp2GHxOmQhApIQA8Evb6w.png" /></figure><p>Minkowski Distance, also known as Lp form can be generalized for Euclidean and Manhattan distance as well by changing the value of p.</p><p>For p=2, we get the L2 form, which is Manhattan Distance</p><p>p=1, we get the L1 form, which is Euclidean Distance</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/231/1*qyZWglKRI9vz0VuHpVkpWg.png" /></figure><h4>Applications in Machine Learning</h4><ul><li><a href="https://www.researchgate.net/publication/227992742_Fuzzy_Clustering_with_Minkowski_Distance_Functions">Fuzzy Clustering with Minkowski Distance Functions</a></li><li><a href="http://www.ijeei.org/docs-3956607575028549a3320b.pdf">A Framework for a Minkowski Distance Based Multi Metric Quality of Service Monitoring Infrastructure for Mobile Ad Hoc Networks</a></li></ul><h3>Hamming Distance</h3><p>As we know, Environmental interference and physical defects in the communication medium can cause random bit errors during data transmission. Error coding is a method of detecting and correcting these errors to ensure information is transferred intact from its source to its destination.</p><p>Hamming distance is one such error-correcting code to measure the distance between two codewords, we just count the number of bits that differ between them. If we are doing this in hardware or software, we can just XOR the two codewords and count the number of 1 bit in the result. This count is called the <strong>Hamming distance.</strong></p><p>Suppose there are two strings 11011001 and 10011101.</p><p>11011001 ⊕ 10011101 = 01000100. Since, this contains two 1s, the Hamming distance, d(11011001, 10011101) is 2.</p><p>Here are a few useful references :</p><ul><li><a href="http://www.eecs.umich.edu/courses/eecs373.w05/lecture/hamming1.pdf">Error Correction with Hamming Codes</a></li><li><a href="http://www.eecs.umich.edu/courses/eecs373.w05/lecture/hamming2.pdf">Calculating the Hamming Code</a></li><li><a href="http://www.eecs.umich.edu/courses/eecs373.w05/lecture/hamming3.pdf">Applying Hamming Code to blocks of data</a></li></ul><h3>Cosine Distance and Cosine Similarity</h3><p>These two terms are closely related and widely used in the recommendation system. Cosine similarity measures the similarity between two vectors by calculating the cosine of the angle between them</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/342/1*8PE_mOJos-4a_p8a2xtPbg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/304/1*UDib6BXlbj8jyZlvbeqA3g.png" /></figure><p>Cosine distance=1-Cosine_Similarity. So, similarity increases when distance decreases and vice versa.</p><p>Hence, we can say that Cosine Similarity lies between -1 and +1 as the range of Cos theta is [-1,+1]</p><ul><li>Cosine value 1 is for vectors pointing in the same direction i.e. there are similarities between the data points.</li><li>At zero for orthogonal vectors i.e. Unrelated(some similarity found).</li><li>-1 for vectors pointing in opposite directions(No similarity).</li></ul><h4>Euclidean Distance VS Cosine Distance for Recommendations</h4><p>The crux of this whole discussion is to choose the best metric for recommendation systems.</p><p>Let’s say I have a database of users who rate movies on a scale of 1–10. If I want to make a list of the top 5 most-watched movies similar to my profile; my first approach to finding similar users was to use Cosine Similarity and just treat user ratings as vector components. The main problem with this approach is that it just measures vector angles and doesn’t take the rating scale or magnitude into consideration. Magnitude matters in this case because I want the top 5 movies that are the closest to my user profile averages. Euclidean Distance can be used in such a scenario as it accounts for magnitude as well.</p><p>Cosine Similarity is more useful for instances when you do not want magnitude to skew the results. This is most useful in <a href="https://towardsdatascience.com/understanding-nlp-word-embeddings-text-vectorization-1a23744f7223"><em>word vectorization </em></a>because normalizing the data makes a long document comparable to a short document. The euclidian distance will be very large between documents of different word length which would skew your results.</p><p>Also, cosine similarity finds its application in text mining.</p><h3>Conclusion</h3><p>Summing everything up, we can search for alternatives instead of mindlessly employing predefined metrics. We are not bound to use handcrafted designs since we can make machines come up with tailored solutions for our specific problems; this is what machine learning is about!</p><p>Using the Distance Metric as a hyperparameter is a good optimisation feature. There are times where it will come in handy, and there are times where it won’t. The typical use cases are combating the dimensionality curse, saving computational costs, interpreting data, and improving accuracy.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79518a7db822" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Role of Mathematics in Machine Learning]]></title>
            <link>https://medium.com/analytics-vidhya/role-of-mathematics-in-machine-learning-afc75101f877?source=rss-f09f736c263c------2</link>
            <guid isPermaLink="false">https://medium.com/p/afc75101f877</guid>
            <category><![CDATA[linear-algebra]]></category>
            <category><![CDATA[probability]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[calculus]]></category>
            <dc:creator><![CDATA[Alekhyo Banerjee]]></dc:creator>
            <pubDate>Sun, 15 Mar 2020 14:15:34 GMT</pubDate>
            <atom:updated>2020-03-21T15:12:41.677Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*maqmu2RX4TqzAL55mgQUPA.jpeg" /></figure><p>Nowadays many people are planning the transition to AI/ML/Data Science world which is very encouraging and matching the pace of changing the world.</p><p>But these people are confused with questions like :</p><ul><li><em>I wanna be ML specialist without diving deep into mathematics, is it possible?</em></li><li><em>Why is mathematics important in data science and AI/ML world?</em></li><li><em>As mentioned, a vast array of libraries exist to perform various machine learning tasks so it’s easy to avoid the mathematical part of the field.</em></li></ul><p>If you think of these questions too, this article is for you.</p><p>Various problems are hard to solve using traditional programming methods like computer games, self-driven cars, recognizing objects. One way round is to tell computers how to learn from data, this is machine learning.</p><p>Machine Learning helps Amazon suggest products for you, Youtube recommend videos, classify spam mail, etc. To make this possible, we combine mathematics with lots of programming.</p><p>Machine Learning is all about creating algorithms that can learn data to make a prediction. Machine Learning is built on mathematical prerequisites.</p><p>Mathematics is important for solving the Data Science project, Deep Learning use cases. Mathematics defines the underlying concept behind the algorithms and tells which one is better and why.</p><h3>Well, you can create models without even knowing the maths behind the working of algorithms but wait… How would you know which one is better and where to use it?</h3><p>Let me clear the air. You need to understand the mathematics behind machine learning algorithms to become a data scientist. There is no way around it. It is a fundamental part of a data scientist’s role and every recruiter and experienced machine learning professional will vouch for this.</p><p><strong>Machine Learning </strong>is structured on the foundation of these four pillars of Mathematics:</p><ol><li>Linear Algebra</li><li>Calculus</li><li>Statistics</li><li>Probability</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/785/1*GPbjqjTO1vqHzOI_sHhZYw.png" /></figure><p><strong>Linear Algebra:</strong> It makes possible for algorithms to run on massive datasets.</p><ul><li>Linear algebra shows up everywhere in the machine learning world</li><li>Without linear algebra, machine learning methods cannot be developed, a complex data structure cannot be handled and manipulated, matrix operations on large datasets are not possible</li><li>Linear Algebra concepts one needs to know — Vectors, Vector Spaces, Scalars, Orthonormalization, Matrix Operations, Projections, Eigenvalues &amp; Eigenvectors, etc.</li></ul><p>Linear algebra acts as a stage or a platform over which all the machine learning algorithms display their results.</p><h3>But why linear algebra?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*EF8Z2KMzejDmz_GgEjnpHg.png" /></figure><p>Linear algebra can transform datasets into matrices on which several operations can be performed. <strong>NumPy</strong> is such a library used in Machine Learning which performs several operations on N-d array.</p><p><strong>Calculus: </strong>It is used to finetune the result. It optimizes the performance of the algorithm.</p><ul><li>Calculus plays an integral role in many machine learning algorithms like in the gradient descent algorithm and backpropagation to train deep learning neural networks.</li><li>Calculus knowledge helps in the optimization of the model’s performance.</li><li>Calculus concepts one needs to know — Differential and Integral Calculus, Partial Derivatives, Vector-Values Functions, Directional Gradient, Jacobian, etc.</li></ul><p>In Machine Learning, we try to find the inputs which enable a function to best match the data. The slope or descent describes the rate of change off the output with respect to an input. Determining the influence of each input on the output is also one of the critical tasks. All this requires a solid understanding of Multivariate Calculus.</p><p>For example:</p><p><strong>Linear Regression </strong>is<strong> </strong>a linear model that establishes the relationship between a dependent variable <em>y(Target) </em>and one or more independent variables denoted <em>X(Inputs) </em>using a best fit straight line (also known as a regression line).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*mQKu01FX4Xspi7jnOtDtlA.jpeg" /></figure><p>This algorithm shows how calculus is used in finding slope, gradient descent and working behind this algorithm.</p><p><strong>Probability:</strong> It helps in predicting the likelihood of future events in machine learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g0bUhIIQRKpowUwM9iJNAg.jpeg" /></figure><ul><li>The main sources of uncertain events which introduce imperfection in machine learning models are noisy and scarcity of relevant data.</li><li>Probability concepts that one needs to know — Joint, Marginal, and Conditional Probability, Probability Distributions (Discrete, Continuous), Density Estimation, Maximum Likelihood Estimation, Regression with Maximum Likelihood, Bayes Theorem, etc. We use them to carry out hypothesis testing where an understanding of probability is quite essential.</li></ul><p>The <strong>Naive Bayes algorithm</strong> is an example that works on a similar principle, with a simple assumption that all the input features are independent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7oMbMO_u_XaJvkW8dlRnkg.png" /></figure><p><strong>Statistics:</strong> Statistics can be used to draw logical conclusions from the given data.</p><ul><li>Statistics is a collection of tools that helps to identify the goal from the available data and information.</li><li>Statistics helps to understand the data insights and transform the sample observations into meaningful information.</li><li>No system in the world has perfect data stored and readily available as needed. Every system has data anomalies like incomplete, corrupted data, etc. Statistical concepts will be your best friend to help in such complex situations.</li><li>It helps to answer questions such as :<br><em>1. What product was so sold highest at any particular month?<br>2. Who scored the most runs in cricket tournament?</em></li><li>Statistical concepts that one needs to know — Distribution, Central Tendency, Skewness, Linear Correlation Coefficient, Central Limit Theorem, Hypothesis Testing, etc.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/630/1*G7aDvSi5mI-wh5sogE_oqA.jpeg" /></figure><p>Statistics is the most commonly used part of Machine Learning algorithms. A Data Analyst’s job role is to draw conclusions/questions from the given data and he/she is dependent on statistics for it.</p><h3>Conclusion</h3><p>Mathematics for machine learning is an essential facet that is often overlooked or approached with the wrong perspective.</p><p>As a soft prerequisite, there is an expectation that one should have a fair understanding of these mathematical concepts.</p><p>This doesn’t mean you have to go through your high school notes and mug up the theorems. The best way to get a taste of mathematics is to take an ML algorithm, find a use case, solve and understand the maths behind it.</p><p>When one encounters real-world problems in machine learning, it becomes easy to resolve them if he/she has good understanding and intuitions of mathematical concepts. It enhances critical problem-solving skills too.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=afc75101f877" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/role-of-mathematics-in-machine-learning-afc75101f877">Role of Mathematics in Machine Learning</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Job Roles]]></title>
            <link>https://alekhyo.medium.com/understanding-job-roles-22002174bcea?source=rss-f09f736c263c------2</link>
            <guid isPermaLink="false">https://medium.com/p/22002174bcea</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-scientist]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-analyst]]></category>
            <dc:creator><![CDATA[Alekhyo Banerjee]]></dc:creator>
            <pubDate>Sat, 25 Jan 2020 18:32:57 GMT</pubDate>
            <atom:updated>2020-01-29T03:40:02.533Z</atom:updated>
            <content:encoded><![CDATA[<h3>Understanding Different Job Roles in the field of Data Science</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gb45yrDxoidYKXuw20sxzA.jpeg" /><figcaption>The relation between skillset and job roles</figcaption></figure><p>It’s very common these days to come across these terms — data science, artificial intelligence, machine learning, deep learning, and much more. As a consequence, a bunch of different data science roles came our way, but it’s tough to get a general understanding of how they differ and the skills they require.</p><p>This article aims to present brief insights of different job roles, responsibilities of each title along with the skills/qualifications required and estimated salary earned in different data science job roles. Let’s get started then.</p><h3>The Essential Dilemma:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YwPz5ajVZhd9tHXqHEnPBw.jpeg" /></figure><p>There are major four job roles:</p><ol><li>Data Engineer</li><li>Data Analyst</li><li>Data Scientist</li><li>ML Engineer</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*UHbRCj2OqgdJ1vvx1urWig.png" /><figcaption>Source: Google Trends</figcaption></figure><p>Current trends on these four job roles according to Google Trends.</p><p>But do you know which one is the right to kickstart your career and why you should choose one over the other? Let’s dive into more details</p><h3>Data Engineer:</h3><p>We are in the age of the data revolution. Prominent enterprises now base their decision-making skills on insights derived from the analysis of data. For many organizations, data engineers are the first hires on a data team. Data Engineers are the ones who gather data from other websites through web scrapping, API’s or IoT devices and ingests the data into the data warehouse. Examples of data warehousing systems include Amazon Redshift or Google Cloud. Finally, Data Engineers create ETL (Extract, Transform and Load) processes to make sure that the data gets into the data warehouse.</p><p>For example, think of a song recommender system machine learning model that needs to be deployed as part of a company’s application to make the application better. Each time a user plays a song, a new piece of data is being created. A <strong>Data Engineer</strong> would define how to collect this data, what types of metadata should be appended to each click event, and how to store the data in an easy-to-access format</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w63WsugqfA4N_0J2S58YzA.jpeg" /></figure><p>The crucial tasks included in Data Engineer’s job role are:</p><ul><li>Collect data through web-scraping, API’s or IoT devices.</li><li>Ingest the collected data into a data warehouse.</li><li>Manage ETL(Extract, Transform &amp; Load) process</li><li>Introducing new data management tools &amp; data models for easy access to the data.</li></ul><p><strong>Technical Aspects:</strong></p><ul><li>Experience with big data tools: Hadoop, Spark, Kafka, etc.</li><li>Experience with relational SQL and NoSQL databases, including Postgres and Cassandra</li><li>Experience with AWS cloud services.</li><li>Experience with object-oriented/object function scripting languages like Python, Java, C++, Scala, etc.</li></ul><h4>Job Scenario:</h4><p><strong>Data </strong>engineers have an annual salary growth of about 9%. The average starting salary of a big data engineer can range from INR 6,00,000 to INR 10,00,000. According to a survey performed by the <a href="https://www.northeastern.edu/graduate/blog/highest-paying-big-data-careers/">Internal Revenue Service (IRS)</a>, the top salary bracket makes big data engineers the top 5% of the highest-earning roles. An increasing number of enterprises have now started adopting data in their projects, while others have already made plans to incorporate data into their future projects. The sports industry, for instance, has an increased demand for data engineers to track metrics of consumers like social media behavior, ticket-purchasing habits, demographics, brand interests, and psychographic profiles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*fkq4_6lk0RDih-pLY6s3NA.jpeg" /></figure><h3>Data Analyst:</h3><p>The <strong>Data Analysts</strong>, perform exploratory data analysis, run statistical analysis and creates visualization based on analysis and forwards further to perform suitable algorithms to train the model. Analysts implement feature engineering, feature selection, clean the data using programming languages, spreadsheets, and business intelligence tools to describe and categorize the data.</p><p>For example, Incase of a music recommendation integrating model, a <strong>Data Analyst</strong> would create visualizations to track which artist is played the most, the genres of songs which are played the most and how much money the company is making.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*RHYNgLMoDSqN361pJ45VKg.gif" /></figure><p>The crucial tasks included in Data Analyst’s job role are:</p><ul><li>Managing master data, including creation, updates, and deletion.</li><li>Processing confidential data and information according to guidelines.</li><li>Create reports and analysis.</li><li>Managing and designing the reporting environment, including data sources, security, and metadata.</li><li>Providing expertise in data storage structures, data mining, and data cleansing.</li></ul><p>Technical Aspects:</p><ul><li>Structured Query Language (SQL)</li><li>Data mining, Cleaning, and Munging</li><li>R or Python-Statistical Programming(Numpy, pandas library)</li><li>Data Visualization and Presentation Skills (Tableau, Power BI, seaborn library in python)</li><li>SQL</li><li>Data Story Telling</li></ul><h4>Job Scenario:</h4><p>More than 97,000 analytics positions remain vacant in India due to the shortage of talent. <strong>11 lakhs is the average salary in analytics and data science.</strong>BFSI sector has the maximum demand for data science skills in India followed by e-commerce and telecom. Python continues to be the tool of choice among data analysts and data scientists and this is reflected in the hiring market as well with 17% jobs listing the language as a core capability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*AeuuMzMUZqTOkrBVAE_kSA.jpeg" /></figure><h3>Data Scientist:</h3><p>It is considered as the sexiest job of the 21st century. They work based on the visualization provided by the data analytics team to build and optimize classifiers using machine learning techniques.</p><h4>‘Data Scientists know more about statistics than any software developer and know more about software development than any statistician.’ — Josh Wills, Director, Slack.</h4><p>For example, A <strong>Data Scientist</strong> would take the data on the artists, music genres and train the machine learning to recommend songs/albums for each user.</p><p>The crucial tasks included in the Data Scientist’s job role are:</p><ul><li>Thoroughly clean data to discard irrelevant information and prepare the data for preprocessing and modeling</li><li>Perform exploratory data analysis (EDA) to determine how to handle missing data.</li><li>Discovering new algorithms to solve problems and build programs to improve current strategies.</li><li>Perform feature engineering, feature selection to implement analytical methods, machine learning and statistical methods to prepare data for use in predictive and prescriptive modeling</li></ul><p>Technical Aspects:</p><ul><li>Programming</li><li>Applying Machine Learning algorithms and libraries(Scikit Learn, Tensorflow, PyTorch)</li><li>Data Visualization and Reporting</li><li>Statistical analysis and Math</li><li>Effective Communication</li><li>Software Engineering Skills</li><li>Data Mining, Cleaning, and Munging</li></ul><h4>Job Scenario:</h4><p>Data Scientists can also work as data analysts which makes Data Scientists as one of the highest-paid jobs. Data Scientists ranked first among the most promising jobs in the United States in 2019 as per LinkedIn’s report. According to the same report, <strong>Data Scientists averaged US$130,000 in the basic salary this year. Furthermore, the job vacancies increased by 56 percent from the last year</strong>. Currently, <strong>there are more than 4,000 Data Science vacancies across the country.</strong> No wonder: Data Scientists are led to the top as the best job in the United States, thanks to its high demand, high salaries, and high job satisfaction. Also<strong>, </strong>Analytics India Magazine predicts that the demand for Data Science professionals in India will increase sevenfold in the next seven years and the market will reach US$20 billion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*8ogtloI25bPgBNi0HDixcQ.jpeg" /></figure><h3>Machine Learning Engineer:</h3><p>It’s the final stage of a data science project cycle where the model created is deployed to be integrated into the application or website. Machine Learning Engineers are neither experts in Data Science nor web development. They ideally use both the knowledge to deploy the final model. The work of a Machine Learning Engineer is to bridge the gap between Data Scientist’s work and production environment. A Machine Learning Engineer is more concerned with deploying production-ready models.</p><p>For example, Machine Learning Engineer deploys the song recommender machine learning model to a production-ready environment. It involves different types of engineering work such as integrating the model to a software system, optimizing the model for performance and scalability, and re-training it with new data, monitoring, and maintenance the ML system</p><p>The crucial tasks included in the ML Engineer’s job role are:</p><ul><li>Removing errors from data sets and find correct data representation methods</li><li>Deploying the machine learning model to be integrated into the application/ website</li><li>Scaling and optimizing the model for production</li><li>Monitoring and maintenance of deployed models</li></ul><p>Technical Aspects:</p><ol><li>Probability and statistics</li><li>Data Modeling and Evaluation</li><li>Applying Machine Learning algorithms and libraries(Tensorflow, Pytorch)</li><li>Software Engineering and system design(AWS, Azure)</li></ol><h4>Job Scenario:</h4><p>As a fresher, there is a median salary of almost 13 Lakhs and rising for a Machine Learning Engineer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KiuOnV1cGErAJtePeS_SEA.png" /></figure><p>This is one of the trendiest and the coolest jobs to have as per a survey conducted earlier this year.</p><p>A Machine learning engineer in the USA gets an annual pay of about 140 thousand dollars. It’s about 50,000 pounds in the UK and about 13 Lakhs in India.</p><h3>CONCLUSION:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mnxgYF1lym6xToPEpYNsaQ.png" /></figure><p>I hope you’ve got a brief idea about the insights of these job roles. Now, it’s all about selecting the right profession to kickstart your career which suits your skills and interests. The upward swing in Data Science career opportunities is expected to continue for a long time to come. As data pervades our life and companies try to make sense of the data generated, skilled Data Scientists will be continued to be wooed by businesses big and small.</p><p><strong>All the best for your career and job search in Data Science!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22002174bcea" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>