<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shubham Chaudhari on Medium]]></title>
        <description><![CDATA[Stories by Shubham Chaudhari on Medium]]></description>
        <link>https://medium.com/@shubhamc03?source=rss-10419c65c3ee------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by Shubham Chaudhari on Medium</title>
            <link>https://medium.com/@shubhamc03?source=rss-10419c65c3ee------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 15:36:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shubhamc03/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Clustering: DBSCAN Clustering]]></title>
            <link>https://medium.com/@shubhamc03/clustering-dbscan-clustering-92440b9617a2?source=rss-10419c65c3ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/92440b9617a2</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[dbscan]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Shubham Chaudhari]]></dc:creator>
            <pubDate>Tue, 18 Apr 2023 10:55:11 GMT</pubDate>
            <atom:updated>2023-04-18T10:55:11.620Z</atom:updated>
            <content:encoded><![CDATA[<h4>A density-based clustering method that handles noise and represents clusters of non-spherical shape.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vgTLy3agujrECC-a_EfUIA.jpeg" /></figure><p><strong>DBSCAN: Density-Based Spatial Clustering Application With Noise.</strong></p><p>Hierarchical and K-means clustering are unable to identify outliers.DBScan can identify outliers.</p><p>Terminologies:</p><ol><li>Epsilon — Radius of circle</li><li>min_sample</li><li>core-point</li><li>Border point</li><li>Noise point</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*79pzKD3AdMkKgu006gXTmA.png" /><figcaption><strong>Core Point, Border Point and Noise Point</strong></figcaption></figure><ol><li>Epsilon — Radius of the circle around datapoint, epsilon value is user-defined</li><li>min_sample — we need to tell how many data points will be there in the circle</li><li>Core point — the core point is that datapoint if the circle is drawn around that datapoint it should satisfy min-sample criteria. It should have a minimum of data points which are 3 in this case. the datapoint around which the circle is drawn is also counted in the minimum sample criteria.</li><li>Border point — A border point is a data point if a circle is drawn around that data point. if it is not drawn around that data point. if it does not satisfy the minimum sample criteria means there are fewer data points in a circle than the minimum sample criteria in this case point B has only two points inside the circle. but the point around which the circle is drawn should be in the circle of another data point then only we can say it is a border point. there are two conditions for border point. first is is should have less number of data points in the circle than stated in the minimum sample criteria. and second is the same data point should be in a circle of another data point.</li><li>Noise point — This point is not part of any other cluster and does not satisfy minimum sample criteria in this case point C does not satisfy minimum sample criteria and the point is not in the circle or cluster of any other data point. The noise point is called an outlier.</li></ol><ul><li>Every data point will have the same radius which has passed for the epsilon value</li></ul><p><strong>Drawbacks of K-means and hierarchical clustering</strong></p><p><strong>K-means clustering drawbacks:-</strong></p><ul><li>Partitioning method:- Partitioning the dataset to k Number of groups. the partition is an iterative process that allocates each data point of the dataset to a group where it belongs. After the allocation of points, the mean of the group is calculated by taking an average for all the points in the group. The most well-known partitioning method is k-means</li><li>The significant drawbacks to the partition method:-</li></ul><ol><li>The k-means algorithm requires specifying the number of clusters</li><li>The k-means does not perform well in finding convex/non-spherical shapes of clusters</li><li>The k-means are sensitive to noise data.</li><li>DBScan-Wherever there is density will consider it as a cluster.</li></ol><p><strong>Hierarchical clustering drawbacks:-</strong></p><ol><li>it is not suitable for big datasets which have high computational complexity.</li><li>Need a metric for merging the cluster (Linkage) that affects the clustering results.</li><li>Sensitive to noise.</li></ol><p>As we can see that the main disadvantage of the partitioning method and hierarchical clustering is handling noise and getting bad results with finding clusters of non-spherical shape.</p><p><strong>DBSCAN:-</strong></p><p>DBSCAN is a density-based clustering method that discovers clusters of non-spherical shape data. The DBScan clustering method can represent clusters of arbitrary shape and handles noise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yboE9LS6MaHKvtbe37_YQg.png" /><figcaption><strong>Working with DBSCAN and k-means Clustering</strong></figcaption></figure><p>If there are no linearly separable data k-means and hierarchical clustering are not going to do well. In that case, DBSCAN going to do well.</p><p><strong>DBSCAN Parameters: ε (or eps or epsilon)</strong></p><ul><li>If a higher epsilon value is passed there are chances that the circle around the datapoint will take all or nearly most of the points inside the circle and outliers will be an inside circle.</li><li>If a lower epsilon value is passed it will not satisfy the minimum sample criteria.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*67YN8RgxumKV7tm-2pSYIg.png" /><figcaption><strong>Clustering with High and Low Epsilon value</strong></figcaption></figure><p>point x and its neighbors would be in one neighborhood and y and it is few neighbors would be in another in the end, however, point x and y and its neighbors would probably be in one cluster whereas point y and its neighbor would consider outliers or noise.</p><p>This is because the neighborhood of y is not dense enough as a neighborhood contains more points the dense it becomes</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2o8K-yx_J1kWFIGMCBZt1A.png" /><figcaption><strong>Cluster making of X and Y Data Points</strong></figcaption></figure><p><strong>How can we define that neighborhood is dense enough? In order to do that the DBSCAN uses the parameter minpts.</strong></p><p>Minpts:- A neighborhood is deemed to be dense if it includes at least Minpts points in its ε-neighborhood, where Minpts is the density threshold. Within such a neighborhood, the points that meet this criterion are considered core points.</p><p>Let’s check the images, If the Minpts parameters are 3. The point x will be a core point because the size of its ε neighborhood is 9 and its bigger than 3 point y won’t be a core point because its ε neighborhood contains two points</p><p>The number of points in the ε neighborhood of point x contains itself.</p><p>Border point:- Border point ε neighborhood that contains less than Minpts points(so it’s not a core point). but it belongs to the ε neighborhood of another core point. If a point isn’t a core point and isn’t a border point, it’s a noise point or an outlier.</p><p>We can see that x is a core point .it has more than 11 points in its ε neighborhood.</p><p>Point y isn’t a core point because it has less than 11 points in its ε neighborhood, but because it belongs to the ε neighborhood of point x, and point x is a core point, point y is a border point. we can easily see that point z isn’t a core point of y and point y isn’t a core point, therefore point z is a noise point.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fw36JsOUInsYlJM4Iv4lWw.png" /><figcaption><strong>X, Y, And Z Data points</strong></figcaption></figure><p><strong>Density connected:-</strong></p><p>A point x is density connected to a point y, if there is a point o that both x and y are density reachable from this way that DBScan connects core objects and their neighbors in a dense region.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ODFps1AObroUgMFlM7lkqg.png" /><figcaption><strong>Working on Density connection in DBSCAN Clustering.</strong></figcaption></figure><ul><li>If Circle is drawn around that data point and that data points have another data point to that data point circle is drawn and it continues and form density, in the way are used and connected at point o in the above example</li><li>If overlapping in a circle it makes one cluster</li><li>If overlapping of a circle is not happing then the circle is drawn around other data to form other clusters.</li><li>Minimum sample criteria will decide whether it is a dense region or not.</li></ul><p><strong>How to choose Epsilon?</strong></p><ul><li>Parameters Estimation:- If epsilon has a small value, many points may be considered outliers. They wouldn’t be core points(the ε-neighborhood will be very small). A large value for ε may cause a huge amount of points to be in the same cluster.</li></ul><p><strong>How to evaluate clustering models?</strong></p><ol><li>Silhouette score</li><li>Rand Index</li><li>Jaccord Index</li></ol><p>Silhotte score is more accurate than the Rand index and Jaccord score</p><p>Silhotte score:- The silhouette score is calculated utilizing the mean intra-cluster distance between points, and the mean nearest cluster distance for instance, a cluster with a lot of data points very close to each other (high density) and is far away from the next nearest cluster (suggesting the cluster is very unique in comparison to the next close) will have a strong silhouette score.</p><p>A silhouette score range from -1 to 1, with -1 being the worst score possible and 1 being the best score. A silhouette score of 0 suggests overlapping clusters</p><p>This silhouette score can be used for the evaluation of any other clustering models also.</p><p><strong>Parameters</strong></p><p>DBScan algorithm requires two parameters to pass:</p><ol><li>eps:- One way to find the Eps value is based on the k-distance graph.</li><li>Minpts:- Minimum number of neighbors (data points) within eps radius.Larger the datasets the larger minpts value must be chosen. As a general rule, the minimum minpts can be derived from the number of dimensions D in the dataset as, Minpts≥D+1. The minimum value of minpts must choose at least 3. It is only a basic thumb rule. A minimum minpt value of 3 will be good. Dimension D means features or columns.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92440b9617a2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Clustering: K-means Clustering]]></title>
            <link>https://medium.com/@shubhamc03/clustering-k-means-clustering-53e7b661998c?source=rss-10419c65c3ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/53e7b661998c</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[k-means]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Shubham Chaudhari]]></dc:creator>
            <pubDate>Sun, 09 Apr 2023 15:29:34 GMT</pubDate>
            <atom:updated>2023-04-09T15:29:34.546Z</atom:updated>
            <content:encoded><![CDATA[<h4>Concept and working of K-means clustering, Optimal Clusters in K-means Clustering, Mathematics Behind K-means Clustering, Deciding the Optimal Number of Clusters in K-means Clustering, Comparison of K-means Clustering with Hierarchical Clustering.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5zg0uEbEKGm3OKuSRbmmFw.jpeg" /></figure><p><strong>K-means Clustering</strong></p><p>K-means clustering is a centroid base algorithm.</p><ul><li>k presents a number of clusters we want to create</li><li>For example, consider we want to create 2 clusters.</li><li>Initially, the algorithm will randomly generate two centroids anywhere over the dataset</li><li>The graph shows that all these data points will calculate the distance between centroid C1 and centroid C2.</li><li>The data points which are near centroid C1 Will go to cluster C1 And The data points which are near centroid C2 will go to cluster C2.</li><li>The distance is calculated using Euclidean Distance Formula.</li><li>K_means clustering is an iterative process.</li><li>Every time location of the centroid changes a new data point is added because of this it is called an iterative process.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/669/1*OWgaH5hDZ188jOMt7p_GEg.png" /><figcaption><strong>1st Iteration</strong></figcaption></figure><p>Next iteration — 2nd</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/839/1*HIxCVGhsS4qx5z3v0Mzxyg.png" /><figcaption><strong>2nd Iteration</strong></figcaption></figure><p>It does not calculate distance within data points. it calculates the distance of the data point between centroids.</p><p>Next Iteration — 3rd</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*UMSmg-KYvWMbe8oiq9af5g.png" /><figcaption><strong>Final Clusters</strong></figcaption></figure><p>This will not end until all the datapoints will not go into any of their a cluster.</p><p>k-means working on a similarity basis</p><ul><li>In the end, centroids will be at the center of clusters.</li><li>We get the center of any data point from the average or mean.</li><li>Centroids are depends on what k number is going to set.</li><li>Every time a new data point is added coordinate of the centroid is going to change.</li></ul><p><strong>What if the centroid randomly comes to the same location?</strong></p><p>Assume, k=2</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/755/1*xcN-R4EqU6zbGuXii8RwWg.png" /><figcaption><strong>Centroid Comes At Same Location</strong></figcaption></figure><ul><li>If both of the centroids come ta the same location it will create only one cluster and it is not optimal</li><li>Due to these researchers have done some modifications. so anytime clusters are getting created the centroids are having optimal distance between them, this improvement is done and the new algorithm is called k-means++.</li><li>Sklearn is using k-means ++ in the backend, not k-means.</li><li>The problem with k-means is that centroid might get at the location.</li><li>An improved version of k-means is called K-means ++.</li></ul><p><strong>How to define how many clusters we want? and which cluster is optimal?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/969/1*YkJbTapa30ukAAb5EnylJQ.png" /><figcaption><strong>Optimal cluster</strong></figcaption></figure><p>Let&#39;s take an example, and consider there are two cases case 1 and case 2, both cases have the same datasets and the same number of clusters.</p><ul><li>For any data points which are grouped together the distance between them should be less.</li><li>Varienec within a cluster should be low.</li><li>The variance between clusters should be high.</li><li>K-means only consider variance within clusters.</li></ul><p>From the Graph, Case 1 has less variance within the cluster and Case 2 has case 2, and also variance between clusters in the case is less as compared to case 2. so Case 1 has optimal clusters.</p><p><strong>Mathematics Behind K-means</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*8Tj_y_LkgAcQVucvvpNEsA.png" /><figcaption><strong>Table For Mathematical Calculation</strong></figcaption></figure><p>Assume,k=2</p><p>and datapoint 1 as C1 and datapoint 2 as C2 as centroids</p><p>Distance of data point 3 From C1</p><p>= √( xo - xc )² + ( yo — yc )²</p><p>= √( 150 — 180 )² + ( 75 — 65 )²</p><p>= 31.62</p><p>Distance of data point 3 From C2</p><p>= √( xo — xc )² + ( yo — yc )²</p><p>= √( 150–190)² + ( 75–80 )²</p><p>= 40.31</p><p>As a data point, 3 has less distance between C1 than C2 it will go into a cluster of Centroid 2</p><p>As a new data point is added to cluster C1 the next to calculate the new centroid for cluster C1</p><p>For Calculating the new centroid use the following Formula</p><p>NC = (( xp + xnew )/ 2 ,( yp + ynew )/ 2 )</p><p>xp = x co-ordinate of the previous centroid</p><p>xnew = x co-ordinate of newly added centroid</p><p>yp = y co-ordinate of the previous centroid</p><p>ynew = y co-ordinate of newly added centroid</p><p>= ((150+180)/2,(75+65)/2))</p><p>New Centroids For C1 Cluster=(165,70)</p><p><strong>Deciding the Optimal Number for k</strong></p><ul><li>k-means gives an approach called elbow curve</li><li>Within cluster sum of square(WCSS/Inertia)</li><li>In sklearn, it is called inertia</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*aUFMgwbIIouN0s4sx-ngZQ.png" /><figcaption><strong>Optimal value of k</strong></figcaption></figure><ul><li>WCSS values tell how much variance within the cluster</li><li>WCSS should be minimum</li><li>select the value of k where the WCSS value drastically decreases</li><li>after k = 3, there is no huge difference</li><li>k value at the corner or elbow of the line should be selected.</li></ul><p><strong>Comparison Between Hierarchical Clustering And k-means Clustering</strong></p><ul><li>k-means is part of divisive clustering</li><li>HC is a part of agglomerative clustering</li><li>Initially, HC considers every data point in the cluster, if 10000 data points are there, 10000 clusters will be formed then after it will calculate the distance and new clusters will be formed this process call a bottom-to-top approach or agglomerative clustering.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*cknOOZGMih_nK56UJEhVAA.png" /><figcaption><strong>Bottom To Top Approach (Agglomerative Clustering)</strong></figcaption></figure><ul><li>In k-means generate random centroid and calculating the distance of data points between centroid is called divisive clustering</li><li>the default value of k = 8</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53e7b661998c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Clustering: Hierarchical Clustering]]></title>
            <link>https://medium.com/@shubhamc03/clustering-hierarchical-clustering-8cfa8748696c?source=rss-10419c65c3ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/8cfa8748696c</guid>
            <category><![CDATA[hierarchical-clustering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[data-analytics]]></category>
            <dc:creator><![CDATA[Shubham Chaudhari]]></dc:creator>
            <pubDate>Wed, 05 Apr 2023 17:29:15 GMT</pubDate>
            <atom:updated>2023-04-05T17:41:52.588Z</atom:updated>
            <content:encoded><![CDATA[<h4>“Concept of Hierarchical Clustering And Linkages”</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qKI_3MJN_gsYQlkOkGdl7w.jpeg" /></figure><p><strong>Unsupervised learning</strong></p><p>Clustering:-Hierarchical clustering, K-means, DBScan</p><p>Dimension reduction:-PCA, T-SNE</p><p>Recommendation system</p><p><strong>Data Mining:-</strong>Extracting helpful information from huge amounts of unlabelled data is called data mining</p><p><strong>Dimension Reduction Technique:-</strong> Dimension reduction techniques are approaches to making data ready for machine learning or unsupervised machine learning part</p><p><strong>What is Clustering?</strong></p><p>Cluster Analysis (“Data segmentation”) is an exploratory method for identifying homogenous groups (“Cluster”) of records</p><p>Similar records should belong to the same cluster and Disilmilar records should belong to different clusters</p><p><strong>So what and why clustering?</strong></p><ul><li>Clustering = Grouping similar things together</li><li>Understand/Discover structure in data</li><li>summarize data points by “cluster center”</li><li>Datamining techniques can be to give labels to unlabelled data</li></ul><p><strong>Clustering</strong></p><ul><li>Agglomerative Clustering(Bottom-up)</li><li>Divisible (Top to bottom approach)</li></ul><p><strong>Hierarchical clustering</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*Cz6eEobqIbHGJuk6-Q_ORA.png" /><figcaption><strong>Example for Calculation of Euclidean distance</strong></figcaption></figure><p>Euclidean distance=√Σ(y2 - y1)²+(x2 - x1)²</p><ul><li>Every data point will calculate its distance from each other data points by using Euclidean distance.</li><li>Hierarchical clustering makes a cluster of the nearest data points</li><li>Any cluster should not have a single data point</li><li>From the graph, A, and B will form one cluster and D and E will form another cluster</li><li>The remaining point C will calculate the distance from A and B clusters and D and E clusters from which cluster it is near it will be classified into that Cluster per example point C is classified into the DE cluster.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/805/1*HYE7bm9E2f5mTOmj9o-skQ.png" /><figcaption><strong>Dendrogram</strong></figcaption></figure><ul><li>This representation is called a dendrogram</li><li>Hierarchical clustering depends on Euclidean distances</li><li>to calculate the distance between a single data point and a cluster we use a linkage</li></ul><p><strong>linkage:-</strong></p><ol><li>Single</li><li>Complete</li><li>Average</li><li>Centroid</li><li>Single:-From graph DE is cluster as the C datapoint is near to E than any other data point. C is classified into the DE cluster.</li><li>Complete:-It uses maximum distance to form a cluster between the single data point and a cluster.</li><li>Average:- In average both data points are taken into consideration C Will be classified into DE Cluster.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*MvsnhYZKpe5wC4O8dK2tDw.png" /><figcaption><strong>Linkage Calculation Average</strong></figcaption></figure><p>AB → C → 4</p><p>DE →C →2.5</p><p>Point C will classify into a DE cluster as the distance between Cluster DE and Data point C is less.</p><p>4. Centroid:- The center of the cluster is calculated whichever distance is less will classify that data point into clusters that have distance.</p><p>centroid: Centroid means average between datapoint within the cluster.</p><p><strong>Euclidean distance</strong></p><p>Two variables,n=5 items</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/322/1*0WCy5LkaVKa4PmSMdfUoRg.png" /><figcaption>Variables</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/753/1*MgkjZdn82YlVC9w2wXNZBg.png" /><figcaption><strong>Graph for Euclidean Distances</strong></figcaption></figure><p>Euclidean distance=√Σ(y2 — y1)²+(x2 — x1)²</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*wk6fyx8BhmuzkhoqMuSUIg.png" /><figcaption><strong>Euclidean Distances</strong></figcaption></figure><p>As we use distances in clustering to standardization of features is necessary</p><p>for point number 3 linkage with centroid can be use classified into cluster</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/1*iut-MbRGEAg9Keq86rvnCA.png" /><figcaption><strong>Dendrogram of Hierarchical clustering</strong></figcaption></figure><ul><li>More the distance between clusters better clustering will be</li><li>Look for the longest line in the dendrogram and cut it from there</li></ul><p><strong>Advantages:</strong></p><ol><li>It is easy to perform</li><li>it is easy to interpret as well</li></ol><p><strong>Disadvantages</strong></p><ol><li>It does not give the optimal value of clusters. it depends on us to identify and put the number of clusters in the algorithm from the dendrogram.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8cfa8748696c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[K-Nearest Neighbor: The Simple Concept Behind It]]></title>
            <link>https://medium.com/@shubhamc03/k-nearest-neighbor-the-simple-concept-behind-it-e24af375f1fe?source=rss-10419c65c3ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/e24af375f1fe</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[knn]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[k-nearest-neighbours]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Shubham Chaudhari]]></dc:creator>
            <pubDate>Thu, 30 Mar 2023 11:52:27 GMT</pubDate>
            <atom:updated>2023-03-30T11:54:41.515Z</atom:updated>
            <content:encoded><![CDATA[<h4>An Introduction to K-Nearest Neighbor: How it Works and Why it Matters</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nB2KwzOUUuwIeGLX3zixcw.jpeg" /></figure><p><strong>Introduction to K-Nearest Neighbor</strong></p><p>K-Nearest Neighbor is Machine Learning Algorithm that can be used for Regression as well as Classification analysis. It is a non-pragmatic algorithm that does not require any assumption about the underlying distribution of the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*znmxNsFIRQ08hXzYNIZ5wA.png" /></figure><p><strong>Linearly Separable classes</strong></p><ul><li>This applies to binary classification</li><li>Classifier whose decision boundary separates the positive object from the negative ones</li><li>When we draw a single line through data points if it is unable to classify data points clearly we will go for KNN algorithm</li><li>When data points are not linearly separable we can use K-Nearest Neighbor Algorithm</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*35ZnlKi25R5IEmWjScLoNw.png" /></figure><p><strong>K-Nearest Neighbor</strong></p><ol><li>The default value of k is 5.</li><li>k is the number of nearest neighbors or data points that have the least distance from the new observation.</li><li>Working of KNN</li></ol><ul><li>The new data point will calculate the distance from all records.</li><li>After calculating distance it decides which are the nearest data points to a new data point.</li><li>The distance is calculated using Euclidean distance.</li><li>If the value of k = 5, then the KNN algorithm will consider the 5 nearest data points.</li><li>Consider we have 5 data points and out of 5 data points 3 points have a yes class, and 2 data points have a No class if the new data point is near to the data points of the Yes class then KNN will put that new point in Yes class this is done using Euclidean Distance.</li></ul><p><strong>Calculating Euclidean distance</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*t7no0pu2rLnb0JVzJ0qygA.png" /></figure><p>New data point</p><p>X-test = (python = 7 , ML = 6)</p><p>Objective: To decide whether new data points have category pass or fail.</p><p>Euclidean distance</p><p>= √(Σ(Xo-XA))²</p><p>Xo=Observed data (New data)</p><p>XA=Actual values</p><p>= √(7–5)²+(6–7)² = 1.73</p><p>= √(7–2)²+(6–4)² = 5.38</p><p>= √(7–9)²+(6–5)² = 1.73</p><p>= √(7–2)²+(6–3)² = 5.83</p><p>= √(7–8)²+(6–7)² = 1.41</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*_2H_XZ80fpCgYEZBk0MKOQ.png" /></figure><p>Consider k = 3</p><p>least distances = 1.41, 1.73,1.73</p><p>Mode = Pass</p><ul><li>For Classification consider the Mode value.</li><li>For Regression consider the Mean value.</li></ul><p><strong>Step for K-Nearest Neighbor Model Building(For basic Understanding)</strong></p><ol><li>Importing libraries</li></ol><pre>import pandas as pd<br>import numpy as np<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>from sklearn.model_selection import train_test_split<br>from sklearn.neighbors import KNeighborsClassifier</pre><p>2. Importing Dataset</p><pre>df=pd.read_csv(&quot;file_path&quot;)</pre><p>3. EDA</p><pre># Find out data type, null values and duplicate values<br><br>#For data type<br>df.info()<br><br>#For null values<br>df.isnull().sum()<br><br>#duplicate <br>df.duplicated().sum()<br> <br>#And other according to requirement</pre><p>4. Visualization</p><pre>#Use visual for insights<br>sns.distplot(df[&#39;Column_name&#39;])<br>sns.countplot(df[&#39;Column_name&#39;])<br>sns.heatmap(df)<br>sns.pairplot(df)<br>#and other according to requirement</pre><p>5. Splitting of Dataset</p><pre>#Split data into x and y variables<br>x=df[:,:]<br>y=df[:,:]<br>#put target column in y variable and other columns in x variable</pre><p>6. Splitting dataset in train and test</p><pre>xtrain,xtest,ytrain,ytest=train_test_split(x,y,test_size=0.25,random_state=42)</pre><p>7. Model building</p><pre>def predict(model):<br>    model.fit(xtrain,ytrain)<br>    ypred=model.predict(xtrain)<br>    <br>    trainac=model.score(xtrain,ytrain)<br>    testac=model.score(xtest,ytest)<br>    <br>    print(f&quot;Train AC {trainac}\nTest AC {testac}&quot;)<br><br>predict(KNeighborsClassifier())</pre><p>8. Hyperparameter Tuning</p><pre>traiac=[]<br>testac=[]<br><br>for i in range(1,21):<br>    knn=KNeighborsClassifier(n_neighbors=i)<br>    knn.fit(xtrain,ytrain)<br>    ypred=knn.predict(xtest)<br>    <br>    traiac.append(knn.score(xtrain,ytrain))<br>    testac.append(knn.score(xtest,ytest))</pre><pre>sns.set_style(style=&#39;darkgrid&#39;)<br><br>plt.plot(range(1,21),traiac)<br>plt.plot(range(1,21),testac)<br>plt.xlabel(&#39;Number of K&#39;)<br>plt.ylabel(&#39;Accuracy&#39;)<br><br>#put Number of k (n_neighbors) where to line meet from Gragh.<br>#By doing this we are able to minimize the gap between training and testing <br>#accuarcy. and then again use predict function and put value k at n_neighbor <br>#which is determine from graph</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e24af375f1fe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ensemble Techniques: Bagging, Boosting, and Stacking]]></title>
            <link>https://medium.com/@shubhamc03/ensemble-techniques-bagging-boosting-and-stacking-8c3c376026e7?source=rss-10419c65c3ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c3c376026e7</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ensemble-method]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[random-forest]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Shubham Chaudhari]]></dc:creator>
            <pubDate>Wed, 29 Mar 2023 18:08:59 GMT</pubDate>
            <atom:updated>2023-03-30T10:39:14.156Z</atom:updated>
            <content:encoded><![CDATA[<h4>Ensemble Methods Demystified: An Overview of Bagging, Boosting, and Stacking</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0t98eGOU8u2CiZMSZVYlog.jpeg" /></figure><p>The purpose of the ensemble technique is to combine weak learners and form a strong model</p><p><strong>Ensemble Techniques:</strong></p><ol><li>Bagging → Random Forest</li><li>Boosting → Adaboost, Gradient boosting, and Extreme Gradient Boosting</li><li>Stacking</li></ol><p><strong>1. Bagging:-</strong></p><p>Homogeneous weak learners: All models of the same algorithm. For example in bagging all models are built on a single algorithm either linear regression, multilinear regression, or any other algorithm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZBjGGsQg0ztvrskfKlDsnQ.png" /><figcaption><strong>Working of Bagging</strong></figcaption></figure><p>Bagging creates model Paralley: All models learn independently but simultaneously. The learning process in bagging is parallel, not sequential.</p><p><strong>2. Boosting:-</strong></p><p>In Boosting there are homogeneous weak learners. All models have the same algorithm, According to the requirement if you want to build a model on multilinear regression then all other models also build on multilinear regression.</p><p>The learning process in boosting is sequential, not parallel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*WVU9LhE0CiNkyIeRAcvBBw.png" /><figcaption><strong>Working of Boosting</strong></figcaption></figure><p>Model M2 will be completely dependent on model M1 unlike Bagging, bagging considers all models to learn independently.</p><p><strong>3. Stacking:</strong></p><p>In Stacking there are Heterogeneous weak learners. the learning process in stacking is parallel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*plyjqQHwF_I-jin7mtPX9A.png" /><figcaption><strong>Working of Stacking</strong></figcaption></figure><p><strong>Understanding Ensembles</strong></p><p>the meta-learning approach that utilizes a similar principle of creating a varied team of experts is known as an ensemble. All the ensembles method are based on the idea that by combining multiple weak learner models strong models are created</p><p><strong>Working of Ensembles Techniques</strong></p><p>Training data → Allocation of function → (Model=M1,M2,M3) → Combine Function → Ensembles models</p><ol><li><strong>Bagging Technique</strong></li></ol><p>The bagging technique uses bootstrap aggregation</p><p>Bootstrap means sampling with replacement</p><p>For example, Dataset having datapoint between 1 to 100</p><p>M1 = 1, 2, 50, 100</p><p>M2 = 1, 3, 10, 20</p><p>M3 = 1, 5, 3, 50</p><p>Model 1 is trained on a random sample of the training dataset, where each data point is selected with a replacement. Model 2 is trained on another random sample of the training dataset, again with the replacement. This process is repeated for multiple models, with each model trained on a different random sample of the training dataset. Finally, the predictions of all the models are combined, typically by averaging, to obtain the final prediction.</p><p>For regression take a mean of all predicted values of all models as the final output</p><p>For Classification take a mode of all output values as a final output</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U3jdt8C1Md2ecQ7nNtmybQ.png" /><figcaption><strong>Prediction of Classification</strong></figcaption></figure><p><strong>Random Forest</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/858/1*M66eoArMx9IZbMqJNVbSHA.png" /><figcaption><strong>Decision tree nodes description</strong></figcaption></figure><ul><li>Random forest based on Decision Tree</li><li>Root node → First or starting node of decision tree</li><li>leaf node → Node after that split can not happen</li><li>Decision node → Node after that split can happen</li><li>Based on Entropy and Gini impurity which feature will be the root node decided</li></ul><p>Random Forest:- It combines multiple Decision trees therefore known as Random Forest</p><p>In the case of random sampling that is training and testing dataset. data sample is taken out from columns as well as rows.</p><p>In random forests, multiple decision trees are built and their combined output is considered.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*wrOsT89nOILLJu9W66J6pw.png" /><figcaption><strong>Working of Random Forest</strong></figcaption></figure><p><strong>Random Forest pseudocode</strong></p><ol><li>Randomly select “k” features from “m” features, where k&lt;&lt;m</li><li>k = selected column for Sampling</li><li>m = All columns of the dataset</li><li>Among the k column, calculate the node “d” using the best-split point</li><li>Split the node into daughter nodes using the best split</li></ol><p>In this case, X1, X10, and X18 are randomly selected features, and among them, X10 has the lowest Gini impurity, which means it is the best feature to split the data. Therefore, the algorithm will split the data using the X10 feature.</p><p>6. Repeat 1 to 5 steps until the “l” number of nodes has been reached</p><p>7. Build a Forest by repeating steps 1 to 4 for “n” number times to create “n” number of tree</p><p><strong>Advantage of Random Forest</strong></p><ol><li>The same random forest algorithm can be used for Regression as well as classification of both task</li><li>Random forest classifiers can handle missing values</li><li>The overfitting problem will never come when we use the random forest algorithm in any classification problem</li><li>The random forest algorithm can be used for feature engineering which means identifying the most important features out of the available features from the training dataset</li></ol><p>Random Forest can not have other models other than the Decision tree</p><p><strong>2. Boosting</strong></p><p>Boosting algorithms usually work by assigning weights to the data points based on their error rate in the previous models. The weights are then used to sample the data for the next model, with more weight given to the misclassified data points. This approach ensures that the models focus on the difficult examples and gradually improve their accuracy.</p><ol><li><strong>Adaboost (Adaptive boosting)</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/594/1*izCOQ49bCwnsd2uLj4k76w.png" /></figure><p>step 1: Assign weights to all records</p><p>wo = 1/n = 1/0.5 = 0.2 -Giving equal importance to all data points</p><p>step 2: Calculate the misclassification rate</p><p>Misclassification rate MR = Wrongly classified data / Total data points</p><p>MR = (WD/TD) = ( 2/5 ) = 0.4</p><p>step 3: Calculate the performance of the stump</p><p>Decision stump: A decision tree having a single depth is called a decision stump</p><p>in boosting if decision trees have data for further splitting then also it is not split it only split for only single depth.</p><p>stump also called as stage</p><p>stage = ln(1-MR/MR)</p><p>=ln ( 1–0.4/0.4 ) = 0.6/0.4 = 0.4</p><p>ln is a natural log</p><p>step 4: Update weights</p><p>wn = wo x e^stump</p><p>= (0.2 x e ^ 0.4 )</p><p>= 0.29</p><p>The sum of total weight should be 1 for wo = 1 for wn = 1.18</p><p>for wn = New weight</p><p>wn should be normalized</p><p>New weight =wn/Total Weight</p><p><strong>2) Gradient boosting</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/668/1*FmPQreUFKIbeB1QPTyCwrg.png" /></figure><p>ȳ = Avg of target column = (160+167+170+185+200)/5=176</p><p>r1 = ( y-ȳ) = (160–176) = -16</p><p>r1 = residual</p><p>r2 = This model works on the residual of r1 model</p><p>ŷ= ȳ + l1 x r2</p><p>= 176 + 0.1 x (-13)</p><p>= 174.7</p><p>l1 = Learning rate is hyperparameter, Default lr = 0.1, lr range = 0 to 1</p><p>the process is repeated until you reach the desired value.</p><p><strong>3. Stacking:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*ML7jcEWzQDqOm-UEC1LWMg.png" /><figcaption><strong>Working of Stacking for Different Models</strong></figcaption></figure><p>Stacking works on Heterogeneous models all data will pass through M1, M2, and M3 whatever prediction gets from all these models. we will treat all these predictions as features</p><p>these features are passed through the model against the original target column</p><p>in stacking boosting, the algorithm can be used</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/719/1*LQ2DbXR3EpZqJ85Pg7ICAw.png" /><figcaption><strong>Working of stacking for Ensemble techniques</strong></figcaption></figure><p>prediction from these ensemble techniques used as features till desired output gets.</p><p>Step for model building using Ensembles Techniques(For basic understanding )</p><ol><li>Import Libraries</li></ol><pre>import pandas as pd<br>import numpy as np<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>from sklearn.model_selection import train_test_split<br>from sklearn.linear_model import LogisticRegression<br>from sklearn.tree import DecisionTreeClassifier<br>from sklearn.neighbors import KNeighborsClassifier <br>from sklearn.ensemble import VotingClassifier,BaggingClassifier,StackingClassifier<br>from sklearn.ensemble import AdaBoostClassifier,GradientBoostingClassifier<br>from xgboost import XGBClassifier<br>from sklearn.metrics import classification_report,confusion_matrix</pre><p>2. Importing Dataset</p><pre>df=pd.read_csv(&#39;file_path&#39;)</pre><p>3. EDA</p><pre># Find out data type, null values and duplicate values<br><br>#For data type<br>df.info()<br><br>#For null values<br>df.isnull().sum()<br><br>#duplicate <br>df.duplicated().sum()<br> <br>#And other according to requirement</pre><p>4. Visualization</p><pre>#Use visual for insights<br>sns.distplot(df[&#39;Column_name&#39;])<br>sns.countplot(df[&#39;Column_name&#39;])<br>sns.heatmap(df)<br>#and other according to requirement</pre><p>5. Splitting of Dataset</p><pre>#Split data into x and y variables<br>x=df[:,:]<br>y=df[:,:]<br>#put target column in y variable and other columns in x variable</pre><p>6. Splitting dataset in train and test</p><pre>xtrain,xtest,ytrain,ytest=train_test_split(x,y,test_size=0.25,random_state=42)</pre><p>7. Model building</p><pre># Make a function and provide Ensemble techniques as input, Use hyperparameter<br>#tuning according to requirement<br>def predict(model):<br>    model.fit(xtrain,ytrain)<br>    ypred=model.predict(xtest)<br>    <br>    traiac=model.score(xtrain,ytrain)<br>    testac=model.score(xtest,ytest)<br>    <br>    print(f&quot;Training accuracy {trainac}\n test accuracy {testac}&quot;)<br>    <br>    print(classification_report(ytest,ypred))<br>    print(confusion_matrix(ytest,ypred))</pre><p>8. Voting Classifier</p><pre>predict(VotingClassifier(estimators=models))</pre><p>9. Bagging classifier</p><pre>#Logistic Regression<br>predict(BaggingClassifier(LogisticRegression()))<br><br>#Knighbor classifier<br>predict(BaggingClassifier(KNeighborsClassifier()))<br><br>#Decision Tree<br>predict(BaggingClassifier(DecisionTreeClassifier()))</pre><p>10. Random Forest</p><pre>predict(RandomForestClassifier())<br># Use hyperparameter tuning to get desired result</pre><p>11. Ada-boost</p><pre>predict(AdaBoostClassifier())</pre><p>12. Gradient Boosting Classifier</p><pre>predict(GradientBoostingClassifier())</pre><p>13. XGBClassifier</p><pre>predict (XGBClassifier(max_depth=2))</pre><p>14. Stacking</p><pre>st=StackingClassifier(estimators=models,final_estimator=DecisionTreeClassifier(criterion=&#39;entropy&#39;))<br>st.fit(xtrain,ytrain)<br>yhat=st.predict(xtest)<br><br>print(st.score(xtrain,ytrain))<br>print(st.score(xtest,ytest))</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c3c376026e7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep Learning: Artificial Neural Network]]></title>
            <link>https://medium.com/@shubhamc03/deep-learning-artificial-neural-network-54c955a14b43?source=rss-10419c65c3ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/54c955a14b43</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[artificial-neural-network]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Shubham Chaudhari]]></dc:creator>
            <pubDate>Tue, 28 Mar 2023 14:56:10 GMT</pubDate>
            <atom:updated>2023-03-28T16:19:48.013Z</atom:updated>
            <content:encoded><![CDATA[<h4>Concept of an Artificial Neural Network, Gradient Descent, and Model building steps</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rL3XKlNAl6P9l20tS68G_w.jpeg" /></figure><p>Artificial Intelligence Mimic human behavior, Machine learning train machines for predictions, and Deep learning Mimic brain and make Decision.</p><p>Deep learning: Supervised deep learning</p><p><strong>Brain parts and their functions</strong></p><ol><li>Temporal lobe:- Responsible for Data preprocessing → Artificial Neural Network.</li><li>Occipital lobe:-Responsible for Vision → Convolutional Neural Network.</li><li>Frontal lobe:- Responsible for Logical and analytical thinking → Recurrent Neural Network.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iW5sJ7rjDIRED6sNcxgXaw.png" /><figcaption><strong>Structure of Neuron</strong></figcaption></figure><h3><strong>Artificial Neural Network</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*egJwUrwSS43C6Bfe9uiBvQ.png" /><figcaption><strong>Working of Neural Network</strong></figcaption></figure><p>X1, X2, X3 = Input Features</p><p>B = Bias (Error) - External Bias</p><p>Σf(x) = Activation function</p><p>w = Random amount of weight</p><p>yp = predicted y</p><p>y = Actual y</p><ul><li>X1, X2, and X3 are input features, along with these features add some Bias</li><li>Bias is some amount of error</li><li>All these input features are passed through ANN Algorithm</li><li>The middle part is called Neuron in this neuron we have something called Activation Function</li><li>By default, the algorithm will assign some weight to each and every feature</li><li>After assigning weight to a feature it calculates the weighted sum of input features</li><li>The Activation function does the weighted sum of input features. An activation function is calculations and formulae</li><li>After getting weighted sum prediction is done (y predict)</li><li>and then error calculation Is done</li><li>Error in ANN is calculated by Mean square error method (MSE)</li></ul><p>mse = ( y- ŷ / N )</p><p>Input features → Activation function → Prediction → Calculation of error. This flow is called forward propagation</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*fuWzhcrjJvgC5p3CSslJ5w.png" /><figcaption><strong>Error Minimisation after the first cycle</strong></figcaption></figure><ul><li>Based on the learning from predictions input features will have New weights.</li><li>Based on which features will features are important new weights are assigned.</li><li>New bias values are also passed</li><li>In order to decrease error it comes back to input features and reassigns new weights to them. that will increase the accuracy and decrease error.</li><li>This process which is going backward called Backword Propagation</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*laNzD6u9gfgdT6sVz-cf9w.png" /><figcaption><strong>Error Minimisation after the second cycle</strong></figcaption></figure><ul><li>This cycle of forward propagation and backward propagation is continuous until it gives the least or minimum error.</li><li>Backpropagation is also called feed-forward Propagation</li></ul><p><strong>Equation Use by ANN:-</strong></p><p>Linear regression equation :- y = mx + c</p><p>y = wx + B</p><p>y = Target</p><p>w = weight</p><p>x = input features</p><p>B =Bias</p><p><strong>Gradient Descent:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*g465jtc6cXJ4EqGI2j5k1A.png" /><figcaption><strong>Gradient Descent</strong></figcaption></figure><p>The optimal value of weight and Bias can be determined by using Gradient descent</p><ul><li>After one forward propagation cycle, it gives an error of 7 and n-step</li><li>After the second forward propagation cycle, it gives an error of 6 and n+1 step and it follows until we get global minima.</li><li>The distance between n and n+1 is called the learning rate.</li><li>The learning rate is a hyperparameter</li><li>The minimum point is called the Global minima</li><li>The goal of gradient distance is to reach global minima</li><li>It is recommended that use small values of learning rate as large values will miss global minima</li><li>for identifying global minima after step n+7</li></ul><p><strong>Types of Gradient Descent</strong></p><ol><li>Gradient Descent</li><li>Stochastic Gradient Descent</li><li>Mini-Batch Gradient Descent</li></ol><p><strong>1.Gradient Descent:-</strong></p><p>Gradient descent is used for error calculation</p><p>mse = ( y- ŷ / N )</p><p>and it is time-consuming.</p><p><strong>2. Stochastic Gradient Descent:-</strong></p><p>The error is calculated on one and only on a single data point.</p><p>error = ( y - yp )²</p><p>Divided by N is excluded because of a single data point.</p><p><strong>3. Mini-batch Gradient Descen</strong>t</p><p>error = ( y - yp )²/ K</p><p>k is the number of batches selected for error calculation suppose randomly it will choose 1 million records from 10 million records. This gradient descent will have more accuracy than stochastic gradient descent as stochastic gradient descent use only one data point for error calculation.</p><p>k value should be very much less than N</p><p>The formula used by gradient descent to assign a weight for input features</p><p>wn= wo — lr X (d/dw)(mse)</p><p>wn=New Weight</p><p>wo=Old Weight</p><p>lr = learning rate</p><p>(d/dw) = Derivative of mse</p><p><strong>Activation Function</strong></p><ol><li>Threshold function</li><li>Sigmoid function</li><li>Rectified linear unit (Relu)</li><li><strong>Threshold function</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*UDzb_00tvUDPjiGjBy_CpQ.png" /><figcaption><strong>Working of Threshold function</strong></figcaption></figure><p>f(x) = 1,if x ≥ 0</p><p>and f(x) = 0, if x &lt; 0</p><p>return 1 if x is greater than 0 and return 0 if x is less than 0</p><p><strong>2. Sigmoid function</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/569/1*2W5fQ-BFsszfy88cSTBS2A.png" /><figcaption><strong>Working of sigmoid function</strong></figcaption></figure><p>f(x) = 1/(1+e^-z)</p><p><strong>3. Rectified linear unit (Relu)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/608/1*jmn6kaTN6R9P98ozEpr6Og.png" /><figcaption><strong>Working of relu function</strong></figcaption></figure><p>f(x) = max(0,x)</p><ul><li>maximum value will be 0</li><li>return 0 if x value is 0, as maximum value is 0 ,return x the exact value that is negative value</li><li>return 0 if value of x is 0 return x that is function value if negative</li></ul><p>A neural network with a single nucleus called the perceptron</p><p>Neural Network with multiple nucleus called the multi-level perceptron</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/977/1*DoePbst1M3KKVpNS-P1o2Q.png" /><figcaption><strong>Gradient descent three dimensional</strong></figcaption></figure><p><strong>Gradient Descent 3D:-</strong></p><p>Normally we plot data in two-dimensional space but actual data is in three-dimensional space</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qjfbINmkeuoQBdxQnQMVSA.png" /><figcaption><strong>Decreasing learning rate and Application of Adam Optimizer</strong></figcaption></figure><p>If you keep the same learning rate throughout the analysis it is going to miss global minima</p><p>as decreasing learning rate should also decrease this is done by algorithm RMSProp</p><p>the problem with RMSProp is that it will reach glabal minima in zigzag mammer</p><p>this convergence will be time-consuming</p><p>it should go in straight to global minima which add some momentum</p><p><strong>Adam Optimizer:-</strong></p><ul><li>Optimize or improve the version of the gradient descent</li><li>Adam Optimizer is a combination of RMSProp + Momentum</li></ul><p><strong>Multilevel Neural Network</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/1*EMgZE0BEmlaUatpkiiDBnw.png" /><figcaption><strong>Multilevel neural network</strong></figcaption></figure><ul><li>Every input feature passes through every neuron</li><li>if you assign activation function Relu all neurons will have activation function Relu</li><li>only one activation function is used in all hidden layers</li><li>If the data is linearly separable only then a single layer called perceptron is going to work</li><li>If the data is non-linear separable then multiple hidden layers will help to analyze the data</li></ul><p>In the case of classification, the hidden activation function should be Relu (Rectified linear unit)</p><p>In case of classification hidden Output layer (Target column having binary classification(0,1), (Yes, No) use a sigmoid Activation function</p><p>In case of regression for hidden layers as well as output layers use Relu (Rectified linear unit)</p><p>Use Hyperparameter tuning in case of Overfitting(early, dropout)</p><p><strong>Steps for Artificial Neural Network</strong></p><ol><li>Importing Libraries</li></ol><pre>import pandas as pd<br>import numpy as np<br>import matplot.pyplot as plt<br>import seaborn as sns<br>from sklearn.preprecessing import StandardScaler<br>from sklearn.model_selection import train_test_split<br>from tensorflow.keras.models import Sequential<br>from tensorflow.keras.layers import Dense<br>from tensorflow.keras.callbacks import EarlyStopping</pre><p>2. Importing Dataset</p><pre>df=pd.read_csv(&quot;file_path&quot;)</pre><p>3. EDA</p><pre># Find out data type, null values and duplicate values<br><br>#For data type<br>df.info()<br><br>#For null values<br>df.isnull().sum()<br><br>#duplicate <br>df.duplicated().sum()</pre><p>4. Visualization</p><pre>#Use visual for insights<br>sns.distplot(df[&#39;Column_name&#39;])<br>sns.countplot(df[&#39;Column_name&#39;])<br>sns.heatmap(df)<br>#and other according to requirement</pre><p>5. Standardisation</p><pre>sc=StandardScaler()<br>sc.fit_transform(df)</pre><p>6. Splitting of Dataset</p><pre>#Split data into x and y variables<br>x=df[:,:]<br>y=df[:,:]<br>#put target column in y variable and other columns in x variable</pre><p>7. Splitting dataset in train and test</p><pre>xtrain,xtest,ytrain,ytest=train_test_split(x,y,test_size=0.25,random_state=42)</pre><p>8. Model building</p><pre>#For bianry classification use sigmoid activation function for output layer<br>#Model<br>ann=Sequential()<br>ann.add(Dense(units=6,activation=&#39;relu&#39;))<br>ann.add(Dense(units=6,activation=&#39;relu&#39;))<br>ann.add(Dense(units=1))<br><br>#Compile<br>ann.compile(optimizer=&#39;adam&#39;,loss=&#39;mean_squared_error&#39;)<br><br>#Fitting data <br>ann.fit(x=xtrain,y=ytrain,epochs=100,batch_size=32,validation_data=(xtest,ytest),callbacks=EarlyStopping(monitor=&#39;val_loss&#39;,patience=4))</pre><p>9. Evaluation</p><pre>#Training Evaluation<br>ann.evaluate(xtrain,ytrain)<br><br>#Testing Evaluation<br>ann.evaluate(xtest,ytest)</pre><p>10. Predicted Values</p><pre>#for comparing real values and predicted values<br>predictions=ann.predict(xtest)<br>predictions_df=pd.DataFrame(np.ravel(predictions),columns=[&#39;predictions&#39;])<br>comparison_df=pd.concat([pd.DataFrame(ytest,columns=[&#39;Real values&#39;]),predictions_df],axis=1)<br>comparison_df</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=54c955a14b43" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Support Vector Machine]]></title>
            <link>https://medium.com/@shubhamc03/understanding-support-vector-machine-df7e2ac7e98e?source=rss-10419c65c3ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/df7e2ac7e98e</guid>
            <category><![CDATA[machine-learing]]></category>
            <category><![CDATA[support-vector-machine]]></category>
            <category><![CDATA[kernel]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Shubham Chaudhari]]></dc:creator>
            <pubDate>Mon, 27 Mar 2023 15:59:18 GMT</pubDate>
            <atom:updated>2023-03-28T07:15:07.348Z</atom:updated>
            <content:encoded><![CDATA[<h4>Concept of Support Vector Machine and Model building Steps with code</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U6zRugD0cKBUMzu418575g.jpeg" /><figcaption>Machine Learning</figcaption></figure><p><strong>Support Vector Machine :</strong></p><p>Support vector machine is a supervised machine learning method that can be used for Regression and Classification purposes.</p><p><strong>Point estimate</strong>: A point estimate is nothing but a single value and the probability associated with single values is zero.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*CcE2c_UkjTTpyqXO3gpiMA.png" /><figcaption><strong>Point Estimate</strong></figcaption></figure><p><strong>Interval Estimate</strong>: If we give a certain amount of range there are more chances of getting the prediction right as the value will fall in a certain range and a confidence interval or interval estimate is always better than a point estimate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/789/1*cwuCg-_i5tzRLqRmd1jNsw.png" /><figcaption><strong>Interval Estimate</strong></figcaption></figure><p>If the data can be classified with a line then it is called linearly separable data</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*wXS-EAa9VG9ro8RQOLz6xw.png" /><figcaption><strong>Linearly Separable Data</strong></figcaption></figure><p>and if the data can not be classified with a line then it is called a lineally non-separable data in that case we should use SUPPORT VECTOR MACHINE</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*0IOXOvKnk26oRsVVDQ-ecQ.png" /><figcaption><strong>Non Linearally Separable Data</strong></figcaption></figure><h4><strong>Concept of Support Vector Machine</strong></h4><p>Let’s take an example for better understanding consider we take a bottle and take look from the top view. if we have to determine the height of the bottle from the top view then is not possible so we have to change angle of direction of looking at the bottle (side view) then only we can determine the height of the bottle.</p><p>Whatever data we are seeing is non-linear is because of angle that we are seeing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/1*0td8drZyWXSxPCGIqUZ33A.png" /><figcaption><strong>Top and Side View</strong></figcaption></figure><p>In SVM we will convert this two dimension data into three dimensional subspace. just to check whether there is a difference between angle or not so there are chances that the majority of the time non linearlly separable data in two dimensional space can be separable in three dimentional space.if you try to map two dimentional data into three-dimensional spaces it will look linear</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qcMpaTVPMi08u5IRAufzbg.png" /><figcaption><strong>Working Of Support Vector Machine</strong></figcaption></figure><h4>Terminologies in Support Vector Machine</h4><ol><li>In Three dimensional data, the line which separates data is called a PLANE.</li><li>Support vectors are those data points that are close to the hyperplane</li><li>If any data points are inside the margin of error they will not be considered as your error data points they will ignore them will keep some amount of margin with respect to the error</li><li>Decision boundary separates two classes of hyperplane</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*1lNBbVhQopeV9jmcTlnbvw.png" /><figcaption><strong>Terminology</strong></figcaption></figure><h4>Kernel</h4><p>Support vector machine maps two-dimensional data into three-dimensional data with the help of kernel</p><p>In two-dimensional data, there are two axes X and Y but in three-dimensional data, there are three axes X, Y, and Z to add a third axis in two-dimensional data to make it a three-dimensional support vector machine used different kernels</p><p><strong>Types of Kernels:</strong></p><ol><li>Radial Basis Function Kernel</li><li>Polynomial Kernel</li><li>Linear Kernel</li></ol><p><strong>1.Radial Basis Function Kernel (RBF kernel):</strong></p><p>Rbf creates a circular decision boundary, not a linear or straight line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*gX_8YwXyYv2xEu2kIMadZw.png" /><figcaption><strong>Working Of Rbf Kernel</strong></figcaption></figure><p><strong>2. Polynomial kernel:</strong></p><p>polynomial kernel changes one-dimensional data into two-dimensional subspace</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uruVpsRZ_UvF_TMvMl7kMg.png" /><figcaption><strong>Working Of Polynomial kernel</strong></figcaption></figure><p><strong>3. Linear Kernel</strong></p><p>If data is linearly separable we use the linear kernel</p><p>All kernels use an equation to map two-dimensional data into three-dimensional data</p><p>support vectors are responsible to create decision boundaries</p><h3>Steps of Support Vector Machine Model Building</h3><ol><li><strong>Importing Libraries</strong></li></ol><pre>import Pandas as pd<br>import numpy as np<br>import Matplotlib.pyplot as plt<br>import seaborn as sns<br>from sklearn.model_selection import train_test_split<br>from sklearn.preprocessing import StandardScaler<br>from sklearn.metrics import classification_report<br>from sklearn.svm import SVC</pre><p><strong>2. Importing Dataset</strong></p><pre>df=pd.read_csv(&#39;file_path&#39;)</pre><p><strong>3. EDA</strong></p><p>Find Data type, null values, Statistical data and duplicate values and other EDA should be done according to requirement.</p><pre># For Data type and other info<br>df.info()<br><br># For NUll Values<br>df.isnull().sum()<br><br># For Stastical Data<br>df.describe()<br><br># For duplicates<br>df.duplicated().sum()</pre><p><strong>4. Label Encoding</strong></p><p>According to the requirement label encoding of categorical data should be done</p><pre>le=LabelEncoder()<br>le.fit_transform(df)</pre><p><strong>5. Standardization</strong></p><p>According to requirements Standardization should be done</p><pre>SC=StandardScaler()<br>sc.fit_transform(df)</pre><p><strong>6. Visualization</strong></p><p>For a Better understanding of data and to get insights Visualization should be done</p><p>matplotlib and seaborn libraries can be used for visualization</p><ol><li>Distplot</li><li>Heatmap</li><li>countplot and others according to the requirement</li></ol><p><strong>7. Splitting of data</strong></p><p>Divide data into x and y variable</p><p>x will be data on which target values will be calculated</p><p>y will be the target column, put columns index into brackets</p><pre>x=df.iloc[:,:]<br>y=df.iloc[:,:]</pre><p><strong>8. Splitting data into train and test dataset</strong></p><pre>xtrain,xtest,ytrain,ytest=train_Test_split(x,y,test_size=0.25,random_state=42)</pre><p><strong>9. Model building</strong></p><pre># For Rbf kernel<br>svc_rbf=SVC(kernel=&#39;rbf&#39;,random_state)<br>svc_rbf.fit.(xtrain,ytrain)<br>ypred_rbf=svc_rbf.predict(xtest)<br>trainac_rbf=svc_rbf.score(xtrain,ytrain)<br>testac_rbf=svc_rbf.score(xtest,ytest)<br>print(classification_report(ytest,ypred_rbf))</pre><pre># For poly Kernel<br>svc_poly=SVC(kernel=&#39;poly&#39;,random_state=1)<br>svc_poly.fit.(xtrain,ytrain)<br>ypred_poly=svc_poly.predict(xtest)<br>trainac_poly=svc_poly.score(xtrain,ytrain)<br>testac_poly=svc_poly.score(xtest,ytest)<br>print(classification_report(ytest,ypred_poly))</pre><pre># For linear kernel<br>svc_linear=SVC(kernel=&#39;linear&#39;,random_state=1)<br>svc_linear.fit.(xtrain,ytrain)<br>ypred_linear=svc_linear.predict(xtest)<br>trainac_linear=svc_linear.score(xtrain,ytrain)<br>testac_linear=svc_linear.score(xtest,ytest)<br>print(classification_report(ytest,ypred_linear))</pre><p><strong>10. Evaluate Results</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df7e2ac7e98e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>