<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by DITI MODI on Medium]]></title>
        <description><![CDATA[Stories by DITI MODI on Medium]]></description>
        <link>https://medium.com/@diti.modi?source=rss-35e9bf238b81------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Z8KyGgd3nokuObhiGOAcbg.jpeg</url>
            <title>Stories by DITI MODI on Medium</title>
            <link>https://medium.com/@diti.modi?source=rss-35e9bf238b81------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 16:58:11 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@diti.modi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Here’s why you should not miss Kamshet from your Lonavala itinerary…]]></title>
            <link>https://medium.com/@diti.modi/heres-why-you-should-not-miss-kamshet-from-your-lonavala-itinerary-df07e307f92f?source=rss-35e9bf238b81------2</link>
            <guid isPermaLink="false">https://medium.com/p/df07e307f92f</guid>
            <category><![CDATA[travel-writing]]></category>
            <category><![CDATA[india]]></category>
            <category><![CDATA[travel]]></category>
            <category><![CDATA[adventure]]></category>
            <category><![CDATA[paragliding]]></category>
            <dc:creator><![CDATA[DITI MODI]]></dc:creator>
            <pubDate>Sun, 27 Dec 2020 12:38:12 GMT</pubDate>
            <atom:updated>2021-01-07T16:16:00.496Z</atom:updated>
            <content:encoded><![CDATA[<h3>Here are Top 4 reasons why you will love Kamshet more than Lonavala…</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HxQTIJ1BQ8w9ogrQBMDcwQ.jpeg" /><figcaption>Kamshet</figcaption></figure><p>Yes, being a Mumbaikar, just like you, I , too, scorn at the idea of visiting the much-famous Lonavala for weekends, especially during holiday season :P</p><p>So to escape the chaos of Lonavala on almost every weekend, especially during holidays, here is a great place that you cannot miss-Kamshet! If you are an aspiring adventurer, I am sure you would already know that Kamshet is most known for paragliding. In fact, many foreigners visit Kamshet especially for cross-country paragliding. Here is an attempt to explore Kamshet in this blog and also discover few bonus tips to ensure best travel time.</p><h3>My Experience at Kamshet</h3><p>As I opened the door of the balcony, the sublime view instantly made me experience the serenity of being connected to nature.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aeovSlK68dzj8E4vFIG8rg.jpeg" /><figcaption>Balcony view</figcaption></figure><p>I settled down into a comfortable chair in the spacious balcony. As the gusty winter winds brushed my hair on my face, the warm sunlight painted my skin with yellowish golden hues. The mightier the winds got every minute, stronger the sunlight gleamed.</p><p>As I reached out towards the railings of the balcony, I could breathe the bougainvillea scented air and hear the rustling leaves. It was as if the copse welcomed me with enthusiasm and awe to their luscious abode. They never knew that my sojourn would end soon.</p><p>At a distance, I could sight a part of the sparkling lake through the branches of the trees. The leaves and branches blocked my clear view of the blue lakeside. The sparrows chirped in sync with each other. They were conversing with the winds as if they requested it to slow down so that they could build their nests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*87wmNkXJNsoNm-o47EB0rQ.jpeg" /><figcaption>Sunlight beaming in it’s full potential</figcaption></figure><p>So are you already in love with this amazing place?</p><h3>Kamshet- Epicenter for Paragliding</h3><p>The weather at this time of the year is said to be the best for chilling and paragliding as the wind conditions are much favored for learning the sport. Also, if you are an aspiring ornithologist or florist, there are a lot of flowers and migrating birds that visit Kamshet around this time since it is surrounded by lakes from all sides. But wait, don’t mistake the carefree paragliders all over the village to be some new kind of birds :P</p><p>Carefully watch the image below clicked by my colleague on a random evening visit in the village. Those are nothing but the carefree paragliders who swiftly glide and soar in the evening skies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_noIKV0noFCwIN19JrIZXQ.jpeg" /><figcaption>Yes, those are paragliders and not birds.</figcaption></figure><p>Since Kamshet is most known for the paragliding courses and enjoying the windy skies, it truly makes you anticipate and wonder about the joys of flying.</p><h3>Where is it located?</h3><p>From Mumbai: Around 110 kilometers. (~2.5 hours)</p><p>From Pune: Around 45 kilometers. (~1.5 hours)</p><p>From Lonavala: Around 28 kilometers. (~45 mins)</p><p><strong>Bonus Tip #1: </strong>You will reach Kamshet village which has market at it’s center and a local railway station nearby. So if you do not have any food for your journey ahead, then it is recommended that you buy something here itself.</p><h4>Network Connectivity</h4><p>Being a village,<strong> </strong>the network connectivity is definitely not the best here. But it is actually one of the best things for people like me who prefer solitude and disconnectivity from all the chaos of city life.</p><blockquote><em>“The first step to really connect with yourself is to disconnect with everything else.”</em></blockquote><p><strong>Bonus Tip #2:</strong> Jio network works quite well than any other ;)</p><p>So here we are, with the much-awaited discussion as to why should I include Kamshet in my itinerary or “To-Be-Explored” list. Let me take you through the top 4 reasons why you would want to visit Kamshet over and over again:</p><h4>1. Paragliding</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FvbS6qhdqoXU%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DvbS6qhdqoXU&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FvbS6qhdqoXU%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2a9a3150cb2d2593fa37458b284d50d7/href">https://medium.com/media/2a9a3150cb2d2593fa37458b284d50d7/href</a></iframe><p>I know, this is predictable. But while I was visiting Kamshet, though I knew about paragliding, I was more inclined to just chilling rather than paragliding. But on the first day itself, when I visited one of the paragliding sites (Ashram site) with other aspiring paragliders, I instantly got excited to learn this sport the next time I visit Kamshet. Here are few snaps of the site that I had visited.</p><p>There are 3 sites for learning paragliding in Kamshet according to the learning stage you are at- Ashram, Tower and Shelar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a90s4ruYSIH215Nga9pNOw.png" /><figcaption>Ashram site</figcaption></figure><p>Ashram site is only used for the groundwork learning like handling the glider, managing wind direction, observing Windsoc(apparently a tool to judge the wind direction) etc. before you actually learn gliding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x33CT1KD4Q1_KAcwEtIFEg.jpeg" /><figcaption>A fellow learner learning the technicalities of using the glider for the first time at Ashram</figcaption></figure><p>Below is the video of the Tower site when the learners have completed their groundwork training and now are ready to fly with the help of the instructors.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F92kHrxLgMcI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D92kHrxLgMcI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F92kHrxLgMcI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/85ad3c597c4b7f7f0c05c5757a17b8aa/href">https://medium.com/media/85ad3c597c4b7f7f0c05c5757a17b8aa/href</a></iframe><p>I can write another full blog about paragliding sometime later but I hope, by now, these videos and images have inspired you enough to learn this exciting sport.</p><p>This is the most intuitive sport to learn and requires a lot less efforts compared to learning any other adventure sport. There are various levels of learning/ courses that are offered according to the time availability and willingness of the learner. Also if you just want to have fun in the air without having to learn paragliding, there is something called as “Tandem” wherein an expert pilot would be in front of you to perform all the technicalities in the air while you just sit back and relax to enjoy the view and feels from the top.(“Top of the world” feels :P)</p><h4>2. Hiking/ Trekking</h4><p>There are many famous one-day treks in and around Kamshet like Lohagad fort trek.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FMlkR2mcgjQ8fMfB1EAtJA.jpeg" /><figcaption>View from a nearby hilltop</figcaption></figure><p>Initially, I had thought of trekking to Lohagad but later on, hiked to a nearby hilltop with a fellow localite impromptu.</p><p>Though it was just a 30-minute hike to a cave on a nearby hilltop, it was one of the most exciting hikes I have had. The views from a hilltop on an early winter morning were definitely a big reason that added to my excitement. Here are few snaps to visualize.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jb_7xo0lR6PvcHV26d6kBg.jpeg" /><figcaption>View of the Western Ghats from the hilltop</figcaption></figure><p>But one of the reasons why I enjoyed the hike was the details and new facts that I learnt on the way to the top from the fellow native with whom I hiked to the top, which also leads me to my next point…</p><h4>3. Talking to a local villager or fellow sojourner</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pIxeKV5vT2KIYJY1Gsx1GQ.jpeg" /></figure><p>Talking to a native who was staying at Kamshet since quite some time, I realized subtle nuances about how mother nature has devised the best form of natural paragliders in the form of birds and the best science to learn in the form of plants and insects.</p><p>For me, this reason is my personal favorite to visit any travel destination across the globe.</p><p>Talking to fellow travelers, we realize how each of our stories is unique and exciting. For example, here, when I was staying at my accommodation, I had conversations with multiple sojourners who shared their varied experiences of learning other sports or travelling across the country, staying in jungles at times or undertaking a 2-month road trip to Rajasthan even during such pandemic times.(Yeah, quite exciting isn’t it?)</p><p><strong>Bonus Tip #3: </strong>One of the most important survival tip that I got to learn this way is if you are a beginner for solo trekking/hiking and often find yourself confused with the trails or get lost all the time while returning back from the summit/hilltop, it is always advised to stop at the diversion while going uphill itself and turn.</p><p>Now look back at the view from that point and trace back the same way from where you came. Now mark a landmark like the huge Eucalyptus tree on the left or a small stream of water on the right or the small hut nearby. This allows you to visualize and landmark certain key points in your mind so that while returning, you can quickly get down with the help of these marks.</p><h4>4. Chilling</h4><p>I know this would be your best to-do for a short trip to such places since this place is one of the most calming and chaos-free place outside of the city life. After all, chilling on a hammock under the soothing shade of a huge tree is indeed, quite cherishing at the end.</p><p><strong>Bonus #4: </strong>For adding cherry on the cake, you could also go camping here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lxxfpRurYjmzSBYIHdykJQ.jpeg" /><figcaption>Yes, those are my legs while chilling on a hammock</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jmdxSqCrfNQywuN2UWRrtw.jpeg" /><figcaption>Just the right kind of shade and sunlight</figcaption></figure><h4>4. Visiting the lake</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QjycKf5_z0gEyqwHghW_NA.jpeg" /><figcaption>Lake at it’s best during noon</figcaption></figure><p>And the last Must-Do at Kamshet is to visit at least one lakeside. As Kamshet is surrounded by multiple lakes, sitting by any lake side would be as rewarding as hiking to a hilltop. Since I had an easy access to one of the lakes just near the place that I stayed at<strong>,</strong> an early morning stroll around the lake was a no-brainer.</p><p><strong>Bonus Tip #5:</strong> It is recommended to visit the lake in the morning or evening before sunset since the place might get a bit scary post sunset to sit by :P</p><h3>Accommodation/Food Cost</h3><p>Though Kamshet is not that commercialized as any other tourist destinations, a couple of glamping resorts and bungalows are being newly constructed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vZjXuKHvRLDB5McisIwFBg.jpeg" /><figcaption>A bungalow type arrangement built on a downhill</figcaption></figure><p>I was staying at Native Place with Fly Nirvana. Native Place is the place of accommodation where the hosts were quite warm and accommodating. Breakfast, Lunch, Dinner was included in the package itself at an affordable price. Flying Nirvana was the name of the organization that offered paragliding courses. Below are few glimpses of the place.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a3zLwZ8t6igOdtgd5BpfAA.jpeg" /><figcaption>Terrace view from Native Place at dawn break</figcaption></figure><p>Here is the Google Maps pinned location for Fly Nirvana if you would need.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.google.com%2Fmaps%2Fembed%2Fv1%2Fplace%3Fcenter%3D18.8265951%252C73.5149251%26key%3DAIzaSyD9HrlRuI1Ani0-MTZ7pvzxwxi4pgW0BCY%26zoom%3D17%26q%3DFly%2BNirvana&amp;display_name=Google+Maps&amp;url=https%3A%2F%2Fwww.google.com%2Fmaps%2Fplace%2FFly%2BNirvana%2F%4018.8265951%2C73.5149251%2C17z%2Fdata%3D%213m1%214b1%214m5%213m4%211s0x3bc2c065a7c56561%3A0xd99d77ab968684d7%218m2%213d18.8265951%214d73.5171138&amp;image=http%3A%2F%2Fmaps-api-ssl.google.com%2Fmaps%2Fapi%2Fstaticmap%3Fcenter%3D18.8265951%2C73.5149251%26zoom%3D15%26size%3D250x250%26sensor%3Dfalse&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=google" width="600" height="450" frameborder="0" scrolling="no"><a href="https://medium.com/media/43c64502a577c81eea8e457db53dc47f/href">https://medium.com/media/43c64502a577c81eea8e457db53dc47f/href</a></iframe><p>As I was only visiting Kamshet for a weekend, I could explore limited places/activities but all of those summed up to be a great experience starting from chatting with fellow travelers to visiting paragliding sight to hiking to chilling besides the lake. It was all worth it. I hope, by now, you are all geared to visit Kamshet and learn paragliding.</p><p><strong><em>Please note this blog is not at all a sponsored blog, nor in anyway is benefited from any sort of commission or any monetary/accommodation benefits from Native Place/Flying Nirvana. This is only my individual experience which I thought to share with you all.</em></strong></p><p>Comment down below if you have any suggestions/reviews about the blog or questions to know more about Kamshet.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df07e307f92f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Diving into K-Means…]]></title>
            <link>https://medium.com/data-science/k-means-clustering-implementation-2018-ac5cd1e51d0a?source=rss-35e9bf238b81------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac5cd1e51d0a</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[unsupervised-learning]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[DITI MODI]]></dc:creator>
            <pubDate>Mon, 10 Sep 2018 02:09:45 GMT</pubDate>
            <atom:updated>2019-01-13T13:40:39.013Z</atom:updated>
            <content:encoded><![CDATA[<p>We have completed our first basic supervised learning model i.e. Linear Regression model in the last post <a href="https://medium.com/@diti.modi/linear-regression-model-899558ba0fc4"><strong><em>here</em></strong></a>. Thus in this post we get started with the most basic unsupervised learning algorithm- <strong>K-means Clustering</strong>. Let’s get started without further ado!</p><h4><strong>Background:</strong></h4><p>K-means clustering as the name itself suggests, is a clustering algorithm, with no pre determined labels defined ,like we had for Linear Regression model, thus called as an Unsupervised Learning algorithm.</p><h4><strong>Logic and working:</strong></h4><blockquote><em>K-means simply partitions the given dataset into various clusters(groups) with different features.</em></blockquote><p><strong>How exactly?</strong></p><p>K refers to the total number of clusters to be defined in the entire dataset.There is a centroid chosen for a given cluster type which is used to calculate the distance of a given data point.The distance essentially represents the similarity of features of a data point to a cluster type.</p><blockquote>We can relate this by taking the example of a school bus here. If we have many students(data points) from a particular area(cluster) traveling to school via the school bus, the school declares a central bus stop(centroid) where all the students from nearby locality can assemble to board the bus(clustering process).</blockquote><p><strong>Constraints:</strong></p><p>Only <strong>numerical data </strong>can be used. Generally k-means works best for 2 dimensional numerical data. Visualization is possible in 2d or 3d data. But in reality there are always multiple features to be considered at a time.</p><p>Thus multi-dimensional data can be used but dimensionality reduction has to be performed to the data before using it for k-means.</p><p><em>Choosing K:</em></p><p>There is no exact way of determining the perfect value of K(total number of clusters for division). We have to try different values of K for the given dataset and compare the results thus obtained.</p><h4>Why to use K-means?</h4><blockquote>Using k-means, the data is clustered after analyzing the data and not primitively defining it under a group based on pre-defined labels. Each centroid is a collection of features that essentially represent the type of cluster it belongs to. Thus a centroid can be used to interpret the type of cluster formed.</blockquote><h4>Real life applications:</h4><ol><li>Segmentation for customer retail dividing into clusters for various data analytics application for eg. for knowing loyal customers, for analyzing the spending behavior of a customer or needs of certain types of customer.</li><li>Fraud detection for cyber crime frauds.</li><li>MP3 files, cellular phones are the general areas that use this technique.<br>There are around 40 different types of sounds in English language.</li></ol><h3>TL;DR</h3><p>Here we perform entire Machine Learning process and not just the algorithm implementation.</p><blockquote>NOTE: We are implementing K-means from scratch here. “scikit-learn” library, the Machine Learning library in Python has in-built K-means method which can be directly used.</blockquote><h4>Steps performed for Machine Learning approach are:</h4><ol><li>Data Pre Processing</li><li>Data Analysis</li><li>Model Implementation</li></ol><h4>Basic steps for k-means algorithm:</h4><p>STEP 1: Choosing random value of k centroids from the values present in dataset(k=2 here)</p><p>STEP 2: Calculate Euclidean distance for each point with each centroid</p><p>STEP 3: Comparing the distances and assigning clusters</p><p>STEP 4: Taking mean and repeat till zero error</p><p>4. Result Visualization</p><h4><strong>IMPLEMENTATION:</strong></h4><blockquote>1. DATA PRE PROCESSING</blockquote><p>Before we use the data that we have, we have to pre process it to get accurate results. Pre Processing includes Data Cleaning, Data Wrangling, Feature Extraction, Data Reduction etc. These vary as per the data and as required.</p><p>Here we import the numpy, pandas, matplotlib libraries for array processing, dataframe use and visualizing the data respectively. The data is read and stored in the dataframe named “data”. The encoding is specified here as “ISO-8859–1” because the csv file that we have is in “utf-8” format to make it a readable format. data.head() displays the first 5 rows of the entire dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yOf0fY-1vVsygKWw-CEuzA.png" /></figure><p>We find the size of the dataframe by data.shape in (total_rows,total_columns) format. Data Cleaning is done to clear the null or missing values. For that we need to know the number of null values in our data using data.isnull().sum(). It provides total null values in each attribute column. We can use “data.drop.na()” for deleting all null valued entries.</p><p>Since we want to use only numeric data for K-means here and there is no null value in that attributes, we can let the data be as it is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*CjARYEHoDkZQiGF2EARkQw.png" /></figure><p>Feature extraction is done further in Data Analysis part.</p><blockquote>2. DATA ANALYSIS</blockquote><p>We obtain the summary of our data with data.describe() storing in “summary” variable. We transpose it to exchange the rows and columns. summary.head() shows the summary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/758/1*_9nsPtk57w4yg4YSXsTMjA.png" /></figure><p>Data Visualization can be done to visualize the distribution in that column whether it is skewed or not plotting Histograms,Boxplots, Scatter plots. Here Histogram is plotted for “SALES” , “QUANTITYORDERED” and “PRICEEACH”. We can use “SALES” and “QUANTITYORDERED” to form clusters based on sales and quantity ordered for a product.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YTsJoLUVj9qKSmfM_QsKXQ.png" /></figure><p>Now we plot a scatterplot. “x” is used to save only the 2 columns that we need from the “data” with all row entries.Next, we plot the scatterplot by setting the figuresize to be 16x9 and style to be ggplot which makes a grid. plt.scatter() plots the 0th column and 1st column of x dataframe with black colored dots and size as 7.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/551/1*ucPf3qLkDbL0W1DXewA9lg.png" /></figure><p>Our initial data looks like this ,unclustered:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M29OMasy2ZDeXRv-FGciew.png" /></figure><p>There are outliers in our data. Outliers are nothing but the values which are far off from other data points and which do not fit in our data and could hinder our accuracy.</p><p>As we can see there are many outliers in our data,we delete the outliers by calculating z-score. zscore() is an in-built function in scipy library of python.</p><p>Z score normalizes our data. Here we get absolute value of z-score that is in the range of -3 to 3, all the points are stored back in x. All other rows are deleted.</p><p>As we notice the shape of x changes from original number of rows to lesser numbers as few rows are deleted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/622/1*Z1uzKo6z7dUj8_eq_moD8g.png" /></figure><blockquote>3. K-MEANS IMPLEMENTATION</blockquote><p>So now we finally start with out actual algorithm implementation after obtaining our final data. We just import copy library also here alongwith the other basic libraries. deepcopy is used to copy one array into another. We use it to store the history of centroids that can be used for calculating error ahead.</p><p>We define a function “euclidean” to calculate the distance between 2 points ‘a’ and ‘b’. np.linalg.norm() is the inbuilt function in numpy library which caculates the Euclidean distance for a and b here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/1*qC1PnYx-V0k0iQbWb4gbtQ.png" /></figure><p>Next we define the main() function. Here we apply all the k-mean algorithm steps.</p><p><strong>STEP 1 for K-Means: Choose random centroids</strong></p><p>So, step 1 being the choice of random centroids from the dataset itself. Numpy has the inbuilt function np.random.choice() with parameters as x.shape[0] which gives the total number of rows, 2 columns i.e the number of clusters and replace is set to false i.e x is not replaced with these values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*S6mrIj60kB10Fqq5zRP4Lg.png" /></figure><p>We can print out the random values of Centroids after which we define all the variables to be used ahead. we define “total” variable as the shape values of x. “ditance_1” and “distance_2” are defined as distances from each point in data to centroid 1 and centroid 2 respectively which we initialize with zeros of size total[0]. Thus these are 1D arrays of the total number of rows in x.</p><p>“belongs_to” is used to store the values of clusters, the data point belongs to. It corresponds to each data point in x by the index. It is also initialized with zeros and the size of total[0] so as to store cluster number(here-0,1 numbered clusters) for each data point row wise. “c_old” stores the old values of centroids. Here it is initialized with zeros and is of shape of “centroid”.</p><p>“error” is used to store the Euclidean distance of the current centroid and old centroid that is if both the centroid values are same, error becomes zero. “mean” is used to store the new centroids. Thus it is the size of the “centroid” and is initialized to zero. “np.zeros()” is the in-built function in Numpy which initializes the given array to zero by passing the size as parameter. And finally “iterator” counts the total number of iterations performed. It is zero currently.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/437/1*nhjB6y9hOJqIWvKqwAE-Xw.png" /></figure><p><strong>STEP 2 for K-Means: Calculate Euclidean distance</strong></p><p>Next, we perform the step 2 and calculate Euclidean distance for each data point with each centroid. The looping condition being error not equal to zero, if error becomes zero, then the centroids calculated are same as previous one so the loop would stop there. We print out the iteration numbers and start the for loop for calculating the Euclidean distances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*zrG1X273WiCXI9I2hPj8lA.png" /></figure><p><strong>STEP 3 for K-Means: Comparing the distances</strong></p><p>Step 3 is thus comparing these distances with each other and assigning clusters accordingly. Thus the ‘if’ condition assigns 0 for first cluster and 1 for second cluster. “belongs_to” value changes depending on the distance of each data point from first and second centroid.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*QODAE4WXPvHd4DfGoGoO7w.png" /></figure><p><strong>STEP 4 for K-Means: Taking Mean and repeat</strong></p><p>And the final step now calculates the new centroids by taking mean over the cluster 1 data points and cluster 2 data points separately. We copy the values of old centroid in c_old using “deepcopy”. The for loop goes till the range of belongs_to to assign means of each column for first cluster. “np.mean()” is the in-built Numpy function for calculating mean. Now we know why we use Numpy so extensively in Python! :P</p><p>I have used separate “mean” variable here just for clarity. We can replace “centroid” variable and re use it here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/419/1*ZeW3IA8P5KY77YHBn0MmCg.png" /></figure><p>We follow same loop for mean of cluster 2 where belongs_to() value is 1, whereas for cluster 1 it was 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/465/1*QvJajWcqlPr_ehVpsCuysg.png" /></figure><p>We then assign the mean values to centroid and calculate error of the old centroid and new centroid. Iterator is increased by 1 and the conditional check for error is done here itself. If error is 0 then the message of same centroid values is conveyed and then the looping stops.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/349/1*LLbQRzk76-JPG58yXAg2dw.png" /></figure><blockquote>4. VISUALIZATION OF RESULTS</blockquote><p>We finally perform the last step of our Machine Learning process being the visualization of our clustered points. We use the same plot as used before. The only things changes is the color for each cluster. We use red ‘r’ and green ‘g’ color to represent each cluster. We assign “points” as array of x in range of it’s row where belongs_to[] value is either 0 or 1. We just scatter red points(color[0]) to cluster 1(belongs_to[]=0) and green points(color[1]) to cluster 2(belongs_to[]=1). We assign the star(*) marker for spotting the final centroid with black color.</p><p>Finally we call the main function to execute the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*4oXHkCrCe9AUJzaeWICvXQ.png" /></figure><p>The entire code and dataset are available <a href="https://github.com/ditsme/Machine-Learning/tree/master/100-Days-Of-ML-Code/Day-06-07-k-means">here</a>. I have also added one more implementation of athlete data which could make the Data Cleaning part clear as we delete the repeated and null values there.</p><h4><strong>OUTPUT:</strong></h4><p>Output that I got is as follows:</p><blockquote>NOTE: The output may vary here as we randomly choose our centroids initially. Here only 2 Iterations are performed till error becomes zero.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/1*Emq2P9-j28FSz1IK-6PJfA.png" /></figure><p>Clustering:</p><p>Final iteration centroid and clusters are considered for plotting the scatter plot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*nuzFplyzvIO5V9J9wsaV1g.png" /></figure><p>For any doubts or reviews, please comment down below and let me know if you liked the blog or not! ;)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac5cd1e51d0a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/k-means-clustering-implementation-2018-ac5cd1e51d0a">Diving into K-Means…</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Neural Networks Simplified!]]></title>
            <link>https://medium.com/data-science/neuralnetworks-implementation-d55cc6fc2f62?source=rss-35e9bf238b81------2</link>
            <guid isPermaLink="false">https://medium.com/p/d55cc6fc2f62</guid>
            <category><![CDATA[artificial-neural-network]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[DITI MODI]]></dc:creator>
            <pubDate>Wed, 18 Jul 2018 18:15:01 GMT</pubDate>
            <atom:updated>2019-01-15T15:27:42.695Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GAIpAXrXG9JUySf9" /><figcaption>courtesy- <a href="https://bitsdroid.com/artificial-neural-network-ann/">here</a></figcaption></figure><p>Since how long had you been waiting to implement your first ANN but couldn’t find correct resource with simplified explanation? Well, here it is. Before going through this post, I would suggest you to have a clarity of Neural Networks from my last post <a href="https://medium.com/getting-started-with-neural-networks/getting-started-with-neural-networks-d98a6e470fae">here</a> , so that this next explanation feels quite comprehensible.</p><p>Before diving deeper into neural networks, I would like to discuss the applications of neural networks in general so as to answer the question- <strong><em>Where would I find Neural Networks applicable in real life?</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*9ZDpoNHGg5MLDbl9p8h-vg.jpeg" /><figcaption>courtesy- <a href="https://memegenerator.net/instance/66443036/x-x-everywhere-neural-networks-neural-networks-everywhere">here</a></figcaption></figure><h4><strong>APPLICATIONS IN REAL LIFE:</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/0*a186p1h9X2BwHWls.gif" /><figcaption>courtesy- <a href="https://www.google.com/search?q=machine+learning+stock+prediction+gif&amp;client=ubuntu&amp;hs=Nov&amp;channel=fs&amp;source=lnms&amp;tbm=isch&amp;sa=X&amp;ved=0ahUKEwid3pjW5KvcAhWLsY8KHVfhAHYQ_AUICigB&amp;biw=1301&amp;bih=670#imgrc=rlQu5Ze7HsMIyM:">here</a></figcaption></figure><ol><li>Have you heard somewhere about a machine predicting the stock market prices? Yes, so cool isn’t it? Since neural networks are really good at processing complex data and help analyse it, <strong>stock prediction</strong> is one of the applications where neural networks work seamlessly great. Such predictive type applications like valuation of a property,employee job performance etc. hugely are applied using neural networks.</li><li>The application being <strong>image processing and computer vision</strong>, neural nets offer a very good advantage here over the traditional gray scale conversion methods is the most important of all . Image to text, image restoration, image compression, pattern recognition or the most famous of all- object detection and face recognition , are just few of the major examples.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/0*eBIUbplIXS6rDQsF.gif" /><figcaption>courtesy- <a href="https://github.com/torch/torch.github.io/blob/master/blog/_posts/2015-11-13-gan.md">here</a></figcaption></figure><p>3. <strong>Generative Adversarial Network(</strong>GAN) is another application where neural network can generate images on it’s own. It can greatly be beneficial for the precise image generation and merging of two images. This is the latest application of neural nets and is been researched widely.</p><p>4. Voice recognition, weather prediction, fingerprint recognition, <strong>handwriting recognition etc. are o</strong>ther small-scale applications using neural nets.</p><p>5.<strong> Medical diagnosis </strong>is an application where major improvements can be made using neural nets.</p><p>Can you guess the application of neural networks in your everyday life?</p><p>It is used by your mobile and apps that you use almost everyday. So here it is- the <strong>automatic facial recognition</strong> in your cameras of your mobile phones is the most used example of Convolutional Neural Networks, the automatic recognition of faces by Facebook and Photos in Google Photos apps are all paragons based out of these Convolutional Neural Networks. The list of applications can be endless here.</p><p>Now we know that neural networks are a jackpot! I am sure you now might be more interested in knowing about neural nets.</p><p><strong><em>How to go about this blog…</em></strong></p><p>I would like to set forth the theoretical concepts and logic behind a normal ANN with the practical implementation alongside which actually deepens our understanding of the theory. Hence I have divided the entire blog into parts of practical implementation, theory behind it and the logic used to implement it. It is all a part of one implementation with continuous flow but just the parts are differentiated for better understanding and correlation.</p><p><strong><em>Do you know how did the idea of neural nets actually emerge?</em></strong></p><p>It was inspired by our own human biological neural network! The working totally resembles the effective working of layers and layers of neurons to form a biological neural network. For artificial neural network, the flow and operational qualities are all simulated from the biological one since the basic principle that we require to model is the simulation of functionality of a human brain to learn from examples as I have mentioned in my previous <a href="https://medium.com/getting-started-with-neural-networks/getting-started-with-neural-networks-d98a6e470fae"><strong>blog</strong></a><strong>.</strong></p><p><strong><em>WE THUS BEGIN…</em></strong></p><p>We take the implementation of ANN model in Python using the most famous MNIST dataset of classifying handwritten digits from 0 to 9 in Tensorflow library.</p><p><strong><em>PRE REQUISITES:</em></strong></p><p>Tensorflow is the most efficient and useful library used for neural networks implementation in Python. I prefer to use Jupyter Notebook for implementation. However you can use any IDE of your choice- Spyder, Sublime Text 3 etc.</p><p>Before we start the program, we need to install Tensorflow. The installation is quite easy and can be installed from <a href="https://www.tensorflow.org/install/">here</a>. CPU version can be downloaded for a normal Desktop.</p><p>The entire code and output screenshot is available on my github <a href="https://github.com/ditsme/Machine-Learning/tree/master/Neural%20Networks">here</a>. Feel free to fork!</p><h3>THEORY AND TERMS:</h3><p>To understand that, we need to introduce 4 basic terms here:</p><ol><li><strong>PERCEPTRON</strong>: It is nothing but a simple layer neural network inspired by a biological neuron as shown below.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*JVm4m33dGS1-dxsAh_H4QA.png" /><figcaption>courtesy-<a href="https://towardsdatascience.com/from-fiction-to-reality-a-beginners-guide-to-artificial-neural-networks-d0411777571b">https://towardsdatascience.com/from-fiction-to-reality-a-beginners-guide-to-artificial-neural-networks-d0411777571b</a></figcaption></figure><p><strong>2.</strong> <strong>INPUT:</strong> Input is nothing but the data that we want to feed to the network. It is usually defined as x1,x2,…xn. Each input is a sample or independent variable used to determine the output. Input can be of any form-text, images or video as a whole. Here we have images of handwritten digits as input.</p><p><strong>3. OUTPUT:</strong> The output is generated for the respective neuron that fires the activation.</p><p><strong>4.</strong> <strong>ACTIVATION FUNCTION:</strong> The input signal after going through an activation function can further be processed by other layers or can directly generate output. An activation function can be essentially termed as a processing unit for processing input to get the output. It is used to map the output as to between 0 to 1 or -1 to 1 depending on the type of activation function. There are various types of activation functions including sigmoidal, step, exponential functions etc.</p><p>An input symbolizes the dendrites of a biological neuron which is the receiver for the neuron to transmit information from one neuron to another. Similarly other similarities are as follows according to the functions defined.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*1hphTyhMnriG8gIPiYLmgg.jpeg" /><figcaption>courtesy- <a href="https://www.slideshare.net/MohdArafatShaikh/artificial-neural-network-80825958">here</a></figcaption></figure><h3><strong>IMPLEMENTATION:</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*Ua1AfdWLagtKbdR_U83RvQ.png" /><figcaption>Import Dependencies</figcaption></figure><p>Next we import our dependencies that is Tensorflow and the MNIST data that we are going to use for training ahead.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/1*d72WcZHsPcFbmzOPiYY42g.png" /><figcaption>Read data</figcaption></figure><p>Next, we read the dataset and save to the variable “mnist” specifying the path as first parameter and one_hot as TRUE. Now what is one_hot you may think.</p><h3><strong>LOGIC USED IN APPLICATION:</strong></h3><p>FOR INPUT:</p><p>A neuron essentially in an ANN model implementation is nothing but the placeholder to hold a numerical value like here we have gray scale values ranging from 0(black) to 1(white) stored in the neuron.</p><p>These neurons together form the input as a black and white image input.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/468/0*tWKgUsSb3Qcn-5HG.gif" /><figcaption>courtesy-<a href="https://gfycat.com/gifs/tag/neural+networks">https://gfycat.com/gifs/tag/neural+networks</a></figcaption></figure><p>FOR OUTPUT:</p><p>One_hot is set true for activating an output node whenever we get the corresponding output.</p><p>For example here we have the total number of possible outcomes as 10 since we have 0–9 digits to be identified. So for any input image provided as input, any one neuron out of 10 corresponding to each output identified by our model would be lit up. For instance,</p><p>for digit 0=[1 0 0 0 0 0 0 0 0 0],</p><p>for digit 1=[0 1 0 0 0 0 0 0 0 0] so on…</p><p>where the array contains set of binary values for respective output neurons.</p><p>So we conclude that,</p><p><strong><em>The first layer for a neural network is always an input layer while the last layer always being the output layer.</em></strong></p><h3><strong>THEORY AND TERMS:</strong></h3><p><strong><em>Then what about the middle layers where actually the processing takes place?</em></strong></p><p>These layers are nothing but the <strong>hidden layers.</strong> This is where all the magic begins!</p><p>An Artificial Neural Network is inspired by a biological neural network consisting of multiple neurons working simultaneously and in co-ordination to perform various tasks for our body. Thus we have multiple layers of neurons in ANN as well for better performance. The deeper we go, the better but for a small dataset it would be a problem.</p><p><strong><em>So how to decide the number of hidden layers to be used?</em></strong></p><p>There can be as many hidden layers as we like but it also depends on the size of the dataset. Though more number of hidden layers are better as the net becomes deeper, for a smaller dataset it would reduce the accuracy instead of increasing it.</p><h3>IMPLEMENTATION:</h3><p>We choose 3 hidden layers for our model and 500,600,300 respectivel are the total number of nodes in that layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*D11BQZMoTdNicxNmGZ9XnQ.png" /><figcaption>Defining total number of nodes in hidden layers</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*ff_rNGBF3sN9pRWaJPZp2g.png" /><figcaption>Defining variables</figcaption></figure><p>Later we define the total number of output classes, batch size, and the tensors(placeholders) to define our input x and output y in terms of variables.</p><h3><strong>LOGIC USED IN APPLICATION:</strong></h3><p><strong><em>PROCESSING IN HIDDEN LAYERS:</em></strong></p><p>Analogous to biological neuron, in ANN the 784 input neurons with specific gray scale value fire(trigger) specific neurons in next hidden layer only which in turn would trigger some pattern in the one after it and finally a specific pattern in output layer. The brightest neuron is chosen by the network as the final output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*DxMWP3GCeiVNxLw9.gif" /><figcaption>courtesy- <a href="https://gfycat.com/gifs/tag/neural+networks">here</a></figcaption></figure><p>What essentially hidden layers do is they separate the digits into separate parts of loops and lines that form that digit. For example below we see 9 has one loop and a line which combines to form the digit. Specific pattern is thus generated at various layers which is different than the pattern generated for 8 which has 2 loops. Thus we need deeper nets to identify the key patterns for larger datasets with varying features and huge images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/0*EvrKm5jQlxvCTcNp.gif" /><figcaption>courtesy- <a href="https://gfycat.com/gifs/tag/neural+networks">here</a></figcaption></figure><p>In short:</p><h4><strong>Pixels — &gt;Edges — &gt;Patterns — &gt;Digits</strong></h4><h3>THEORY AND TERMS:</h3><p><strong><em>Now,How do we make our model fire only specific neurons for specific patterns?</em></strong></p><p>The answer is by using “weights” and “biases”.</p><p><strong>5. WEIGHTS:</strong> Weights are nothing but the axon in a biological neuron. The axon as we see in the image is used to transmit information from one neuron to other. It is nothing but the connection link. This is a very important information as weights decide the excitation and inhibition of a signal to be communicated. Simply it follows the reward-punishment way that is the excitation causes one neuron to activate another neuron and opposite for inhibitory ones. This is the same as biological transfer of messages in the neural network.</p><p><strong>6. BIAS:</strong> It can be visualized as a switch which is always on that is it’s value is always one. The basic idea is that human neurons take a bunch of inputs and “add” them together. If the sum of the inputs is greater than some threshold, then the neuron will “fire” (produce an output that goes to other neurons). This threshold is essentially the same thing as a bias. So, in this way, the bias in artificial neural nets helps to replicate the behavior of real, human neurons.</p><h3>IMPLEMENTATION:</h3><p>So we initially define random weights and biases for each hidden layer and the output layer for total number of neurons defined for that layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*498f_qd3JcibylSDUrJ2Hw.png" /><figcaption>Define weights and biases</figcaption></figure><p>After defining all our variables, we start defining our model and the functions of the layers using weights and biases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*Vj2SpVHokhNRy2q2FPmLqQ.png" /><figcaption>Activation function</figcaption></figure><h3>LOGIC USED IN APPLICATION:</h3><p>We assign certain positive weights to excite/fire a neuron and a negative weight to inhibit/stop from firing a neuron for a given pattern in that layer.</p><p>To mathematically compute the values, we multiply the input with the respective weight for the entire layer and finally sum it up with the bias:</p><p><strong><em>weighted sum=(weight*input)+bias</em></strong></p><p>We then feed this weighted sum to the activation function called “RELU” called Rectified Linear Unit.The activation is simply thresholded at zero.</p><p>It computes the function <em>f</em>(<em>x</em>)=max(0,<em>x</em>). It is the most used activation function in deep learning as it greatly accelerates the training part.</p><h3>THEORY AND TERMS:</h3><p><strong><em>So where exactly is the learning part here of “deep learning”?</em></strong></p><p>If you remember, we had initialized random weights and biases initially because we were not aware of them at the start.</p><p>We finally aim to minimize the error caused in predicting an output. Learning specifically thus means to determine these weights and biases in such as way that the error is minimized. This is done by the optimizers.</p><p>Here we optimize our model using <strong>Adam Optimizer</strong> to minimize the cost. Adam Optimizer using Stochastic Gradient method to optimize our model. There are various other optimizers available using different methods.</p><h3>IMPLEMENTATION:</h3><p>The last part after defining the model is the definition if how the model should be trained.</p><p>We predict the output for each input x and then determine the cost/error that our model has committed by comparing the predicted value with the labeled correct value y in our dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/1*tRS-AkI96JNNFTUEzAsRQA.png" /><figcaption>Prediction of output</figcaption></figure><p>We define the session for looping till total epoch number is reached. We train the input in batches together and run the optimizer.</p><p>Finally we use “correct” variable to check the one_hot value of the predicted output against the expected value.</p><p>Finally calculating and printing the accuracy of our model to see the effectiveness.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*K9c-uVk0ghDByudRozpUeg.png" /></figure><p>The last line is to call the training function with input x and output y to initiate the training.</p><h3>LOGIC USED IN APPLICATION:</h3><p><strong><em>How long does our model train itself?</em></strong></p><p>The model keeps training till the error reaches a threshold predefined or by defining the total epochs(loops) needed.</p><p>We can tweak many parameters like number of hidden layers, number of nodes or optimizer used etc. to check and lower the error rate.</p><p>Essentially the work of a neural network can be summarized as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/165/1*ToIIDFNFnl2SzxlaiazQyw.png" /><figcaption>Summarizing the process</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/1*8DKLaI9Gob4Cm-oYXe2j9Q.png" /><figcaption>courtesy- <a href="https://towardsdatascience.com/random-initialization-for-neural-networks-a-thing-of-the-past-bfcdd806bf9e">here</a></figcaption></figure><p>Phew! That was a bit overwhelming for a beginner, isn’t it?</p><p>I know how that feels at the beginning :/</p><p>But trust me, it would all feel basic once we are thorough with it after practice. For any doubts, comment down below. Let me know your views about this post and any topic you would want me to post about next.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d55cc6fc2f62" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/neuralnetworks-implementation-d55cc6fc2f62">Neural Networks Simplified!</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to build a simple Linear Regression Model?]]></title>
            <link>https://medium.com/data-science/linear-regression-model-899558ba0fc4?source=rss-35e9bf238b81------2</link>
            <guid isPermaLink="false">https://medium.com/p/899558ba0fc4</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[numpy]]></category>
            <category><![CDATA[regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <dc:creator><![CDATA[DITI MODI]]></dc:creator>
            <pubDate>Thu, 14 Jun 2018 18:23:00 GMT</pubDate>
            <atom:updated>2022-09-09T11:13:05.391Z</atom:updated>
            <content:encoded><![CDATA[<p>This post is about implementing simple linear regression model for ML beginners in step by step way with detailed explanation.</p><p>If you are new to machine learning, check <a href="https://medium.com/@diti.modi/machine-learning-simplified-3f964c1a64a6"><strong><em>this</em></strong></a> post for getting a clear idea about Machine Learning and it’s basics.</p><p><strong>What is the logic behind simple linear regression model?</strong></p><p>As the name suggests, linear regression follows the linear mathematical model for determining the value of one dependent variable from value of one given independent variable. Remember the linear equation from school?</p><h3>y=mx+c</h3><p>where y is the dependent variable, m is slope, x is the independent variable and c is the intercept for a given line.</p><p>We also have multiple regression model where multiple independent variables are used to calculate one dependent variable.</p><p>I have used Jupyter Notebook for implementation. Any Python IDE can be used of your choice. So let’s get rolling..</p><p>Step 1: Importing libraries</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XhPNXcPI8rtlLfRGrK0mkQ.png" /><figcaption>Step 1</figcaption></figure><p>There are already developed libraries in Python for implementation of Machine Learning models.</p><p>First library called matplotlib is used to plot the graph in last step. ”plt” is used as variable name for using this library in code ahead.</p><p>sklearn is official machine learning library in python for various model implementation.</p><p>numpy is used to convert data into arrays for actual use by sklearn library.</p><p>pandas is used to access .csv file of our dataset.</p><p>Step 2: Loading dataset</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*MCwFNUzJF9mokgz2no8VOw.png" /></figure><p>Our dataset is in a .csv file type. Pandas variable pd is used to access the dataset with read_csv() function.</p><p>Step 3: Split to independent and dependent variables</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*mGUxLFpCR8wBiMPmXLbHIw.png" /></figure><p>We define x as the independent variable in dataset by iloc(index location) value. [] is used to define array elements. “:” inside [] indicates consider all rows in dataset and separating by using “,” we specify the number of column which we want to use as independent or dependent variable values starting the count from zero in dataset.</p><p>Step 4: Splitting data into training and testing data</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*4OsNhmwslwgeFq10ukFtMw.png" /></figure><p>Now, entire dataset is divided into training and testing set so that prediction does not overfit or underfit and correct values are obtained. train_test_split() is inbuilt function from scikit learn for splitting x and y variables data. “test_size” parameter is used to divide (1/3)rd of entire dataset(30%) into test data and remaining as training data.Setting random_state as null would not allow random values to be taken from dataset.</p><p>Step 5: Choosing the Model</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*IrE4WIdiR4viRSF5HuDPjg.png" /></figure><p>We reshape our independent variable as sklearn expects a 2D array as input.</p><p>Linear Regression is our model here with variable name of our model as “lin_reg”. We can try the same dataset with many other models as well. This part varies for any model otherwise all other steps are similar as described here.</p><p>Step 6: Fit our model</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*npLvvzhoNbC9Wql2nmACTA.png" /></figure><p>We now fit our model to the linear regression model by training the model with our independent variable and dependent variables.</p><p>Step 7: Predict the output</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*deIH021RLjOdrd50MttZcA.png" /></figure><p>Finally our model predicts the dependent variable “lin_reg_pred” using the test values of independent variable.</p><p>We can see the coefficient,intercept values for our outlier and also the mean squared error and variance for the predicted values(lin_reg_pred) and actual test value of dependent variable(y_test). Inbuilt methods does the math with the predefined formulae for each value.</p><p>Step 8: Plot the graph</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1013/1*2mnCfmC9XiK_4OcuNNnoSw.png" /></figure><p>We ultimately want to visualize the actual data values and predicted data values in a graphical format. “plt”, matplotlib variable, is used to plot points using “scatter()” and outlier using “plot()” functions.</p><p>Output might vary depending on various system features. Output I got is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/1*Ug_PyF6kggspeTi4Jjremg.png" /><figcaption>Output</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/513/1*4W-T0cXlcmJP9qpj9xlOLA.png" /><figcaption>Output Graph</figcaption></figure><p>This is it! Our first linear regression model is implemented! Below is the link to the entire code for the entire model that we implemented here.</p><p><a href="https://github.com/ditsme/MachineLearning">ditsme/MachineLearning</a></p><p>Follow me on LinkedIn: <a href="http://www.linkedin.com/comm/mynetwork/discovery-see-all?usecase=PEOPLE_FOLLOWS&amp;followMember=ditimodi">www.linkedin.com/comm/mynetwork/discovery-see-all?usecase=PEOPLE_FOLLOWS&amp;followMember=ditimodi</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=899558ba0fc4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/linear-regression-model-899558ba0fc4">How to build a simple Linear Regression Model?</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning Simplified!]]></title>
            <link>https://becominghuman.ai/machine-learning-simplified-3f964c1a64a6?source=rss-35e9bf238b81------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f964c1a64a6</guid>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[unsupervised-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[DITI MODI]]></dc:creator>
            <pubDate>Wed, 13 Jun 2018 15:50:27 GMT</pubDate>
            <atom:updated>2019-01-14T14:46:24.144Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/990/1*obbVLLbTatr2MFOZP8tH_A.jpeg" /></figure><p>Did your mind just pop images from the famous Terminator movie where machines are destroying the entire world with all the news you hear around the words “Artificial Intelligence” and “Machine Learning”? Who knows..there also might be a hulk generated anywhere now and a Thor hanging around to save our lives…But wait right there. Hold down your horses there because this isn’t a fictional movie but sadly reality! Artificial Intelligence and Machine Learning as opposed to what you think currently are NOT anywhere related to such scenes in our near future.</p><p>Machine Learning and AI are here to stay only for the best of humankind. To know how exactly that might be true, let’s clarify your ideas about Machine Learning here…</p><p>This blog is the result of many of my friends from the non-Computer Science background wondering and asking me what is Machine Learning. As the buzz around Machine Learning keeps escalating day by day, everyone wants to know what Machine Learning exactly is.</p><h3>Trending AI Articles:</h3><blockquote><a href="https://becominghuman.ai/lets-build-a-simple-neural-net-f4474256647f">1. Let’s build a simple Neural Net!</a></blockquote><blockquote><a href="https://becominghuman.ai/decision-trees-in-machine-learning-f362b296594a">2. Decision Trees in Machine Learning</a></blockquote><blockquote><a href="https://becominghuman.ai/an-intuitive-introduction-to-machine-learning-57b2b1bd3d6">3. An intuitive introduction to Machine Learning</a></blockquote><blockquote><a href="https://becominghuman.ai/the-balance-of-passive-vs-active-a-i-5e2ab6a0561e">4. The Balance of Passive vs. Active A.I.</a></blockquote><p>I hope by the end of this blog this now-alien-looking term would turn into an interesting field and intrigue you. So let’s get started!</p><h3><strong>Whats and Hows of Machine Learning…</strong></h3><p>Examples of Machine Learning in real life today are- email auto filtering of spam and non-spam ones, online recommendations you get for online shopping, prediction of stock prices and much more.</p><p>Remember how we always ask for examples to understand a concept clearly? That pretty much is what Machine Learning is all about. Yes. It simply means understanding and learning by the means of different examples. For example- to make a machine know what is the difference between a cat and a dog, we provide it with various images of cats and dogs stating the correct answers and then testing the same after it was trained.</p><p>That is the gist of what essentially happens in a Machine Learning algorithm.</p><p>SO HOW IS MACHINE LEARNING DIFFERENT FROM OTHER TRADITIONAL ALGORITHMS?</p><p>To answer, I would say just in the same way we distinguish theoretical direct learning and the indirect way of learning through examples. Machines figure out on their own, the rules and patterns in data.</p><p>ML(Machine Learning) is a sub field of Artificial Intelligence and Data Science.</p><p>So after having a clear idea of what ML is all about conceptually, let’s get started with basics and types followed by implementation of our first ML program.</p><h3><strong>TYPES OF MACHINE LEARNING:</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*xqexSB17cRFbgfJT1wkMlQ.jpeg" /><figcaption>Types of learning(Credits-<a href="https://www.slideshare.net/meydman/machine-learning-ai">https://www.slideshare.net/meydman/machine-learning-ai</a>)</figcaption></figure><p>The image above gives a clear idea of types of machine learning categorised as:</p><p>1.Supervised Learning- In simple terms this type is similar to the way a teacher teaches his students. For eg. if we want to learn what an apple is, teacher simply supervised us with a <strong>label</strong> called “apple”. In similar fashion, we provide a label for certain inputs to get the desired output.</p><p>2.Unsupervised Learning- Opposite of Supervised Learning, the algorithms themselves cluster output into various groups based on some input features. No labels are provided exclusively.</p><p>3.Reinforcement Learning-In this type we do not provide labels as seen in unsupervised but the difference is we provide positive or negative feedback to the output similar to supervised type. The feedback can be termed as hints for algorithm to generally understand what is the correct output required for a particular type of input.</p><p>For more details on types and their respective models, please visit official documentation here:</p><p><a href="http://scikit-learn.org/stable/tutorial/basic/tutorial.html">An introduction to machine learning with scikit-learn</a></p><h3><strong>IMPLEMENTATION PROCESS:</strong></h3><p>Any machine learning model implementation is divided into following steps:</p><ol><li>Importing libraries</li><li>Loading dataset</li><li>Splitting data into independent and dependent variables</li><li>Splitting data into training and testing datasets</li><li>Selecting the model type to use</li><li>Fitting our model</li><li>Predicting the values</li><li>Plotting the graph for our model</li></ol><p>5th step is generally variable for choosing the model to use. Choosing a model depends on various factors which we can discuss in the next implementation post. We are implementing the relation of temperature and relative humidity in the month of June 2018 for Mumbai.</p><p>We will use Python language for implementation so knowledge of Python language, basic linear algebra is a must for implementation now.</p><p>FIRST ML MODEL:</p><p><a href="https://github.com/ditsme/MachineLearning">ditsme/MachineLearning</a></p><p>Above is the link to the entire code for the first model that we are going to implement here.</p><p>We implement Simple Regression model here. Simple Regression Model algorithm comes under supervised learning type and is one of the easiest of all.</p><p>Check out my next blog <a href="https://medium.com/@diti.modi/linear-regression-model-899558ba0fc4"><strong>here</strong></a> to implement this first model with detailed description of each step explained.</p><p>Please let me know your views about this blog in comments and do share if you find it worth-reading. Thankyou :)</p><h3>Don’t forget to give us your 👏 !</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*fwLoMDekrBvhIr5n" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f5f8b%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F8f5f8b&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href">https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href</a></iframe><figure><a href="https://becominghuman.ai/artificial-intelligence-communities-c305f28e674c"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*2f7OqE2AJK1KSrhkmD9ZMw.png" /></a></figure><figure><a href="https://upscri.be/8f5f8b"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*v-PpfkSWHbvlWWamSVHHWg.png" /></a></figure><figure><a href="https://becominghuman.ai/write-for-us-48270209de63"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*Wt2auqISiEAOZxJ-I7brDQ.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f964c1a64a6" width="1" height="1" alt=""><hr><p><a href="https://becominghuman.ai/machine-learning-simplified-3f964c1a64a6">Machine Learning Simplified!</a> was originally published in <a href="https://becominghuman.ai/">Becoming Human: Artificial Intelligence Magazine</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Starting off with Neural Networks]]></title>
            <link>https://medium.com/@diti.modi/getting-started-with-neural-networks-d98a6e470fae?source=rss-35e9bf238b81------2</link>
            <guid isPermaLink="false">https://medium.com/p/d98a6e470fae</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[beginner]]></category>
            <dc:creator><![CDATA[DITI MODI]]></dc:creator>
            <pubDate>Tue, 27 Mar 2018 15:11:46 GMT</pubDate>
            <atom:updated>2018-06-13T15:55:15.714Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*K40TcJ146AygupDi.jpg" /></figure><p>Hi everyone! Let me guess…you are here to know more about Neural Networks for your project at school or maybe because it is an upcoming field all across the world or maybe to pursue neural nets in depth out of interest. I am glad to see your interest in Deep Learning and Neural Networks.</p><p>Being a beginner is always tough, but gradually it all feels all known at the end. So I don’t term myself as a pro for Neural Nets but I would surely like to share my knowledge and beginner experience for getting started with Deep Learning and Neural Nets.</p><p>In this post I wouldn’t want to overwhelm you by noting down too much information. This post is for absolute beginners to get an idea about neural nets in general. My next post would talk more in depth into technicality of Neural Nets. This is just to clear the concept behind Neural Net and their working.</p><h3><strong>So what is a neural network?</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*3fxZCbbTmedMrZIc.jpg" /></figure><h4><em>Concept:</em></h4><p>In simple words, a neural network is nothing but in a way simulation of neurons in your human brain. It is an artificially modelled network of neurons/nodes just as biological network of neurons.</p><h4><em>Implementation:</em></h4><p>Neural net is implemented by an algorithm or a combination of programs and data designed to recognise patterns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*fqKioMC7rLOsHfAm.png" /></figure><h4><em>Working:</em></h4><p>The basic functioning of Artificial Neural Net(ANN) is inspired by working of biological neuron. Human brain learns as a result of getting trained by sample data initially. For example, we learn to call a fruit-apple after initally getting “trained” to see the apple as object in real life or it’s image. I know this sounds bit deep, after all deep learning it is! ;)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*mUuG-QLK8Y0vnsHj.png" /></figure><p><strong><em>Why use them? How is it different from any other algorithm or set of programs?</em></strong></p><p>Now you would ask this question as to why neural net is so important and what is so special about it. Well, the answer is- neural net principles output a much better accuracy than any normal algorithm for the same problem statement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*I-3absdCH-bbNYtr.jpg" /><figcaption>Simple Pattern recognition training</figcaption></figure><p><strong><em>How do they simplify our task?</em></strong></p><p>Neural nets simplify our tasks by recognising patterns from the data provided to it just as human brain would. It is initially trained with a set of data and then tested against another set of data.</p><p>So this is pretty much the simplest introduction of neural networks. Follow my post <a href="https://medium.com/@diti.modi/machine-learning-simplified-3f964c1a64a6"><strong><em>here</em></strong></a><strong><em> </em></strong>to know more about machine learning and model implementation.</p><p>P.S.: As this is my first blog about neural networks, please consider the slip of information if any. I am also open to any suggestions and views about this topic or blog.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d98a6e470fae" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>