<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kunal Gohrani on Medium]]></title>
        <description><![CDATA[Stories by Kunal Gohrani on Medium]]></description>
        <link>https://medium.com/@kunal_gohrani?source=rss-5931e2e03fe2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*bNfYHCdXQiF8elNtk_uV7g.jpeg</url>
            <title>Stories by Kunal Gohrani on Medium</title>
            <link>https://medium.com/@kunal_gohrani?source=rss-5931e2e03fe2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 18:30:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kunal_gohrani/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Future of Software industry: A look into the Current Trends]]></title>
            <link>https://medium.com/@kunal_gohrani/the-future-of-software-industry-a-look-into-the-current-trends-d9289b3f2e77?source=rss-5931e2e03fe2------2</link>
            <guid isPermaLink="false">https://medium.com/p/d9289b3f2e77</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[trends]]></category>
            <category><![CDATA[software-industry]]></category>
            <dc:creator><![CDATA[Kunal Gohrani]]></dc:creator>
            <pubDate>Sat, 11 Jan 2020 22:15:34 GMT</pubDate>
            <atom:updated>2020-01-11T22:15:34.687Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*Mw0ebFQGvegr6oCKCgSK3w.png" /></figure><p>Change is the only constant on this planet. There is no industry from education to healthcare that can resist changes. Every sector welcomes innovations that bring changes in the ongoing IT trends.</p><p>Similar is the case with software design &amp; development trends as well. In fact, the software industry is one of those sectors of contemporary times that witnesses a constant change in its practices because of the growing software technologies.</p><p>For software developers, trends tend to indicate where the industry is headed. New technologies, development methods, and customer preferences can be key indicators regarding where you should focus your development skills. Staying on top of new technology trends is a great way to future-proof your software development skills, ensuring your abilities don’t get outdated.</p><p>In this blog post, I am gonna share the results of my analysis on the<strong> Stack Overflow Developer Survey 2019</strong> data set which I have taken from <a href="https://www.kaggle.com/mchirico/stack-overflow-developer-survey-results-2019">kaggle.com</a> to show some current trends going on in the software industry.</p><blockquote>“ Stack Overflow’s annual Developer Survey is the largest and most comprehensive survey of people who code<strong> around the world.</strong> Each year, we field a survey covering everything from developers’ favorite technologies to their job preferences. This year marks the ninth year we’ve published our annual Developer Survey results, and nearly 90,000 developers took the 20-minute survey earlier this year.”</blockquote><h3>The number of surveys recorded from each country?</h3><p>So first of all, let’s see the top 10 countries having the most number of people who participated in the survey.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*dMvXZuvHxZopRa_tbJ8SQw.png" /></figure><p>Clearly, from the graph, we can see that a vast majority of people from the United States participated in the survey, 16050 of them, followed by 6588 surveys from India, and so on.</p><h3>Profession of the surveyees</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*Ej1Aa9DRU5K4kJBpZOaTJg.png" /></figure><p>We can understand that the majority of the surveyees taking part in the survey were professional developers.<br>Since we wanna focus our attention on Software Industry, for the rest of my analysis, I have only used records of those surveyees who are developers by profession.</p><h3><strong>How many developers have mentioned coding as their hobby?</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*S2nrg9q3Q192SD8ocWHyKg.png" /></figure><p>Wow! Seeing the above graph, we can see that a large number of developers actually love coding and it is their hobby. As it is said, make a career in a field you love, then work becomes a play. :)</p><h3>How many developers contribute to Open Source Software?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*6GP3Q2cKaMJ115KGNz-40A.png" /></figure><p>We can see that 65% of the total number of developers have contributed to Open Source Software out of which, 12% of the developers often contribute to open source software (Once a month or more often).</p><h3>Employment status of the surveyees</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*Ura64qPbZWBSw-5C3BX5Vg.png" /></figure><p>97% of the developers who participated in the survey are employed either full time, part-time, or self-employed.</p><h3>What is the highest level of education received by the developers who participated in the survey?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*iPTB4NnN3Qj2tXB5H6vNrA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*mwxUYYYz8nB-jbVn-tYJTg.png" /></figure><p>From the above-shown data, it is clear that the majority of the surveyees have at least a bachelor’s degree in the fields shown in the second picture. We can also infer that a discipline related to computers (Computer Science, Computer Engineering, or Software Engineering) is common in the software industry.</p><h3>What are the most common types of developers present in the industry?</h3><p>Now we shall see the most common types of developers (front end developers, back end developers) in the industry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/1*fIdzoksTcD1PAyjY2unoFw.png" /></figure><p>We see that the most common types of developers present in the industry are:-</p><ul><li>Full-Stack, Front-end, or Back-end developers</li><li>Mobile Application Developers</li><li>Database Administrator</li></ul><p>Thus we can infer that there is a common trend of choosing to be a full-stack developer.</p><h3>What is the average salary earned by each type of developer?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*omgAw63CusToBm308qBKlg.png" /></figure><p>From the above graph, we can see that Engineering managers, SREs, DevOps specialists, and data scientists/engineers command the highest salaries.</p><h3>Career Satisfaction of the surveyees</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*-gXW17cr0s-lgmvX4DkA0Q.png" /></figure><p>We see that the majority of the surveyees are satisfied with their careers. For a more detailed analysis, let’s see the satisfaction of developers country wise.<br>A great way to do this would be assigning career satisfaction of surveyees with marks, example:-</p><ul><li>Very Satisfied: 7</li><li>Slightly Satisfied: 4</li><li>Neither satisfied nor dissatisfied: 1</li><li>Slightly Dissatisfied: -2</li><li>Very Dissatisfied: -5</li></ul><p>We will be penalizing Slightly Dissatisfied and Very Dissatisfied records with -2 and -5 as I think this will help us have a clear view of the career satisfaction of the developers in each country. Upon analysis, we find that:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*kUslatDIF-gdYyb6HeXTbA.png" /></figure><p>From the above graph, it looks like surveyees based in the United States are, in general, very satisfied by their careers. However, it is sad to see the surveyees based in India are, in general, not that satisfied with their careers.</p><h3>What are the things asked to be done during an interview?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*7jUsMAAwl2FnWzpKmLYkjQ.png" /></figure><p>Seeing the above output, we get to know that most in most of the job interviews, The developers have been asked to:-</p><ul><li>Interview with people in senior/management/peer roles.</li><li>Write code.</li></ul><h3>While deciding on accepting a job offer, which factors about the job are important for the developers?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*tdOFP1LVXICDdwYXWA3SQA.png" /></figure><p>From the above output, we see that the most important factors favoring one job over another are:-</p><ul><li>The type of Language, frameworks and other technologies that the developer will be working with</li><li>Office Environment</li><li>Flexible schedule</li><li>Opportunities for professional development</li><li>Remote Work options</li></ul><p>Hence, the above factors are, in general, considered to be very important by the surveyees while looking for a job.</p><h3>What is the median salary of the developers country wise? (in USD)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*efE0bx2weaP-mjreoKnmSw.png" /></figure><h3>What are the average weekly working hours of the developers country wise?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/351/1*6eoeE0g47W6zfkLm4Z_0PQ.png" /></figure><p>From the results of median salary country wise and average weekly work hours, we can infer that India, though being ranked fourth in average weekly work hours, has the lowest median salary ($10080). On the other hand, the United States, having average weekly work hours a bit lower than that of India, has a median salary record of $112260.</p><h3>Which is the most preferable place of work for developers?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*u1aWUqwiayuFlGIKkIVFPQ.png" /></figure><p>As we see from the graph, the majority of the software developers prefer working in an office, or home. As working from home saves the time needed to commute.</p><h3>Technological Trends</h3><h4>Which are the most popular programming languages used in the software industry?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*DdKgg04_3c6x-aH73mRt1w.png" /></figure><p>JavaScript is the most commonly used programming language in the software industry. This year, Python just edged out Java in the overall ranking. Python is the fastest-growing major programming language today.</p><h4>Which are the top-paying programming languages in the software industry?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*XXtHxapT4r_sL6uuKNr-Mw.png" /></figure><p>Globally, respondents who use Clojure, F#, Elixir, and Rust earn the highest salaries, with median salaries above $70,000 USD. On the other hand, people experienced with Python, Java, C++ have salaries in the range of $40k-$60K</p><h4>Which are the most popular database solutions used by the software industry?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*ffOqk7ttJvNIRBOQMK0UcQ.png" /></figure><h4>Which are the popular development platforms that the surveyees have done extensive development work on?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/1*Nl9ZEO3asJVPDgQGjOefNw.png" /></figure><p>Clearly, Linux is the most loved Development Platform, followed by Windows, Docker, AWS, and Android.</p><h4>Which are the popular development environments/IDE(s)/Text Editors used by the developers?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/1*lUlmsasAs0S9rQ379k86yA.png" /></figure><h4>Which operating systems are preferred by the developers?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/283/1*TUjfangCBuxzcUgGNuT2Uw.png" /></figure><p>About half of the surveyees said they mainly use Windows, and the remainder were about evenly split between MacOS and Linux.</p><h4>Where is container technology (Docker, and so on) mainly used in the industry?</h4><p>“ Container technology is a method of packaging an application so it can be run with isolated <a href="https://www.section.io/engineering-education/history-of-container-technology/">dependencies</a>”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*iRGNjMsNfTNaoFq15xvP8w.png" /></figure><p>Since container technology helps in sharing applications between machines without having to install the dependencies in the other machine as well, it has become very popular in the industry.</p><h4>How is blockchain used in the Industry?</h4><p>In the past few years, you must be hearing a lot about cryptocurrency. So let’s find out first what is blockchain. As Wikipedia says “A blockchain originally blockchain, is a growing list of records, called <em>blocks</em>, that are linked using cryptography. Each block contains a cryptographic hash of the previous block, a timestamp, and transaction data.” Now let us find out how many organization is trying to implement blockchain technology.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*NbYJcX6RA3jHkeamzMnAUg.png" /></figure><p>Data shows that maximum organization is not interested in implementing blockchain technology. Although the companies not implementing the blockchain are maximum we also see that the software industry is gradually trying to implement this technology. Thus, the blockchain will gradually conquer a greater place in the software industry in the coming years.</p><h3>Conclusion</h3><p>In this analysis, I found out various trends going on in the software industry. Several points that were covered in my analysis were:-</p><ul><li>Developer having a degree in the field of computers (Computer Science, Computer Engineering, Software Engineering) is always preferred in the software industry.</li><li>Most common types of developers present in the industry and the average salary of each type.</li><li>Overall career satisfaction of the people in the software industry country wise.</li><li>Things asked to do in a Job Interview, and what are the factors due to which, a developer may prefer one job over another.</li><li>Average Salary, and weekly work hours of developers country wise.</li><li>Preferable place of work</li><li>Several technological trends (Popular programming languages, top-paying technologies, and so on).</li></ul><blockquote>For the whole analysis, please check out the given links:-</blockquote><blockquote><strong>Kaggle:</strong> <a href="https://www.kaggle.com/kunalgohrani/current-trends-in-the-software-industry">https://www.kaggle.com/kunalgohrani/current-trends-in-the-software-industry</a></blockquote><blockquote><strong>Github:</strong> <a href="https://github.com/kunal-gohrani/Stack-Overflow-Developer-Survey-Analysis">https://github.com/kunal-gohrani/Stack-Overflow-Developer-Survey-Analysis</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d9289b3f2e77" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Different Types of Distance Metrics  used in Machine Learning]]></title>
            <link>https://medium.com/@kunal_gohrani/different-types-of-distance-metrics-used-in-machine-learning-e9928c5e26c7?source=rss-5931e2e03fe2------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9928c5e26c7</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Kunal Gohrani]]></dc:creator>
            <pubDate>Sun, 10 Nov 2019 10:02:23 GMT</pubDate>
            <atom:updated>2019-11-10T10:02:23.597Z</atom:updated>
            <content:encoded><![CDATA[<h3>Different Types of Distance Metrics used in Machine Learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TWE9TvvrTzVn12UZ8mGrEA.png" /></figure><p>Many Supervised and Unsupervised machine learning models such as K-NN and K-Means depend upon the distance between two data points to predict the output. Therefore, the metric we use to compute distances plays an important role in these models.</p><p>In this blog post, we are going to learn about some distance metrics used in machine learning models. They are:-</p><ul><li>Minkowski distance</li><li>Manhattan distance</li><li>Euclidean distance</li><li>Hamming distance</li><li>Cosine distance</li></ul><h3>Minkowski Distance</h3><p>According to Wikipedia, “A Normed vector space is a vector space on which a norm is defined.” Suppose A is a vector space then a norm on A is a real-valued function ||A||which satisfies below conditions -</p><ol><li><strong>Zero Vector- </strong>Zero vector will have zero length.</li><li><strong>Scalar Factor-</strong> The direction of the vector doesn’t change when you multiply it with a positive number though its length will be changed.</li><li><strong>Triangle Inequality-</strong> If the distance is a norm then the calculated distance between two points will always be a straight line.</li></ol><p>The distance can be calculated using the below formula:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/260/1*boqym__Ai1n-WxaR1X6Dhw.png" /></figure><p>Minkowski distance is a generalized distance metric. We can manipulate the above formula by substituting ‘p’ to calculate the distance between two data points in different ways. Thus, Minkowski Distance is also known as Lp norm distance.</p><p>Some common values of ‘p’ are:-</p><ul><li>p = 1, Manhattan Distance</li><li>p = 2, Euclidean Distance</li><li>p = infinity, Chebychev Distance</li></ul><p>We will discuss these distance metrics below in detail.</p><h4>Manhattan Distance:</h4><p>We use Manhattan distance, also known as <strong>city block distance</strong>, or <strong>taxicab geometry </strong>if we need to calculate the distance between two data points in a grid-like path. Manhattan distance metric can be understood with the help of a simple example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*52bc_cCCMgVF-SYZuWpTLQ.png" /></figure><p>In the above picture, imagine each cell to be a building, and the grid lines to be roads. Now if I want to travel from Point A to Point B marked in the image and follow the red or the yellow path. We see that the path is not straight and there are turns. In this case, we use the Manhattan distance metric to calculate the distance walked.</p><p>We can get the equation for Manhattan distance by substituting p = 1 in the Minkowski distance formula. The formula is:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*ph2xC44Zy-EHazYOom6tjg.png" /></figure><p><strong>When is Manhattan distance metric preferred in ML?</strong></p><blockquote>Quoting from the paper, <a href="https://bib.dbvis.de/uploadedFiles/155.pdf">“On the Surprising Behavior of Distance Metrics in High Dimensional Space”</a>, by Charu C. Aggarwal, Alexander Hinneburg, and Daniel A. Kiem. “ for a given problem with a fixed (high) value of the dimensionality d, it may be preferable to use lower values of p. This means that the L1 distance metric (Manhattan Distance metric) is the most preferable for high dimensional applications.”</blockquote><p>Thus, Manhattan Distance is preferred over the Euclidean distance metric as the dimension of the data increases. This occurs due to something known as the ‘curse of dimensionality’. For further details, please visit this <a href="https://bib.dbvis.de/uploadedFiles/155.pdf">link</a>.</p><h4>Euclidean Distance:</h4><p>Euclidean distance is the straight line distance between 2 data points in a plane.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yjxwoNqIwb1fXW26HvK_Kw.png" /></figure><p>It is calculated using the Minkowski Distance formula by setting ‘p’ value to 2, thus, also known as the L2 norm distance metric. The formula is:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*EUUQDZQktwUPkdCZaFicSQ.png" /></figure><p>This formula is similar to the Pythagorean theorem formula, Thus it is also known as <strong>the Pythagorean Theorem.</strong></p><h4>Hamming Distance:</h4><p>Hamming distance is a metric for comparing two binary data strings. While comparing two binary strings of equal length, Hamming distance is the number of bit positions in which the two bits are different.</p><p>The Hamming distance between two strings, a and b is denoted as d(a,b).</p><p>In order to calculate the Hamming distance between two strings, and, we perform their XOR operation, (a⊕ b), and then count the total number of 1s in the resultant string.</p><p>Suppose there are two strings 11011001 and 10011101.</p><p>11011001 ⊕ 10011101 = 01000100. Since, this contains two 1s, the Hamming distance, d(11011001, 10011101) = 2.</p><h4>Cosine Distance &amp; Cosine Similarity:</h4><p>Cosine distance &amp; Cosine Similarity metric is mainly used to find similarities between two data points. As the cosine distance between the data points increases, the cosine similarity, or the amount of similarity decreases, and vice versa. Thus, Points closer to each other are more similar than points that are far away from each other. Cosine similarity is given by Cos θ, and cosine distance is 1- Cos θ. Example:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/607/1*rBoComnSSWKgu4gj671wUg.png" /><figcaption>θ = 90</figcaption></figure><p>In the above image, there are two data points shown in blue, the angle between these points is 90 degrees, and Cos 90 = 0. Therefore, the shown two points are not similar, and their cosine distance is 1 — Cos 90 = 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*wTgudU3s2_oCXTTLhmVZkg.png" /><figcaption>θ = 0</figcaption></figure><p>Now if the angle between the two points is 0 degrees in the above figure, then the cosine similarity, Cos 0 = 1 and Cosine distance is 1- Cos 0 = 0. Then we can interpret that the two points are 100% similar to each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*23O5Hck8v5flOF9LtvGr8A.png" /></figure><p>In the above figure, imagine the value of θ to be 60 degrees, then by cosine similarity formula, Cos 60 =0.5 and Cosine distance is 1- 0.5 = 0.5. Therefore the points are 50% similar to each other.</p><h4>Where is it used?</h4><p>Cosine metric is mainly used in Collaborative Filtering based recommendation systems to offer future recommendations to users.</p><p>Taking the example of a movie recommendation system, Suppose one user (User #1) has watched movies like The Fault in our Stars, and The Notebook, which are of romantic genres, and another user (User #2) has watched movies like The Proposal, and Notting Hill, which are also of romantic genres. So the recommendation system will use this data to recommend User #1 to see The Proposal, and Notting Hill as User #1 and User #2 both prefer the romantic genre and its likely that User #1 will like to watch another romantic genre movie and not a horror one.</p><p>Similarly, Suppose User #1 loves to watch movies based on horror, and User #2 loves the romance genre. In this case, User #2 won’t be suggested to watch a horror movie as there is no similarity between the romantic genre and the horror genre.</p><h3>Conclusion</h3><p>In this blog post, we read about the various distance metrics used in Machine Learning models. We studied about Minkowski, Euclidean, Manhattan, Hamming, and Cosine distance metrics and their use cases.</p><p>Manhattan distance is usually preferred over the more common Euclidean distance when there is high dimensionality in the data. Hamming distance is used to measure the distance between categorical variables, and the Cosine distance metric is mainly used to find the amount of similarity between two data points.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9928c5e26c7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Indian Startup Ecosystem (An In-Depth Analysis)]]></title>
            <link>https://medium.com/@kunal_gohrani/indian-startup-ecosystem-analysis-bcd0e03a3692?source=rss-5931e2e03fe2------2</link>
            <guid isPermaLink="false">https://medium.com/p/bcd0e03a3692</guid>
            <category><![CDATA[communication]]></category>
            <category><![CDATA[analysis]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[trends]]></category>
            <category><![CDATA[indian-startup-ecosystem]]></category>
            <dc:creator><![CDATA[Kunal Gohrani]]></dc:creator>
            <pubDate>Tue, 20 Aug 2019 19:06:31 GMT</pubDate>
            <atom:updated>2019-08-20T19:06:31.260Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yfjc70F0fU2jjo5PTcur4A.jpeg" /><figcaption>(<a href="https://www.theindianwire.com/startups/economic-survey-places-india-third-position-global-startup-ecosystem-149732/">Source</a>)</figcaption></figure><p>Interested in forming a startup in India? Want to know more about the Indian Startup Ecosystem? Common trends in the industries? Then this blog is for you!</p><p>In this blog post, I will be sharing the results of my analysis on a fascinating data set I found in kaggle, that is the<a href="https://www.kaggle.com/sudalairajkumar/indian-startup-funding"> <strong>Indian Startups Funding</strong></a><strong> </strong>data set. The analysis has been done in a Python Jupyter Notebook available from this <a href="https://github.com/kunal-gohrani/Indian-Startup-Ecosystem-Analysis">link</a>.</p><p>The data set contains related information about each startup funded between 2015 and 2017, such as:</p><ul><li>Date of funding</li><li>Startup Name</li><li>Type of industry</li><li>City in which the startup is located</li><li>Investors</li><li>Investment types</li><li>Amount funded in USD, and so on.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GODlJjy5s-Kelv1mf4Qfmg.png" /></figure><p>The Questions I am going to answer from this analysis include:</p><ul><li>What is the Quarterly trend in the number of startups funded in the given duration?</li><li>Which startups got the highest funding? (OR) List the Top 20 startups based on the amount funded to them.</li><li>Which Industry has most startups? OR, Which Industry is more favored for funding by investors?</li><li>Which cities have the most number of startups funded?</li><li>Who are the top 10 investors in the Indian Startup Ecosystem?</li><li>What are the most common types of investments made by investors?</li></ul><p>So, let’s get started!</p><h3>1. What is the Quarterly trend in the number of startups funded between 2015 and 2017?</h3><p>The period between 2015 and 2017 saw around 2300+ startups being funded, out of which, 1917 startups were funded at least once by investors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GQvkhUakC0EM3dzFk6p9iQ.png" /></figure><p>According to the graph above, The 3rd Quarter of 2015 saw a huge rise in the number of startups funded by investors. The trend is mostly stable until 2017, where we see a huge decline in the number of startups being funded, with the lowest in the last quarter, having around 50 startups being funded. A factor contributing to this sharp decline would be the announcement of demonetization by the Indian Government back in November 2016.</p><h3>2. Which startups got the highest funding?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s98frrpagbh-FrENYn5D0A.png" /></figure><p>As seen from the above graph, we can see that <strong>Paytm, </strong>and <strong>Flipkart</strong> are the two companies which accumulated highest funding&#39;s by the investors during the given period. Exact funding raised by the top 20 startups are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/303/1*f6GXASmTTh6o4bpXxjJ9BA.png" /></figure><h3>3. Which Industry is more favored by investors for funding? OR, List the most common industries.</h3><p>Now, we will be seeing the industries that are favored by the investors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kbo3IvadZi8tG5PYdnIH5g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/416/1*diP8Wt0AOdAffXM95Nu8fg.png" /></figure><p>From the above two pictures, we can understand that the 3 industries favored by investors for funding are:</p><ul><li><strong>Consumer internet</strong></li><li><strong>Technology</strong></li><li><strong>E commerce.</strong></li></ul><p>Now, let’s look at the top 10 sub-categories favored by investors for funding:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*ibGDW6JSqPFik4iBLXTaaQ.png" /></figure><p>Seeing the above data, It looks like there are 9 startups listed under <strong>Online Pharmacy </strong>sub-category; followed by <strong>Food Delivery Platform</strong>, and <strong>Online Lending Platform </strong>having 8 and 5 startups listed respectively.</p><h4>Consumer Internet</h4><p>Top 10 startups in Consumer Internet industry are:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/259/1*4armsBHMqy_Lw2T6xvHm1Q.png" /></figure><h4>Technology</h4><p>Top 10 startups in the technology industry are:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*NAwQvrPpyJAp7dyQ_kpCFQ.png" /></figure><h4>E-commerce</h4><p>Top 10 startups in the E-commerce industry are:-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/262/1*lFT3uRcjUnwzINGCiv8XIQ.png" /></figure><h3>4. Which cities have the most number of startups?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/1*bOlAsAaFwYBx68QLymn9Nw.png" /></figure><p>Clearly, <strong>Bangalore</strong> has the most number of startups funded, Followed by <strong>Mumbai, New Delhi, Gurgaon, </strong>and <strong>Pune.</strong></p><h3>5. Who are the top 10 investors?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y_d5-Vr7ScmlrCpOi5gdRg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/447/1*1izxi6QLACOwwjyRu_GBXA.png" /></figure><p><strong>Sequoia Capital </strong>has invested in the most number of startups, Followed by <strong>Accel Partners</strong>,<strong> Kalaari Capital</strong>, and <strong>Blume Ventures.</strong></p><p><strong>Note:</strong> The above-mentioned investors have either funded startups as being the only investor or along with other investors.</p><h3>6. What are the most common types of investments made by investors?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5gd2WpZeNkiLJc_oBRmhcA.png" /></figure><p>We can see from the graph, 1200+ startups have been <a href="https://en.wikipedia.org/wiki/Seed_money">seed funded</a>, and 1000+ startups have been funded under <a href="https://www.investopedia.com/terms/p/privateequity.asp">private equity</a> type.</p><h3>Conclusion</h3><p>When you launch a startup, the excitement is high, but the downside is you’ll wear many hats full of high risk and uncertainty. While it can make you feel overwhelmed at times, knowing every area of your business increases the chance of being successful. From my analysis, we get to know a lot about the trends going on in the Indian Startup Ecosystem, which may help you in getting your startup off the ground.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bcd0e03a3692" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The 5 Steps of the Data Analysis Process]]></title>
            <link>https://medium.com/@kunal_gohrani/the-5-steps-of-the-data-analysis-process-2512ba6ac31e?source=rss-5931e2e03fe2------2</link>
            <guid isPermaLink="false">https://medium.com/p/2512ba6ac31e</guid>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[process]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Kunal Gohrani]]></dc:creator>
            <pubDate>Wed, 07 Aug 2019 18:25:17 GMT</pubDate>
            <atom:updated>2019-08-07T18:33:57.111Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*CTXbwESJyQ-4rK0kwRmohg.jpeg" /></figure><p>In today’s blog, We are going to learn about data analysis, and the various processes involved with it by utilizing examples taken from my GitHub repository, on Exploratory Data Analysis of the <a href="https://en.wikipedia.org/wiki/RMS_Titanic">Titanic</a> data set, done in a Jupyter Notebook, which can be viewed using this <a href="https://github.com/kunal-gohrani/Titanic-EDA/blob/master/DataAnalysis.ipynb">link</a>.</p><p>According to <strong>Wikipedia</strong>, Data analysis is a process of inspecting, cleansing, transforming and modeling data to discover useful information, informing conclusions and supporting decision-making.</p><p>In simplified terms, “Data analysis is the process of looking into the historical data of an organization, and analyze it with a particular aim in mind, that is, to draw potential facts and information and support decision-making process. Whatever decision we take in our lives, is by remembering what happened last time. Thus, data analysis greatly influences the decision-making process”</p><p>There are mainly 5 steps involved in the process of data analysis, they are: —</p><h3><strong>STEP 1: Asking the right question(s)</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/310/1*79pQ4b-uMyNFzPBZ-UhXNQ.png" /></figure><p>The first step towards any sort of data analysis is to ask the right question(s) from the given data. Identifying the objective of the analysis, it becomes easier to decide on the type(s) of data we will be needing to draw conclusions.</p><p>The objective behind analyzing the Titanic data set is to <strong>find out the factors which contributed to a person’s chance of survival on board the titanic.</strong></p><h3><strong>STEP 2: Data Wrangling</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WCsUHv7JSHRPO7AVDQDjSQ.jpeg" /><figcaption><a href="https://www.promptcloud.com/blog/what-is-data-wrangling-how-to-effectively-wrangle-data/">Source</a></figcaption></figure><p>“Data wrangling, sometimes referred to as data munging, or Data Pre-Processing, is the process of gathering, assessing, and cleaning of “raw” data into a form suitable for analysis.”</p><h4><strong>Data Wrangling has 3 sub-steps:-</strong></h4><h4><strong>Gathering of data</strong></h4><p>After identifying the objective behind our analysis, the next step is to collect the necessary data required by us to draw appropriate conclusions. There are various methods by which we can collect data. Some of which are: —</p><ul><li><strong><em>API or Web Scraping</em></strong> — If the data needed is available in particular website(s), then we can use the websites API (if available) or Web Scraping techniques to collect, and store data in our local storage/ databases. Often, data collected from the Internet is stored in a JSON format, and further processing is needed to convert JSON to the commonly used “.csv” format.</li><li><strong><em>Databases </em></strong>— If the data required is available in our companies databases, then we can easily use SQL queries to extract the data needed from them.</li><li><strong>Sites</strong> like <a href="http://www.kaggle.com/">kaggle.com </a>store data sets in appropriate formats to be downloaded by the members for practice/ competitions.</li></ul><p>For this blog post, we will use the <a href="https://www.kaggle.com/c/titanic/data">titanic data set</a> uploaded in kaggle.com for our analysis.</p><p>Firstly, Lets import all the libraries and the ‘train.csv’ data set we will be needing throughout our analysis.</p><pre>import numpy as np<br>import pandas as pd<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>%matplotlib inline</pre><pre>data=pd.read_csv(&#39;train.csv&#39;) # Gathering data<br>data.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zC1zaps4MVT87EwUYd2s8A.png" /></figure><h4><strong>Assessing of data</strong></h4><p>After the data has been gathered, stored in a supported format, and assigned to a variable in Python. It’s time to gain some high-level overview of the type of data we are dealing with. It includes gaining information such as: —</p><ul><li>The number of rows and columns present in the data set</li></ul><pre>print(&#39;Number of rows: &#39;,data.shape[0],&#39;\nNumber of columns: &#39;,data.shape[1])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/281/1*BxuEKeOuGRyfNklx3quQYw.png" /></figure><ul><li>columns present in the data set, along with the data type and number of non-null values</li></ul><pre>data.info()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*E-AyNo8rMN42gF2cRJE3RQ.png" /></figure><p>The above output shows the name of columns, along with several non-null values and the data type of each column. From the above output, it is clear that Age, Cabin, and Embarked columns have missing values, which we need to deal with in Data Cleaning stage.</p><p><strong>Let’s look at what information do these features (columns) represent: —</strong></p><ul><li>PassengerId — Unique ID of each passenger.</li><li>Survived — Binary Feature (consisting of only 0 or 1 values) which indicate whether that passenger survived or not. ( 0 = NO, 1 = YES).</li><li>Pclass — Indicates the socioeconomic status of the passenger. ( 1 = Upper Class, 2 = Middle Class, 3 = Lower Class ).</li><li>Name — Passenger name.</li><li>Sex — Male or Female.</li><li>Age — Age of the passenger.</li><li>SibSp — Number of siblings/ spouses of the passenger aboard the titanic.</li><li>Parch — Number of parents/ children of the passenger aboard the titanic.</li><li>Ticket — Ticket number of each passenger.</li><li>Fare — Fare paid by the passenger.</li><li>Cabin — shows the cabin number allotted to each passenger.</li><li>Embarked — Shows the port of embarkation of the passenger. ( C = Cherbourg, Q = Queens town, S = Southampton ).</li></ul><h4><strong>Data Cleaning</strong></h4><p>Data cleaning is the process of detecting and correcting missing, or inaccurate records from a data set. In this process, data present in the “raw” form (having missing, or inaccurate values) are cleaned appropriately so that the output data is void of missing and inaccurate values. Since no two data sets are same, therefore the method of tackling missing and inaccurate values vary greatly between data sets, but most of the time, we either fill up the missing values or remove the feature which cannot be worked upon.</p><p><strong>Fun Fact: </strong>Data Analysts usually spend about 70% of their time cleaning data.</p><p>In the Titanic data set, as noticed before, the age column has some missing values, which we will now deal with.</p><p><strong>Age</strong></p><pre>data[&#39;Age&#39;].describe()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*6w7lVAvc38RYk7zcaHq6QQ.png" /></figure><p>Age column has a mean of 29.69 and a standard deviation of 14.52. That means it’s not possible to simply fill the missing values as the mean value as the standard deviation is very high. So we will need a workaround. That is, we will generate a list of random numbers equal to the length of missing values, between (mean-standard deviation) and (mean+standard deviation). Then we can fill up the missing values in the Data Frame with that of those in the list.</p><pre>import random<br>print(&#39;Number of Missing values in Age:&#39;,data[&#39;Age&#39;].isnull().sum())</pre><pre>mean = data[&#39;Age&#39;].mean()<br>std = data[&#39;Age&#39;].std()<br>lower_limit = round(mean-std,0)<br>upper_limit = round(mean+std,0)<br>random_list=[]<br>for i in range(0,177):<br>    random_list.append(random.randint(lower_limit,upper_limit))</pre><pre>random_list=np.array(random_list)</pre><pre>age=data[&#39;Age&#39;].values</pre><pre>k=0<br>for i,j in enumerate(age):<br>    if np.isnan(age[i]):<br>        age[i]=random_list[k]<br>        k+=1</pre><pre>data[&#39;Age&#39;]=age</pre><pre>print(&#39;Number of missing values in age: &#39;,data[&#39;Age&#39;].isnull().sum())</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/1*CNZvpj2PgOTnctPwrByxiw.png" /></figure><p>So, Age column has been dealt with and all missing values have been replaced by random ages between (mean — standard deviation, mean + standard deviation)</p><blockquote>Please view the Jupyter notebook from the<a href="https://github.com/kunal-gohrani/Titanic-EDA/blob/master/DataAnalysis.ipynb"> link</a> provided to view the full data cleaning process.</blockquote><h3><strong>STEP 3: EXPLORATORY DATA ANALYSIS (EDA)</strong></h3><p>Once the data is collected, cleaned, and processed, it is ready for Analysis. As you manipulate data, you may find you have the exact information you need, or you might need to collect more data. During this phase, you can use data analysis tools and software which will help you to understand, interpret, and derive conclusions based on the requirements.</p><p>As the Titanic data set is now cleaned, we will now do some example EDA’s on it.</p><ol><li><strong><em>Analyze utilizing visualization techniques, which Gender was given more priority during the rescue operation?</em></strong></li></ol><pre># Adding a new column &#39;s&#39; to store survived status as a string for <br># better visualisations.<br>data[&#39;s&#39;]=&#39;&#39;<br>data.loc[(data[&#39;Survived&#39;]==1),&#39;s&#39;] = &#39;Survived&#39;<br>data.loc[(data[&#39;Survived&#39;]==0),&#39;s&#39;] = &#39;Not Survived&#39;<br>sns.countplot(x=&#39;Sex&#39;,hue=&#39;s&#39;,data=data)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HrxIK62kDAIQ_qtabMDXCg.png" /></figure><p>Thus, from the above visualization, we can infer that females were given priority during the rescue operation due to their low mortality count as compared to males.</p><p><strong><em>2. Find out whether class of a person contributed to its likelihood of survival.</em></strong></p><pre>sns.barplot(x=&#39;Pclass&#39;,y=&#39;Survived&#39;,data=data)<br>plt.title(&#39;Class vs Survived&#39;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UXGsoNZheT5d5LTRkFvJMA.png" /></figure><p>From the above visualization, we can infer that people belonging to the upper class were given the highest priority during the rescue operation, followed by middle, and lower classes. Lower classes also had the highest mortality count.</p><p>Exploring the data, SibSp and Parch columns generally show the number of relatives a passenger has on board, so SibSp and Parch, combined as relatives would make more sense.</p><pre>data[&#39;Relatives&#39;]=data[&#39;SibSp&#39;]+data[&#39;Parch&#39;]</pre><p>This step is known as <strong>Feature Engineering. </strong>Where we modify or make new features out of existing ones to better explain our analysis.</p><p><strong>NOTE: </strong>For full data analysis, please view the Jupyter notebook file from this <a href="https://github.com/kunal-gohrani/Titanic-EDA/blob/master/DataAnalysis.ipynb">link</a>.</p><h3>STEP 4: CONCLUSION</h3><p>After the analysis phase is completed, the next step is to interpret our analysis and draw conclusions from it.</p><p>As we interpret the data, there are 3 key questions which should be asked by us: —</p><ul><li>Did the analysis answer my original question?</li><li>Was there any limitation in my analysis which would affect my conclusions?</li><li>Was the analysis sufficient enough to help decision making?</li></ul><p>From the analysis of the titanic data set (<a href="https://github.com/kunal-gohrani/Titanic-EDA/blob/master/DataAnalysis.ipynb">link</a>), we were able to find out the major factors which contributed to a person’s chance of survival.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/1*soGY7Y5FBnwgx41CkhKYPw.png" /></figure><ul><li>Males had a higher chance of survival if they belonged to the upper class. Had an age between 0 to 4 years old, or 18 to 50 years old, and had 1 to 3 relatives traveling onboard the titanic.</li><li>Females had a higher chance of survival irrespective of their class, but if they had an age between 0 to 4 years old, or 15 to 50 years old, and had 0 to 4 relatives traveling onboard the titanic.</li></ul><h3><strong>STEP 5: COMMUNICATING RESULTS</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*sTjxrvr5uDtJZwrxxktCyw.jpeg" /><figcaption><a href="https://www.slicktext.com/blog/2013/09/text-marketing-for-mass-communication/">Source</a></figcaption></figure><p>Now that data has been explored, conclusions have been drawn; it’s time to communicate your findings to the concerned people or communicating to mass employing data storytelling, writing blogs, making presentations or filing reports. Great communication skills are a plus in this stage since your findings need to be communicated in a proper way to other people.</p><h3><strong>A Fun Fact</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/1*YRnAJGj8hI76nLnCsAPV0Q.png" /></figure><p>The five steps of data analysis are not followed linearly, it is actually non-linear in nature. To explain this, let’s consider an example: —</p><p>Supposedly, you have done your analysis, drawn conclusions, then suddenly you find the possibility of representing a feature in a better way, or to construct a new feature out of other features present in the data set; thus, you would go back to step 3, perform feature engineering, and again perform the EDA with the new features added.</p><p>Thus, it is not always possible to follow these steps linearly.</p><h3><strong>C</strong>ONCLUSION</h3><p>The amount of data generated by organizations per day around the world is in the range of zettabytes; and till now, it remains underutilized. Data Analysis can help organizations gain useful insights from their data, and influence a better decision-making process.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2512ba6ac31e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring the Mystery Behind Different Job Titles For Machine Learning and Data Science]]></title>
            <link>https://medium.com/campusx/exploring-the-mystery-behind-different-job-titles-for-machine-learning-and-data-science-9352849f283a?source=rss-5931e2e03fe2------2</link>
            <guid isPermaLink="false">https://medium.com/p/9352849f283a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[job-hunting]]></category>
            <category><![CDATA[careers]]></category>
            <dc:creator><![CDATA[Kunal Gohrani]]></dc:creator>
            <pubDate>Fri, 26 Jul 2019 16:14:16 GMT</pubDate>
            <atom:updated>2019-07-29T16:37:18.809Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Exploring the Mystery Behind Different Job Titles for Machine Learning and Data Science</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sRc3Y7fYb4DttN18slQnyw.png" /><figcaption>Relation between skill set and job title(<a href="https://learn.co/lessons/introduction-to-data-science">Source</a>)</figcaption></figure><p>Do you want to start a career in the field of Data Science and Machine Learning; but confused about the different job titles available in this Data-Driven career and the appropriate skill sets needed to excel in one?</p><p><strong>Then this article is for you!</strong></p><p>This article aims to demystify the different job titles for data science and machine learning based career paths. We would look into some job titles such as <strong>Data Analyst, Data Scientist, Data Engineer, and Machine Learning Engineer; </strong>and learn about the job &amp; responsibilities of each title along with the skills/qualifications required and estimated salary earned in different job titles.</p><h4><strong>DATA ANALYST</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*AeuuMzMUZqTOkrBVAE_kSA.jpeg" /><figcaption><a href="https://www.kdnuggets.com/2015/11/different-data-science-roles-industry.html">Source</a></figcaption></figure><p>As defined in Wikipedia:<strong> </strong>Data analysis is a process of inspecting, cleansing, transforming and modeling data to discover useful information, informing conclusions and supporting decision-making. Data analysts collect, process and perform statistical analysis on the given data.</p><p><strong>Responsibilities of a Data Analyst</strong></p><ul><li><em>Cleaning and organizing Raw data.</em></li><li><em>Analyzing data to derive insights.</em></li><li><em>Creating data visualizations.</em></li><li><em>Producing and maintaining reports.</em></li><li><em>Collaborating with teams/colleagues based on the insight gained.</em></li><li><em>Optimizing data collection procedures.</em></li></ul><p><strong>Skills required to be a data analyst</strong></p><p>A degree in the following subjects is beneficial in developing a career as a data analyst:</p><ul><li><em>Mathematics</em></li><li><em>Computer Science</em></li><li><em>Statistics</em></li></ul><p>Furthermore, additional skills required to be a data analyst are:</p><ul><li><em>Statistical Programming</em></li><li><em>Programming Languages (R/SAS/Python)</em></li><li><em>Creative and Analytical Thinking</em></li><li><em>Business Acumen — Medium to High preferred</em></li><li><em>Strong Communication Skills.</em></li><li><em>Data Mining, Cleaning, and Munging</em></li><li><em>Data Visualization</em></li><li><em>Data Warehousing</em></li><li><em>Data Story Telling</em></li><li><em>SQL Databases</em></li><li><em>Database Querying Languages</em></li><li><em>Advanced Microsoft Excel.</em></li></ul><p><strong>Average Annual Salary of Data Analysts</strong></p><p>An entry-level data analyst may receive an annual salary anywhere between $40,000 — $80,000 while experienced analysts can expect to receive between $65,000 — $120,000 approximately.</p><h4>DATA SCIENTIST</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*bGv2ha_7BYurB3sLzcukbQ.jpeg" /></figure><p>“A data scientist is someone who is better at statistics than any software engineer and better at software engineering than any statistician”. — <a href="https://www.quora.com/What-is-the-difference-between-a-data-scientist-and-a-statistician">Josh Wills on Quora</a></p><p>At the core of data science, Data Analysts and Data Scientists have the same job, that is, getting valuable information from the data. Nevertheless, they differ largely in their job roles &amp; responsibilities.</p><p>Data Analysts derives insights from historical data and answers questions such as “What is the cause behind decreased sales?”, “What is the reason behind a particular products failure?”, and shares it with the concerned people for informed business decisions. On the other hand, Data Scientists, are more forward-looking people. They are more concerned with using these insights combined with Machine Learning, hypothesis, Statistical Tests, A/B testing for further development of products. They are more into asking questions such as “Will a larger motor in my product create more demand for it?”, or “Will targeting a particular market to enter into help me grow my company?”.</p><p>The above is the general difference between the roles of Data Analyst and a Data Scientist, but it may not always be the case since data science is still new and is far from being standardized. You can sometimes have data scientists doing basic analysis work and data analysts performing ML modeling.</p><p>Regardless of the titles, these two career profiles are the most important ones that are being sought after by employers when it comes to analytic roles in data science, and therefore should be our targets when job searching and thinking about the best fit</p><p>To summarize, Skills required to be a Data Scientist include that of Data Analysts, Mathematics, Machine Learning, Deep Learning, Reinforcement Learning, Hypothesis Testing, Subject Matter Expertise (SME), Intuition.</p><p><strong>Salary of Data Scientists</strong></p><p>A Data Scientist’s annual salary ranges anywhere from $80,000 to $160,000 depending upon the experience and subject matter expertise.</p><h4>DATA ENGINEERS</h4><p>Gordon Lindsay Glegg once quoted, “A scientist can discover a new star, but he cannot make one. He would have to ask an engineer to do it for him.”</p><p>So, the role of the Data Engineer is really valuable.</p><p>Data Engineers are people who are in charge of the delivery, storage, and processing of data. The main task of data engineers is to provide a reliable infrastructure for data. If we look at the AI Hierarchy of Needs, data engineering occupies the first 2–3 roles in it: <strong>Collect, Move &amp; Store, Data Preparation.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XakCd8uFi72C2nznkdBYDw.png" /><figcaption>AI Hierarchy of Needs (<a href="https://hackernoon.com/the-ai-hierarchy-of-needs-18f111fcc007">Source</a>)</figcaption></figure><p><strong>Responsibilities of Data Engineer</strong></p><ul><li>Scrape Data from the given sources.</li><li>Move/Store the data in optimal servers/warehouses.</li><li>Build data pipelines/APIs for easy access to the data.</li><li>Handle databases/data warehouses.</li></ul><p><strong>Skills required to be a Data Engineer</strong></p><ul><li>Strong grasp of algorithms and data structures</li><li>Programming Languages (Java/R/Python/Scala) and script writing</li><li>Advanced DBMS’s</li><li>BIG DATA Tools (Apache Spark, Hadoop, Apache Kafka, Apache Hive)</li><li>Cloud Platforms (Amazon Web Services, Google Cloud Platform)</li><li>Distributed Systems</li><li>Data Pipelines</li></ul><p><strong>Is a Data Engineer more in demand than Data Scientist?</strong></p><p>The answer is YES, To explain it, let’s consider an analogy:- A potato chips factory needs lots of potatoes daily to be stored before it can start processing it. But potatoes need to be harvested, cleaned and transported before they can become potato chips. Similarly, A piece of data needs to be first collected, stored, and pipelines need to be made for easy access to data before it can be worked on. Therefore, Data Engineering is the preliminary step in any data related work and it is more in demand than any other data-science based job roles.</p><p><strong>Salary of a Data Engineer</strong></p><p>Salary of a Data Engineer varies anywhere in the range between $110,000 to $155,000 per year depending on skills, experience, and location.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*2iSfhOOwujNj9BLu4O1fHg.png" /><figcaption>Salary Range of Data Engineers</figcaption></figure><h4><strong>MACHINE LEARNING ENGINEER</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VUCxUt0dJVs0if27T_gYdQ.png" /></figure><p>The work of a Machine Learning Engineer is to bridge the gap between Data Scientist’s work and production environment. A Machine Learning Engineer is more concerned with deploying production-ready models. For an example, think of a recommender system machine learning model trained with a data set and has achieved great results, now that model needs to be deployed as part of a company’s website to make the site better and thus, help generate more revenue. But what if that model is not production-ready yet? This is where a Machine Learning Engineer comes into play. Deploying machine learning models to a production-ready environment is an engineering concern that is different from building the model. It involves different types of engineering work such as integrating the model to a software system, optimizing the model for performance and scalability, and re-training it with new data, monitoring, and maintenance the ML system</p><p>Thus, Responsibilities of Machine Learning Engineer include:</p><ul><li>Deploying machine learning models to production ready environment</li><li>Scaling and optimizing the model for production</li><li>Monitoring and maintenance of deployed models</li></ul><p>Skills required to be a Machine Learning Engineer:</p><ul><li>Linear Algebra, Calculus</li><li>Probability and Statistics</li><li>Programming Languages (R/Python/Java/Scala mainly)</li><li>Distributed Systems</li><li>Data model and evaluation</li><li>Machine Learning models</li><li>Software Engineering &amp; Systems design</li></ul><h3>CONCLUSION</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uL7paMWm4_IxCzhw8HBMyA.png" /><figcaption>Comparison of skills required for different job roles</figcaption></figure><p>Presuming, now you have a brief idea about the different job profiles. From here, it’s all about selecting a field you are interested to work in and start planning out on building the necessary skill set. The demand for Data-Driven careers are always gonna soar high as more and more data is being generated every single day and there’s a high demand for people in these fields.</p><p><strong>All the best for your career and job search in Data Science!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9352849f283a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/campusx/exploring-the-mystery-behind-different-job-titles-for-machine-learning-and-data-science-9352849f283a">Exploring the Mystery Behind Different Job Titles For Machine Learning and Data Science</a> was originally published in <a href="https://medium.com/campusx">CampusX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning: A Non-Technical Point of View]]></title>
            <link>https://medium.com/@kunal_gohrani/machine-learning-a-non-technical-explanation-da82e8fef030?source=rss-5931e2e03fe2------2</link>
            <guid isPermaLink="false">https://medium.com/p/da82e8fef030</guid>
            <category><![CDATA[non-technical]]></category>
            <category><![CDATA[explanation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[inside-ai]]></category>
            <dc:creator><![CDATA[Kunal Gohrani]]></dc:creator>
            <pubDate>Tue, 23 Jul 2019 23:07:25 GMT</pubDate>
            <atom:updated>2019-07-23T23:40:46.641Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xm72ucTsCVFEzoozrvkujg.jpeg" /></figure><blockquote>This article aims to make readers understand what Machine Learning is from a Non-Technical point of view. We will jump right off by explaining machine learning with the help of an analogy and look on to the difference between machine learning algorithms based on their tasks. We will also try to understand the impacts of the application of machine learning in various domains.</blockquote><p>Have you ever wondered as a child, when playing games like FIFA, Counter-Strike, on a gaming platform against Computer/ Bots that how does the computer know how to play this game? Who taught it?</p><p>Did you ever want to know that how does the suggest words feature works in Android and iPhone keyboards? How apps like Netflix, Spotify can recommend you movie, songs that you might like? Or how is the automatic face tag feature of Facebook so accurate in tagging people in photos to their profiles?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*Va1ZT758hKY1qX8oc5sngw.jpeg" /></figure><p>All this has been made possible by the application of <strong>Machine Learning </strong>in various domains. For those that don’t already have a clear understanding of what Machine Learning is.</p><blockquote>Machine learning is the ability for a system to learn and improve from experience, without being explicitly programmed.</blockquote><blockquote>Machine Learning is also more commonly known as AI and is a subset of technologies that make up “Artificial Intelligence”.</blockquote><p>Machine Learning is the way a computer can be trained to solve a domain-specific problem by feeding it the domain-related data set (a collection of related data consisting of features, and the response variable, also known as the outcome variable) and then testing it.</p><p>To understand machine learning in a simpler way, let us imagine that we are teaching a student how to read, and write in English, and then we test her gained knowledge by means of examination to see if she has learned how to read sentences and write in English and how proficient she is in reading and writing.</p><p>Thinking of the machine learning model as that student, we first train it to understand the given data and find out the relationship between the given feature(s) in the data and the response variable. Then we test it by examining it based on testing data to evaluate the score of that model. The score of the model indicates how good the trained model is in predicting the response variable on being given some feature(s).</p><h3><strong>Machine Learning’s two key tasks</strong></h3><p>Now that we have a basic understanding of what machine learning is, let’s look at the fundamental tasks in which machine learning can be put to use:- Classification, Regression.</p><p><strong>Classification</strong>: Classification is the process of classifying something, it is the process where a model is given a data set, supposedly images of animals and then after training, the model can classify whether the given image is that of a dog or a cat.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*RhlrK45nbn8Q2rGFW__19Q.jpeg" /></figure><p><strong>Regression:</strong> Regression deals with predicting real numerical value(s) based on some relation between the given feature(s) and the response variable. Example: - We can use a Machine Learning model to evaluate the price of a house based on some features such as the number of rooms, square feet, location of the house and so on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*CKeAPlEzSFn46i3lJSNsZg.jpeg" /></figure><h3><strong>Impact of application of ML in various domains</strong></h3><p><strong><em>Gaming Industry: — </em></strong>Machine learning has a huge impact on the way games are developed in the industries. In the search for more realistic worlds, captivating challenges, and unique content, video game development industries are increasingly turning to machine learning as a useful weapon in game development. Machine learning algorithms, when used in games, can respond to a player’s actions dynamically. Unlike a traditional pre-scripted game where all the Non-Player Characters (NPCs) actions are scripted, a video game with a machine learning engine could react and change how the virtual world, non-player characters (NPCs), or objects behave in real-time, based on the player’s actions and decisions. Enabling games developed with machine learning to react and respond more dynamically and in more not imaginative ways to the player. Thus, the bot players in games like Counter-Strike, FIFA is the result of the application of machine learning in those games to train the NPCs to react in an appropriate way to the actions of the player.</p><p><strong><em>Social Media: </em>— </strong>Social media giants like Facebook, Instagram, and WhatsApp have many machine learning models in place to automate various tasks or incorporate new features on their websites. Example:-</p><ul><li><strong>Facebook</strong> uses a Deep Learning application called <strong>DeepFace</strong> to teach it to recognize people in photos. It says that its most advanced image recognition tool is more successful than humans in recognizing whether two different images are of the same person or not — with DeepFace scoring a 97% success rate compared to humans with 96%.</li><li><strong>WhatsApp </strong>uses a machine learning system designed to<strong> </strong>tackle the problem of spreading of fake news. The company uses this machine learning system to detect users with abusive behavior and those sending bulk messages and creating multiple accounts with the sole purpose of spreading false and abusive content and to ban those accounts.</li><li><strong>Instagram</strong> uses machine learning to fight spam. The spam filter can remove fake messages from accounts written in nine languages including English, Chinese, Russian, Arabic and more. Once messages are detected they are automatically removed. Instagram uses Facebook’s artificial intelligence <a href="https://bernardmarr.com/default.asp?contentID=1754">text analytics</a> algorithm <strong>DeepText</strong> that can understand the context of a message nearly as good as humans. Instagram also started using machine learning to stop bullying, with the help of DeepText and user marking posts as offensive, Instagram is capable of automatically removing posts DeepText finds to be offensive and bullying in nature.</li></ul><p><strong><em>Entertainment and retail: — </em></strong>Apps like Netflix, Amazon Prime Video, Spotify, use recommender systems to suggest you content related to your history and preference of language, genre. Retail apps like Amazon, Flipkart use recommender systems to suggest you new products based on your browsing history, Product searches and so on.</p><p><strong><em>Government Operations: — </em></strong>such as Police, use image recognition systems to monitor the flow of traffic in and around cities and for automatic penalization of rule breakers. Image Recognition systems can also be used for automatic detection of maintenance work needed on the roads, bridges and so on.</p><p><strong><em>Financial Services: — </em></strong>Banking sectors and the Government use machine learning to gain valuable insights from a customer’s data, prevent fraud and tax evasion. Introduction of machine learning in the domain of financial services has greatly improved this sector and has helped in reducing tax evasion, and frauds by allowing the bank to predict whether a particular person will be able to return his/ her loan based on the history of that person.</p><p><strong><em>Transportation sector: </em>— </strong>Industries in the business of delivery cargo, ferrying passengers to their destination use machine learning algorithms to calculate the optimal route for the trip based on conditions such as traffic jam, time taken to reach the destination. This helps them in preventing losses and wastage of fuel incurred due to poor route decisions.</p><h3>Conclusion</h3><p>Machine Learning (ML) is the area of computational science that focuses on analyzing and interpreting patterns in data to enable learning, reasoning, and decision. Simply put, machine learning allows the user to feed a computer algorithm an immense amount of data and have the computer analyze and make data-driven decisions based on only the input data. If any corrections are identified, the algorithm can incorporate that information to improve its future decision making.</p><p>With the increase in available data through data mining, and cheap computing power. Machine Learning has become important because data is the lifeline of all businesses. Data-driven decisions increasingly make the difference between keeping up with the competition or falling further behind.</p><h3><strong>References</strong></h3><ul><li>“Non-Technical Guide to Machine Learning and AI” <a href="https://medium.com/@Cubix.Worldwide/non-technical-guide-to-machine-learning-and-ai-fb75eb6c1964">https://medium.com/@Cubix.Worldwide/non-technical-guide-to-machine-learning-and-ai-fb75eb6c1964</a></li><li>“6 Ways Machine Learning will be used in Game Development” <a href="https://www.logikk.com/articles/machine-learning-in-game-development/">https://www.logikk.com/articles/machine-learning-in-game-development/</a></li><li>“What is Machine Learning?” <a href="https://www.netapp.com/us/info/what-is-machine-learning-ml.aspx">https://www.netapp.com/us/info/what-is-machine-learning-ml.aspx</a></li><li>“WhatsApp uses machine learning to detect bulk messaging and automated behaviour” <a href="https://www.fonearena.com/blog/274685/whatsapp-machine-learning-detect-bulk-messaging-automated-behaviour.html">https://www.fonearena.com/blog/274685/whatsapp-machine-learning-detect-bulk-messaging-automated-behaviour.html</a></li><li>“Instagram now uses machine learning to detect bullying within photos” <a href="https://techcrunch.com/2018/10/09/instagram-bullying-photos/">https://techcrunch.com/2018/10/09/instagram-bullying-photos/</a></li><li><a href="https://medium.com/@vishweshshetty/what-is-machine-learning-in-layman-terms-a-guide-for-non-tech-co-founders-e041cfb0aab2">“</a>Machine Learning in Layman’s Terms — A Guide for Non-Tech Co-Founders!” <a href="https://medium.com/@vishweshshetty/what-is-machine-learning-in-layman-terms-a-guide-for-non-tech-co-founders-e041cfb0aab2">https://medium.com/@vishweshshetty/what-is-machine-learning-in-layman-terms-a-guide-for-non-tech-co-founders-e041cfb0aab2</a></li><li>“10 Machine Learning Terms Explained in Simple English” <a href="http://blog.aylien.com/10-machine-learning-terms-explained-in-simple/">http://blog.aylien.com/10-machine-learning-terms-explained-in-simple/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da82e8fef030" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>