<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Savinay Narendra on Medium]]></title>
        <description><![CDATA[Stories by Savinay Narendra on Medium]]></description>
        <link>https://medium.com/@savinaynarendra?source=rss-549eaeb1e304------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*MLXX47e74SCHOk4kI4FhCg.png</url>
            <title>Stories by Savinay Narendra on Medium</title>
            <link>https://medium.com/@savinaynarendra?source=rss-549eaeb1e304------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 19:57:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@savinaynarendra/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Common python functions to know]]></title>
            <link>https://medium.com/@savinaynarendra/common-python-functions-to-know-4701a533da88?source=rss-549eaeb1e304------2</link>
            <guid isPermaLink="false">https://medium.com/p/4701a533da88</guid>
            <dc:creator><![CDATA[Savinay Narendra]]></dc:creator>
            <pubDate>Sat, 25 Sep 2021 22:00:42 GMT</pubDate>
            <atom:updated>2021-09-25T22:00:42.373Z</atom:updated>
            <content:encoded><![CDATA[<p>Sort a dictionary by value</p><pre>sorted(x.items(), key=lambda item: item[1])</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4701a533da88" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sparkify Project — Predicting User Churn]]></title>
            <link>https://medium.com/@savinaynarendra/sparkify-project-predicting-user-churn-8d9ee4185274?source=rss-549eaeb1e304------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d9ee4185274</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[apache-spark]]></category>
            <dc:creator><![CDATA[Savinay Narendra]]></dc:creator>
            <pubDate>Wed, 24 Jul 2019 14:46:12 GMT</pubDate>
            <atom:updated>2019-08-02T18:24:19.213Z</atom:updated>
            <content:encoded><![CDATA[<h3>Sparkify Project — Predicting User Churn</h3><p>Suppose, we are the data team of a popular digital music service similar to Spotify or Pandora. The users stream their favorite songs either using the free tier that place advertisements between songs or use the premium plan for which they pay a monthly fee without any advertisements between songs. Users can upgrade, downgrade or altogether cancel their services. The dataset given to us contains a log of the activities of each user on the service whether they are playing songs, logging out, upgrading their service or cancelling it. All this data contains key insights in keeping the users of the service happy. Our task in this project is to develop a model that predict which users are at risk. If we can identify users that are at risk to churn either by downgrading from premium or by cancelling their service, the business can offer them incentives and discounts potentially saving millions in revenues.</p><p>In this post, we analyze the data from a song application to predict the churn from users. The dataset (128 MB) we use here is a subset of the whole data. It contains a total of 286500 rows of user events and has 18 columns. The columns include:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/1*Xi6BfZ_8k8PAwR3A9coKFw.png" /></figure><p>The page column contains a log of events mentioned below. The events include:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*q19TCZ-jC3dHh_4WdJmQnQ.png" /></figure><p>Churn is defined as Submit Downgrade or Cancellation Confirmation event. If a user has at some instance performed the above two activities, he is considered a churned user.</p><p>Here are some descriptive statistics of the most relevant features related to the dataset given to us:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZfGS5VZfe9vVOJBB8BcHQw.png" /><figcaption>Descriptive Statistics</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/464/1*VnLKpVZjAY0WpAuuigVfEg.png" /><figcaption>Churned users by gender</figcaption></figure><p>This is the distribution of users by gender. From the visualization, both males and females are equally likely to downgrade their services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/406/1*K8bol1Id62YlmvgQPp_9JQ.png" /><figcaption>Churned users by level</figcaption></figure><p>It is more likely that paid users churn than users who have a free account.</p><h3>Feature Engineering</h3><p>To solve this problem, first we load the dataset into Apache Spark and delete any rows that have missing userIds or sessionIds. Then we perform feature engineering on the dataset to calculate metrics that will serve as inputs and outputs for our machine learning model. The metrics that we calculate include:</p><ol><li>average number of songs per session</li><li>Total number of thumbs down</li><li>Total number of thumbs up</li><li>Total number of errors encountered</li><li>Number of Add to Playlist events</li><li>Number of Add Friends events</li><li>Total number of songs played</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/424/1*jOQTnJpAX7Jgx78T8Ghmfw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/412/1*w_IXLdK8qlpAPI3TYj7PaQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/1*326mrF3XzyrglXwfP8Hj5A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/1*tKjjLLKtQuAZLaGCrdiL7w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/454/1*WDvXRNbqQNJ0GqmZZTkmMQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*SurCP96FG0xjRqlYwwUWXw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*qaLYGkWfLHsvr_skg_9Mpw.png" /></figure><p>From the above visualizations, we learn that:</p><ol><li>Users who downgraded their service had a lesser number of songs on average per session.</li><li>Users who downgraded had more thumbs down than users who did not downgrade.</li><li>Users who downgraded also had more thumbs up events.</li><li>Users who churned on average added more friends</li><li>Churned users played more songs</li></ol><p>The label which needs to be predicted is churn which is defined by a user who has performed either a “Submit Downgrade” event or “Cancellation Confirmation” event.</p><h3>Modeling</h3><p>To build our model, we use the following supervised learning algorithms:</p><ol><li>Logistic Regression (LR)</li><li>Random Forest Classifier (RF)</li><li>Gradient Boosted Tree Classifier (GBT)</li></ol><p>The dataset is split into training and validation data in a ratio of 90% to 10%. A machine learning pipeline is built in which we first convert each row into a single feature vector. Then data in each of the column is normalized and is fed into each of the above classifiers. Also we find the best hyper-parameters for each of the models using grid search and a 3-fold cross validation was performed on the dataset.</p><p>On the dataset prepared from the above features, Gradient Boosted Tree Classifier with number of trees equal to 5 had the best F1 score of 0.9067.</p><p>The training model was also validated by calculating the F1 score for each fold while performing cross validation. We got an average F1 score 0.69.</p><h3>Calculating Feature Importances</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*JEjgoJSihwVmPUGpRaEA3Q.png" /><figcaption>Feature Importances</figcaption></figure><p>This shows the most important features in predicting the output. Count of thumbs up is the one that contributes the most in predicting user churn. Other important features include total songs played and the average length of the sessions for a user.</p><h3>Conclusion</h3><p>In this project, we build a model to predict user churn from a song streaming service. We first perform feature engineering on our dataset based on the activities of the users. After the features are calculated, the data is fed into three models. Out of those, Gradient Boosted Tree classifier gives us the best results. Count of thumbs up event is the most important feature that helps in predicting user churn.</p><p>One of the things I found difficult was deciding which features to calculate to build the model. I used my intuition to calculate features I felt would be useful in predicting churn. Other features that I could have used are:</p><ol><li>Average length of each session for a user</li><li>Logout events</li></ol><p>Implementing these features would make our model more robust and accurate.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d9ee4185274" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How has the technology industry changed over years?]]></title>
            <link>https://medium.com/@savinaynarendra/how-has-the-technology-industry-changed-over-years-c91cbdd24681?source=rss-549eaeb1e304------2</link>
            <guid isPermaLink="false">https://medium.com/p/c91cbdd24681</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[stackoverflow]]></category>
            <dc:creator><![CDATA[Savinay Narendra]]></dc:creator>
            <pubDate>Wed, 05 Jun 2019 05:40:06 GMT</pubDate>
            <atom:updated>2019-06-05T05:40:06.411Z</atom:updated>
            <content:encoded><![CDATA[<p>Are the technologies that we used in 2011 still prevalent or do we need to update ourselves ? Are there currently more number of women in our industry as compared to 2011? Which country produces the most number of developers?</p><p>To get an insight into the technology industry, we analyze data from stackoverflow survey from 2011 to 2019. We discover some really good and interesting insights by doing this analysis. Some of the questions that I have tried to answer here are:</p><ol><li>Has the number of women in technology grown over these years?</li><li>What languages have been most popular for each year?</li><li>Which country has the highest number of developers active on stackoverflow for each year?</li><li>Which is the most loved IDE by developers over the years?</li></ol><p>The stackoverflow survey over the years has become more popular with only 2814 respondents in 2011 to 88883 respondents in 2019. The 2019 survey represented 179 countries and has questions like technology they use, current profession, job satisfaction level, education level and many others.</p><h3>Growth of Women in Technology?</h3><p>One of the first questions that came to my mind was whether the percentage of women in technology force has increased as compared to men. We can observe there has been an increase of 4.7% in 2014 to 7.5% in 2019. This growth can be attributed to including more women and improving diversity within the teams. The trends are encouraging but I had expected the growth rate of participation of women to be more than that is observed. This may be because the number of males are also increasing in the industry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*ZioJsydrgWzSRdcZcTU_4A.png" /></figure><p>Let us also analyze the percentage of male respondents:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/1*DcygnAEj0ojHLFn8MBNFIA.png" /></figure><p>As we can see from the chart above, the male respondents have increased from 91.2% to 93.4%. More number of males are also joining the industry.</p><h3>Most popular programming language?</h3><p>This is a very important question for all developers. Out of a plethora of programming languages, which language should a developer learn to maximize his job and salary opportunities? After analyzing the data from 2011 to 2019, we figured out SQL and Javascript have been the most popular languages that developers work on. Here is a visualization to depict it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/874/1*uScu6fOZ1IuwWCyX4L2wlw.png" /><figcaption>Most popular languages</figcaption></figure><p>Javascript is the most popular language among developers in 2019 with 70% of the developers working on it. So, if you want to be in demand, Javascript is the way to go :)</p><h3>Which country has the most developers?</h3><p>According to Stackoverflow, most of the developers on its platform are from United States for each year. The chart below depicts this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/874/1*qViD0tRnEGMQVd-MMmP8sQ.png" /><figcaption>United States is the clear winner!!</figcaption></figure><p>However, some features of stackoverflow might be blocked in China as the website loads Jquery from Google CDN and Google is blocked in China. So, this might have led to less usage of stackoverflow in China. In my opinion, if the website was as accessible in China as it is in USA, China would have given a close fight for the most number of developers.</p><h3>Most popular Development Environment?</h3><p>This data is only available from 2017 to 2019. Visual Studio is by far the most popular choice among developers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/874/1*p2VYFFp1JxC5GMKjRmtRhQ.png" /><figcaption>Visual Studio — Most popular IDE</figcaption></figure><h3>Conclusion</h3><p>In this post, we try to study the trends in the technology industry from 2011 to 2019. Stackoverflow is a good representation of the technology industry as most programmers use the platform as a question and answer site. We observed the following trends:</p><ol><li>There has been an increase in the percentage of women in the industry. However, the increase is not very steep and we should take more initiatives to improve the diversity.</li><li>Javascript is the most popular language followed closely by SQL</li><li>Most number of developers are located in United States</li><li>Visual Studio is the preferred choice for programmers as a development environment</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c91cbdd24681" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>