<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Hider Toula on Medium]]></title>
        <description><![CDATA[Stories by Hider Toula on Medium]]></description>
        <link>https://medium.com/@hider.toula?source=rss-2b2c7b59cf0c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Hider Toula on Medium</title>
            <link>https://medium.com/@hider.toula?source=rss-2b2c7b59cf0c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 16:03:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hider.toula/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Churn Analysis using PySpark]]></title>
            <link>https://medium.com/@hider.toula/churn-analysis-using-pyspark-dccfea13035f?source=rss-2b2c7b59cf0c------2</link>
            <guid isPermaLink="false">https://medium.com/p/dccfea13035f</guid>
            <category><![CDATA[spark]]></category>
            <category><![CDATA[music]]></category>
            <category><![CDATA[udacity]]></category>
            <dc:creator><![CDATA[Hider Toula]]></dc:creator>
            <pubDate>Sun, 24 May 2020 22:36:16 GMT</pubDate>
            <atom:updated>2020-05-24T22:36:16.348Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_11DxpGxvE5RYx3JvQqDpg.png" /></figure><p>consider sparkfy as a popular digital music service similar to Spotify or Pandora, and that this company has provided us with a set of data regarding their users, Users stream their favorite songs using the free level which places ads between songs or using the premium plan for which they pay a monthly subscription without any advertising between songs. Users can upgrade, downgrade or completely cancel their services. Our task in this project is to develop a model that predicts which users are at risk. If we can identify users who are at risk of opting out either by downgrading the premium or canceling their service, the company can offer them incentives and discounts that potentially save millions of revenues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/1*CrfKnQA5EYOUGHvb5ou2SA.jpeg" /></figure><h3>Our mission</h3><p>The job of the project is to find the characteristics of churned users from the behavioral data of these users, and take measures to retain the potential lost users as early as possible.</p><p>The difficulty of the project is that the data handed to the model training should be one item for each person. While our data are behavioral data, we need to extract the user characteristics from the behavioral data, and then hand it to the model for training and get the results. These features will be generated from exploratory data analysis and feature engineering. In the case of poor results, the process may iterate many times until the model performs well.</p><h3>Organisation of the work</h3><blockquote><em>1. </em><strong><em>Data exploration</em></strong><em>: to understand the data, find data distribution and boundaries, find data anomaly and etc.<br>2. </em><strong><em>Data wrangling</em></strong><em>: to clean data, derive data,visualize and summarize data<br>3.</em><strong><em> Feature engineering:</em></strong><em> to define and generate features and label to be used in the model training, create train and test data set.<br>4.</em><strong><em> Model training and evaluation</em></strong><em>: develop a few pilot models, define evaluation metrics for the model. Conduct initial evaluation for the model.<br>5.</em><strong><em> conclusion:</em></strong><em> conduct parametric search to find best hyper-parameters, and conduct model evaluation with final conclusion.</em></blockquote><h3>1. Data exploration</h3><p>In this step, I tried to understand the data ,find data distribution and boundaries and find data anomaly</p><p>to understand the data we must know what information it contains so the following image shows the schema of the data set</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/514/1*PEOyj6HcwzfRuu_WXJ4h3g.png" /></figure><p>the we we must explain what each column of the data set means so :</p><blockquote><strong>- <em>Artist</em></strong><em>: Composer<br>- </em><strong><em>Auth</em></strong><em>: login status<br>- </em><strong><em>firstName</em></strong><em>: first name the model training, create train and test data set.<br>- </em><strong><em>gender</em></strong><em>: genderthe <br>- </em><strong><em>ItemInSession</em></strong><em>: operation sequence number of this session, from small to large according to time<br>- </em><strong><em>lastName</em></strong><em>: surname<br>- </em><strong><em>length</em></strong><em>: the length of the song<br>- </em><strong><em>Level</em></strong><em>: whether the user is paid<br>- </em><strong><em>location</em></strong><em>: the user location<br>- </em><strong><em>method</em></strong><em>: method of getting web pages (put/get)<br>- </em><strong><em>page</em></strong><em>: browse the page overview<br>- </em><strong><em>registration</em></strong><em>: time stamp for registration point in time<br>- </em><strong><em>sessionId</em></strong><em>: sessionId that should be used to determine a single login operation<br>- </em><strong><em>song</em></strong><em>: song name<br>- </em><strong><em>status</em></strong><em>: page return code (200/307/404)<br>- </em><strong><em>ts</em></strong><em>: the timestamp of the log time<br>- </em><strong><em>UserAgent</em></strong><em>: browse the client information</em></blockquote><p><em>A few key findings:</em></p><ul><li><em>There are several kind of a</em>ctions related to potential user churn, namely, Downgrade, Submit Downgrade, Cancel, Cancellation Confirmation. Not all actions leads to churn. Although Cancel has the same count as Cancellation Confirmation, but we need to consider the potential of server communication error rare case. Therefore, Cancellation Confirmation is considered to be the best indicator that user actually churned.</li><li>A few actions seems to provide good insight in user experience and intention: Thumbs Up/Thumbs Down, Add Friend, Add to Playlist, etc. These actions will be investigated further later.</li></ul><p>after that i look for the messing values and the data without a userID:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*05rtak-aasKO8pI2Sg7vcQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/524/1*jwcHRF8kSi_cqwyqb_XZiw.png" /></figure><h3>2-Data Wrangling</h3><p>Based on previous findings, I marked users that had actions of ‘Cancellation Confirmation’ to be ‘is_churn’ = 1. And the users with ‘is_churn’ = 0, are called non-churn users in the rest of post.</p><p>I wonder if gender plays a role in predicting if user could churn or not. The quick plot suggest that Male user has a slight higher tendence to churn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PdeN0ehUf76OO50gkni5Ew.png" /></figure><p>And I wonder if the level plays a role in predicting user Churn .</p><p>With the findings above, I started feature engineering process.</p><p>Some initial thoughts:</p><p>Quantitatively, what information indicates potential user?</p><ul><li>the number of the songs the user likes and dislikes</li><li>the time that user pass on the platform</li><li>the level of each user</li><li>the number of songs the user listen per hour</li><li>the time that a user passed on the system</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/353/1*zupKXy27xt077M_Fylq9bw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/252/1*LHyS5hArXRb5NumCzBJa9Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/1*zCjnIoKc9sXpKC_Lx2ueZw.png" /></figure><h3>3 - Feature Engineering</h3><p>The selection of features is based in part on user characteristics, including gender, registration time, whether paid, and how many singers have been heard.</p><p>Some are based on behavioral data. The selection of behavioral data is mostly based on the session of each user, including the statistical value of listening time of each session, the number of listening songs in each session, and the number of sessions.</p><p>The final count is the number of pages per 100 views.I generated a dataframe for each feature separately, and finally made all features into a dataframe for the final training model by left linking.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t8DX1Z5467zRGpn80yyCYg.png" /></figure><h3>4 - Model training, evaluation and tuning</h3><p>in the session I Split the datasets into train, test, and validation sets ,Setup a pilot model with some initial parameters ,Define model evaluation criteria and Train and evaluate the model</p><p>Model selection:<br>This is a binary classification problem. There are quite a few possible options, including:<br>- LogisticRegression<br>- SVC<br>- Decision Tree</p><p>the following images shows some Model result comparison and discussion</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*A_kFOJl-TNqGZj2aeyf70w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/391/1*mu7FI7shVRtNiaDSm0q_Sg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/383/1*V7sQeysbsKrvaa8fWAxPwQ.png" /></figure><h3>Reflection</h3><p>This project gives exposure to spark environment to analyze a large volume of data that a personal laptop may not be capable to analyze. By identifying customer with high chance to churn prior to the actual losing, companies are able to use minimal cost to save customers by using targeted messages and offers.</p><p>One of the interesting yet difficult things during the project is brainstorming the features that we can derived from the data we have on hand. Developing useful features is crucial to developing a good model, and requires a lot of energy and efforts. Explanatory and exploratory data analysis play important role in this process.</p><h3>Improvement</h3><p>The features can be improved a lot after considering more factors, adding more domain knowledges and expertise. Although the volume of data may required tools such as spark to analyze, but we can use more data to have better results as the user base grow.</p><p>Currently, we have about 450 records of unique users, and we only use 60% of them to train. That said, the model has a huge potential to improve if the sample size increase, and the expected performance will also increase.</p><h3>conclusion</h3><p>As we can see from the result the predictions are enough good. The use of RandomForestClassifier let us predict users’ future actions with high accuracy. The main pain points of this analysis were feature engineering and pipeline creation as they are slightly different with pyspark-ml than with python&amp;scikitlearn.</p><p>This analysis was done in the workspace provided by Udacity and on the data set mini_sparkify_event_data.json of small size and inspired by the materials learned during the lessons. The next is to continue the analysis on the full dataset is of size 12GB in cloud…</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dccfea13035f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Things you should know before visiting Seattle !]]></title>
            <link>https://medium.com/@hider.toula/things-you-should-know-before-visiting-seattle-1d874992ef0c?source=rss-2b2c7b59cf0c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1d874992ef0c</guid>
            <category><![CDATA[travel]]></category>
            <category><![CDATA[udacity]]></category>
            <category><![CDATA[seattle]]></category>
            <category><![CDATA[airbnb]]></category>
            <category><![CDATA[kaggle]]></category>
            <dc:creator><![CDATA[Hider Toula]]></dc:creator>
            <pubDate>Fri, 01 May 2020 22:21:47 GMT</pubDate>
            <atom:updated>2020-05-01T22:21:47.600Z</atom:updated>
            <content:encoded><![CDATA[<h3>Things you should know before visiting Seattle !</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zlXIDXo_xC1jSZLh.jpeg" /></figure><p>Accommodation is the most important thing to find, when travelling to other city or country. Among the things to take into consideration while choosing it ,i will help you in the three following most important of them .</p><ol><li>the best time of year to visit Seattle?</li><li>the best place for accommodation ?</li><li>which type of accommodation to choose ?</li></ol><p>I will try to answer these questions by analyzing publicly accessible <a href="https://www.airbnb.com/"><strong>AirBnB</strong></a> data , available on <a href="https://www.kaggle.com/airbnb/seattle"><strong>Kaggle</strong></a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*krXocQysEImG0bMK.png" /></figure><h3>The best time of year to visit Seattle?</h3><p>when i talk about the best time of the year to visit Seattle, it was of course compared to the prices of accommodation, because yes the price is not the same throughout the year.<br>here a visualization that I made by a port at the average price for each month of 2016.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*BBf3iXwcSBgk62So.png" /></figure><p>My visualization shows that there are periods of the year where accommodation are more expensive than others, and this is due to the holidays and and vacations and as we can notice the average price in January is lower than that in June or July.<br>So if you have a limited amount of money better travel at the start of the year</p><h3>The best place for accommodation ?</h3><p>Yes, as mentioned in the question, the neighborhood plays a role in raising prices. here is a graph that links the price range by district</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cRAmyJmdV8LZIVdq.png" /></figure><p>we can notice in the graph, and this, for all the months of the year that the prices vary from a district to the other, for example the prices in Ballard are less high than the prices in Downtown,<br>with that you can save money because it does not matter the district where you will reside for a month because you will not live there.</p><h3>Which type of accommodation to choose ?</h3><p>It is not only the places and neighborhoods that vary the price, there is also the type of accommodation.<br>yes it’s not the same if you spend your stay in a chalet or in a small apartment</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*e_dlg8zzzAZ1DWgUIiKOlQ.png" /></figure><p>The image above shows the price variance according to the surface area of the accommodation in orange, and in blue the prices according to the number of bedrooms</p><h3>Conclusion</h3><p>This message is not there to tell you to choose the cheapest or largest accommodation. this post is the result of a study I did and that I wanted to share with you in the hope that it will help you if you plan to visit Seattle one day, indeed the purpose of this little article is to help you to organize you according to your needs… .</p><p>for more information you can visit my <a href="https://github.com/hider-toula">git Hub </a>or contact me via <a href="https://www.linkedin.com/in/hider-toula-0268a7174/">linked in</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1d874992ef0c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>