<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rahil Bagheri on Medium]]></title>
        <description><![CDATA[Stories by Rahil Bagheri on Medium]]></description>
        <link>https://medium.com/@rahil.bagheri?source=rss-9c5f78d9cb7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*ebXT7FR5sdbqO_bdaXcpww.jpeg</url>
            <title>Stories by Rahil Bagheri on Medium</title>
            <link>https://medium.com/@rahil.bagheri?source=rss-9c5f78d9cb7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 00:45:28 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rahil.bagheri/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deep Learning: Build a dog detector and breed classifier using CNN]]></title>
            <link>https://medium.com/data-science/deep-learning-build-a-dog-detector-and-breed-classifier-using-cnn-f6ea2e5d954a?source=rss-9c5f78d9cb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/f6ea2e5d954a</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[transfer-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[image-classification]]></category>
            <dc:creator><![CDATA[Rahil Bagheri]]></dc:creator>
            <pubDate>Tue, 08 Oct 2019 21:00:19 GMT</pubDate>
            <atom:updated>2019-12-03T17:59:54.709Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Deep Learning: How to build a dog detector and breed classifier using CNN?!</strong></h3><p>This work is part of Udacity’s Data Science capstone project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kYvAvJp1ZvvfCK_RFjR24A.png" /></figure><p><strong>Overview:</strong></p><p>You might think recognizing a dog’s breed in an image is an easy task for you. You are right! It might not be difficult to find dog breed pairs with minimal inter-class variation for instance, Curly-Coated Retrievers and American Water Spaniels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/1*K41mIiqQblFLeAFeRWDE1A.png" /></figure><p>But, how about these two?!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*0BqhyxLOdOqCzJU9bZi3bA.png" /></figure><p>Huh, not that easy!</p><p>In the last decade, it is much easier to use deep learning techniques with a few lines of python code to distinguish between dog breeds in images. In this blog, I will walk you through how to create Convolutional Neural Networks (CNN) from scratch and leverage the latest state of art image classification techniques on ImageNet. This model can be used as part of a mobile or web app for the real world and user-provided images. Given an image to the model, it determines if a dog is present and returns the estimated breed. If the image is human, it will return the most resembling dog breed. You can find the code in my <a href="https://github.com/RahilBA/Dog-Breed-Classifier">GitHub repo</a>.</p><p><strong>Human detector</strong></p><p>I used <a href="https://docs.opencv.org/trunk/db/d28/tutorial_cascade_classifier.html">OpenCV’s</a> implementation of the Haar feature-based cascade object classifier to detect human faces. The cascade function is a machine learning-based approach trained on many images with positive(with a face) and negative(without any face) labels. The detectMultiScale gets the coordinates of all the faces then returns them as a list of rectangles. But don’t forget to convert the RGB image into grayscale before using it.</p><p>The following face_detector function counts up how many human faces are in the photo:</p><pre>def face_detector(img_path):<br>    img = cv2.imread(img_path)<br>    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)<br>    faces = face_cascade.detectMultiScale(gray)<br>    return len(faces) &gt; 0</pre><p>The performance of face_detector evaluated on 100 samples of human and dog images. This detector recognizes all the human faces from human data but didn’t perform well on the dog dataset. It had about 11% false positives.</p><p><strong>Dog detector</strong></p><p>It is time to use another detector that performs better on the dog dataset. I used pre-trained ResNet50 weights on ImageNet in Keras, which is trained on over 10 million images containing 1000 labels.</p><p>In the following code, paths_to_tensor takes the path to an image and returns a 4D tensor ready for ResNet50. But, all the pre-trained models in Keras need additional processing like normalization that can be done by using preprocess_input. The dog_detector function returns “True” if a dog is detected in the image stored at img_path.</p><pre>from keras.preprocessing import image                  <br>from tqdm import tqdm<br>from keras.applications.resnet50 import preprocess_input, decode_predictions</pre><pre>def path_to_tensor(img_path):<br>    img = image.load_img(img_path, target_size=(224, 224)<br>    return np.expand_dims(x, axis=0)</pre><pre>def paths_to_tensor(img_paths):<br>    list_of_tensors = [path_to_tensor(img_path) for img_path in       tqdm(img_paths)]<br>    return np.vstack(list_of_tensors)</pre><pre>def ResNet50_predict_labels(img_path):<br>    img = preprocess_input(path_to_tensor(img_path))<br>    return np.argmax(ResNet50_model.predict(img))</pre><pre>def dog_detector(img_path):<br>    prediction = ResNet50_predict_labels(img_path)<br>    return ((prediction &lt;= 268) &amp; (prediction &gt;= 151))</pre><p>To assess the dog detector, I checked if the predicted class of RestNet50 on ImageNet falls into the dog breed categories. The dog detector performs well without any false negatives.</p><p>Now that we recognized dogs in images, it is time to predict breeds. But first we should analyze train dataset a little more in details:</p><p><strong>Data set</strong></p><p>This data set has 8,351 total images with 133 different breeds. The number of available images for the model to learn from is about 62 per kind, which might not be enough for CNN. In this real-world setting, the images have different resolutions, sizes, lightening conditions, also some images have more than one dog. By comparing the pixel intensity distribution of the same labeled images, I noticed, for example, photos for American_staffordshire_terrier are varying in contrast, size, and brightness. The Red, Green, and Blue channels’ intensity values found in these two images are distributed differently. This variation in data makes the task of assigning breed to dogs even more challenging.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/873/1*Ry3xOPSP9gBJaT1lKmtrqw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*qyNdEuGZpcnUCSczRJCarA.png" /></figure><p>The distribution of breed labels in training data shows data is slightly imbalanced, with, on average, 53 samples per class. Most of the breeds have almost the same distribution in train, valid, and test datasets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*GdWXHYc0epolGC2U3uK_Lg.png" /></figure><p>Also, as part of data processing, I descaled the images by dividing every pixel in every image by 255.</p><p><strong>Metrics</strong></p><p>Since we are dealing with a multi-classification problem here and the data is slightly imbalanced, I used accuracy evaluation metric and categorical_crossentropy cost function. But, first, the labels have to be in a categorical format. The target files are the list of encoded dog labels related to the image with this format. This multi-class log loss punishes the classifier if the predicted probability leads to a different label than the actual and cause higher accuracy. A perfect classifier has a loss of zero and an accuracy of 100%.</p><p><strong>Classify dog breeds</strong></p><p>Here I created a 4-layer CNN in Keras with Relu activation function. The model starts with an input image of 224 *224*3 color channels. This input image is big but very shallow, just R, G, and B. The convolution layers squeeze images by reducing width and height while increasing the depth layer by layer. By adding more filters, the network can learn more significant features in the photos and generalize better.</p><p>My first layer produces an output with 16 feature channels that is used as an input for the next layer. The filters are the collection of 16 square matrices, output feature maps, which are weighted sums of input features and kernel. The kernel’s weights are calculated during the training process by ImageNet data, and what it does is to slide across the input feature maps and produce output features. So, the shape of output features depends on the size of the kernel and input features.</p><p>Check <a href="http://setosa.io/ev/image-kernels/">this page</a> for a better understanding of how CNN works.</p><p>I think it would be ideal for input and output features to have the same size. So, I decided to use same padding to go off the edge of images with zero pads for all the layers with the stride of 2. I also used the max-pooling operation to ensure I am not losing information in the picture while lowering the chance of overfitting. Max pooling takes the maximum of pixels around a location.<br>After four Convolutional layers and max-pooling, followed by two fully connected layers, I trained the classifier. The Convolutional layers extract the image features, and the classifier classifies them based on the previously obtained features. The image below shows how the sequence of feature blocks and classifier on top transfer the information from the raw image and predict requested target values.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EI4ph4Wt0J7dQFfbuDFw0g.png" /></figure><p>The CNN model architecture is:</p><pre>from keras.layers import Conv2D, MaxPooling2D, GlobalAveragePooling2D<br>from keras.layers import Dropout, Flatten, Dense<br>from keras.models import Sequential</pre><pre># Model Architecture<br>model = Sequential()</pre><pre>model.add(Conv2D(filters=16, kernel_size=2, padding=&#39;same&#39;, activation=&#39;relu&#39;, input_shape=(224,224,3)))<br>model.add(MaxPooling2D(pool_size=2))</pre><pre><br>model.add(Conv2D(filters=32, kernel_size=2 , padding=&#39;same&#39; , activation=&#39;relu&#39;))<br>model.add(MaxPooling2D(pool_size=2))</pre><pre>model.add(Conv2D(filters=64 , kernel_size=2 , padding=&#39;same&#39; , activation=&#39;relu&#39;))<br>model.add(MaxPooling2D(pool_size=2))<br>model.add(Dropout(0.4))</pre><pre>model.add(Conv2D(filters=128 , kernel_size=2 , padding=&#39;same&#39; , activation=&#39;relu&#39;))<br>model.add(MaxPooling2D(pool_size=2))<br>model.add(Dropout(0.4))</pre><pre>model.add(Flatten())<br>model.add(Dense(512, activation=&#39;relu&#39;))<br>model.add(Dropout(0.4))<br>model.add(Dense(133,activation=&#39;softmax&#39;))</pre><pre>model.summary()</pre><p>The created model from scratch is not performing well, accuracy 12%, due to not having enough images data to train the model. One potential improvement is data augmentation to add more data. This strategy modifies the images by padding, cropping, and rotating images randomly. It also enables the model to generalize better without overfitting, of course, with the appropriate parameters.</p><p>As it was mentioned before, training a CNN classifier made from scratch on a small data like this will lead to underfitting and with so many layers, and parameter tuning often causes overfitting. So, it is time to utilize transfer learning pre-trained networks to create a CNN breed classifier even though these models are not explicitly made for this task. But one advantage of these networks is that they are trained on large datasets, ImageNet with millions of labeled images, and reached the 90% accuracy. Also, they can generalize to other images outside the ImageNet.</p><p>I implemented the pre-trained models like VGG, Inception V3, and ResNet in Keras and compared their accuracy. I used them as fixed feature extractors and fine-tuned the fully-connected layer by minimizing the cross-entropy loss function using stochastic gradient descent, the learning rate of 0.001 and Nesterov momentum. Also, the last fully connected layer is changed to the number of dog breeds, 133. Then, each model was trained on 25 epochs with 20 samples each batch.</p><pre># model Architecture<br>Xception_model = Sequential()<br>Xception_model.add(GlobalAveragePooling2D(input_shape=(train_Xception.shape[1:])))<br>Xception_model.add(Dense(133, activation=&#39;softmax&#39;))</pre><pre>checkpointer = ModelCheckpoint(filepath=&#39;saved_models/weights.best.Xception.hdf5&#39;, verbose = 0, save_best_only=True)</pre><pre>sgd = SGD(lr= 1e-3 , decay=1e-6, momentum=0.9 , nesterov = True)</pre><pre># compile the model<br>Xception_model.compile(loss=&#39;categorical_crossentropy&#39;, optimizer=sgd, metrics=[&#39;accuracy&#39;])</pre><pre># Training the model <br>Xception_model.fit(train_Xception , train_targets, <br>               validation_data = (valid_Xception, valid_targets),<br>               shuffle = True,<br>               batch_size = 20,<br>               epochs = 25,<br>               verbose = 1)</pre><p>The Xception model outperforms all the other models with an accuracy of 86% and a loss of 0.4723 on test data. In general, Inception based models slightly outperform VGG and ResNet on ImageNet; also, it is more computationally efficient. The model performance could be improved by applying more appropriate fine-tuning techniques. For example, training a few top layers related to specific dog features and freeze the others that detect more generic features. Because they are already captured in the ImageNet weights.</p><p><strong>Result</strong></p><p>Now, it is time to combine the detectors and Xception-predict-breed, also, test them on several images with different sizes and resolutions. As mentioned before, accuracy was chosen as a model evaluation metric for this classifier. This code takes a path to an image and first determines whether the image is a dog, human or neither then returns the predicted resembling breed:</p><pre>def Xception_predict_breed (img_path):<br>    bottleneck_feature = extract_Xception(path_to_tensor(img_path)) <br>    predicted_vector = Xception_model.predict(bottleneck_feature) <br>    return dog_names[np.argmax(predicted_vector)]</pre><pre>def display_img(img_path):<br>    img = cv2.imread(img_path)<br>    cv_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)<br>    imgplot = plt.imshow(cv_rgb)<br>    return imgplot</pre><pre>def breed_identifier(img_path):<br>    display_img(img_path)<br>    prediction = Xception_predict_breed(img_path)<br>    if dog_detector(img_path) == True:<br>        print(&#39;picture is a dog&#39;)<br>        return print (f&quot;This dog is a {prediction}\n&quot;)<br>    <br>    if face_detector(img_path) == True:<br>        print(&#39;picture is a human&#39;)<br>        return print (f&quot;This person looks like a {prediction}\n&quot;)<br>    <br>    else:<br>        return print(&#39;The picture is neither dog nor human&#39;)</pre><p>The result shows that the model is performing well. For example, it could predict American_staffordshire_terrier in test data while it was trained on images with different brightness, sizes, and zoom level.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hTA1PwsXXSvlhMNQfbgizA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zWVOpbSCoSTF4do5DUmlrw.png" /></figure><p>And the following is the result on human images:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/757/1*HueovBEtZJioXdKh7Eh_zw.png" /></figure><p>The confusion matrix of the predictions on the test data is given below. But, it is not easy to visualize which breeds are most misclassified.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*GgtkzAOe4z7F83DdKbkCQw.png" /></figure><p>For a better understanding of misclassified cases, I paired the predicted and actual values. The model couldn’t classify between Great_pyrenees and Kuvasz, which both are white, big, and fluffy. It also couldn’t distinguish between Mastiff and Bullmastiff, which is a mix between a bulldog and a mastiff. By adding more data, the model would be able to learn more specific features and distinguish between these similar breeds, which is even confusing for humans.</p><p><strong>Potential improvements</strong></p><p>The model accuracy could improve by data augmentation and adding more training labeled data since the model was slightly imbalanced. With acquiring more data, the Convnet models would be more capable of learning the more relevant and specific dog features from the images. But, it requires a considerable amount of time and memory usage. Another possible way to enhance the model accuracy is averaging the predictions of several networks to get the final prediction known as ensemble techniques. As I mentioned before, one improvement could be applying more appropriate fine-tune strategies to adjust the weights based on the data. Also, a proper analysis of the final model would be to add some noise to the images and see how it is performing.</p><p><strong>Conclusion</strong></p><p>In this work, I have implemented the haar cascade classifier and ResNet50 pre-trained weights to detect human faces and dogs. I also spent some time exploring the data to find images characteristics, which helped me to apply more appropriate techniques. For dog breed classification, I have started with a few numbers of Convolutional layers as a baseline network. The model performed better than a random guess, 1 in 133, but still underfits.</p><p>With the given time and memory power, I decided to leverage several transfer learning architectures to enhance the CNN performance, which resulted in a significant improvement. I used the pre-trained models as fixed features extractor and changed the global average pooling layer, fully connected layer, and applied other fine-tuning approaches. The Xception and ResNet50 produced higher accuracy than others, around 86%. It is quite challenging to get a higher accuracy due to the complexity of these kinds of datasets with some similarity between mixed breeds.</p><p>In the end, I tested the model on the same labeled photos with different intensity distributions, and it performed quite well.</p><p>The most exciting part of this project was how modern ConvNets with a few lines of codes made it easy for us to do the tasks that even humans find challenging. But it is quite hard to improve the model with the given time and memory power. I plan to explore the mentioned improvement approaches and will update this blog with better results.</p><p>please feel free to use <a href="https://github.com/RahilBA/Dog-Breed-Classifier">my code</a> if you found it interesting.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f6ea2e5d954a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/deep-learning-build-a-dog-detector-and-breed-classifier-using-cnn-f6ea2e5d954a">Deep Learning: Build a dog detector and breed classifier using CNN</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using data to help Seattle Airbnb hosts and guests make smarter decisions:]]></title>
            <link>https://medium.com/data-science/using-data-to-help-seattle-airbnb-hosts-and-guests-make-smarter-decisions-39749cc43ae?source=rss-9c5f78d9cb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/39749cc43ae</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[prophet]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[airbnb-seattle-open-data]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <dc:creator><![CDATA[Rahil Bagheri]]></dc:creator>
            <pubDate>Fri, 06 Sep 2019 12:47:19 GMT</pubDate>
            <atom:updated>2022-09-23T15:38:31.924Z</atom:updated>
            <content:encoded><![CDATA[<h3>Why to help Seattle Airbnb hosts and guests make smarter decisions?!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*H6GAE4R6GM70b6sp" /><figcaption>Photo by <a href="https://unsplash.com/@phoebezzf?utm_source=medium&amp;utm_medium=referral">Zhifei Zhou</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>The first question that always comes to mind before traveling anywhere is accommodation. In the last eight years, Airbnb proved it is the most convenient and cheapest solution to this kind of travel concerns. It also allows homeowners to increase their income by sharing their place.</p><p><strong>Motivation:</strong></p><p>Airbnb provides an excellent solution to short term accommodations. But it is time to use data and guide the new hosts to estimate more reasonable prices while maximizing their revenue. Also, help guests to know which accommodation factors they should pay more attention to save some money. The Seattle Airbnb open data can be found <a href="https://www.kaggle.com/airbnb/seattle">here</a>.</p><p>I tried to focus on answering questions that might help both sides since every single of us can be both a host or guest. Based on Airbnb pricing tips, about 60% of the hosts’ income come from the property factors and amenities, and the other 40% comes from the demand throughout the year. So, I decided to apply Machine Learning and Time Series techniques to analyze these key factors; price, revenue, and demand.</p><p><strong>What are the key drivers of higher rates?</strong></p><p>The average listing price for 2016 was about $137 per night. But it varied through different months and weekdays. It even reaches close to 160 during summer. Seattle is a cold place, and it makes sense to be more expensive during summer due to higher demands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/635/1*7a1LDWVbTjFIqf3rbkA4xg.png" /></figure><p>It also seems like the listed places on Seattle Airbnb during weekends are more expensive than other weekdays.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*0sKnEW4178VVwSex1MBD3Q.png" /></figure><p>So, it is statistically significant that time and market are the main drivers and should be considered as a regressor in Machine Learning models. Houses and apartments were the most listed and wanted properties but with reasonable prices, less than $150 per night. But, some features like boat, loft, condo, and townhouses were expensive and rarely listed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sd0lm_mWYkPdFEY__dOAqQ.png" /></figure><p>To identify the drivers of higher ratings, I cleaned, scaled, and formatted the data ready for predictive analysis, also removed the variables that have similar performances, multicollinearity. Then used Random Forest regressor to find most significant listing features to explain what caused the price range variation.</p><p>Random Forest selects the critical features by measuring two performance metrics, decrease node impurity, and increase accuracy. All the possible price drivers are given scores. The higher the weight is, the more significant the effect of listing feature is differentiating price values. The details about how Random Forest permute the values of each element is beyond the scope of this project.</p><p>First, I fitted the Random Forest model and then enhanced it by tuning the model hyperparameters to reach the highest accuracy.</p><p>The result of this proposed solution shows that the following features were contributed to making the critical decision in the tress and can explain 90% of the variation in price range:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/1*TU-A8nkGzE6J2jtfv1gnAg.png" /></figure><p>Essential elements in this regressor were number of bedrooms, number of bathrooms, number of beds, number of included guests, number of reviews, season, neighborhood, cancellation fee, and policy strict.</p><p>In general, these features can be categorized into three broad categories; one is the properties features, previous reviews, and seasonal and weekly demand which I analyze more in the next part.</p><p><strong>How the Airbnb demand and price rates changed over 2016?</strong></p><p>As I mentioned before, the average price from Jan 2016 to Jan 2017 in Seattle shows the market was an essential factor in the price range. During summer, higher demand leads to higher rates and towards the beginning of 2016 and 2017 sharp drop.</p><p>I decomposed daily price series to determine the rates trend and seasonality and understand time-based patterns which are critical for Airbnb business. Also, another fundamental objective of this part is to investigate if it is possible to determine a forecasting model that shows the future pattern using 2016 data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/541/1*19ZPsQ56oVpVG9EMYApPMQ.png" /></figure><p>The average price has a seasonality pattern and is not stationary. To check if the data met the stationary requirement for time series forecasting, I split the data into two partitions and compared the mean and variance of each group. Since the mean and variance are quite different, and the difference is statistically significant; I log-transformed the series to make it stationary.</p><p>The autocorrelation plot shows price at each lag of time; it is highly correlated with its past values and might be predictable by using time series techniques. But instead of classical statistical time series methods, I decided to use Prophet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*47y1UOYyur3ZrW_qPVuPSQ.png" /></figure><p>Prophet is a procedure introduced by Facebook for forecasting time series data based on an additive model where nonlinear trends are fit with different seasonality plus holiday effects. You can read more about Prophet <a href="https://facebook.github.io/prophet/">here</a>. Also, Prophet has easy interpretable parameters that can be tuned to improve trend and seasonality prediction.</p><p>Here I trained an additive regression model with a linear trend, weekly, daily and yearly seasonality, considering holiday effects. In general, I assumed my time series has three main components: trend, seasonality, and holidays.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*Jty5ljQKE1I8JUzp4eMipQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*sQjECfL7zreiRWjbKbavCg.png" /></figure><p>The components plot shows that weekends, summer season and holiday events have a positive effect on increasing prices. Also, the average daily price will increase from 137 to more than 150 in 2017.</p><p>In the following graph, the blue line represents the predicted values while the black dots represent the actual daily values . From the plot, it seems that the model can catch the seasonality and fit historical data very well, but I suggest adding 2017 and 2018 datasets, if available to validate model forecasting capability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*A7h-0O2j3BtaOWXupiA_5Q.png" /></figure><p>The number of trends changepoints and uncertainty intervals in Prophet can be adjusted to overcome overfitting and underfitting.</p><p><strong>Which listings are more profitable and lead to more revenue?</strong></p><p>To answer this question, I defined a metric that measures revenue since there was not such a data point in the original dataset. I assumed the host’s income is estimated by the price per night times minimum times the place was booked. I grouped all the listed properties to two groups, the ones that performed better and caused revenue more than $10000 per year, and the ones that showed the opposite. This threshold can be changed based on how the average income distributed. Then I compared the specific features of top and low listing performance.</p><p>The result shows apartments and houses caused more revenue but just not due to a high price, but because they are booked the most.</p><p>The following bar chart compares the top and low performers 5 number summary between different properties.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/659/1*IhLyopLJsRPAZ-b4smkNhw.png" /></figure><p>To identify which factors caused a listing more profitable compared to others, I applied different classifiers Gradient Boosted, Logistic Regression, Support Vector Machine, and K nearest neighbor. Then I tuned the Gradient Boosted Classifier hyperparameters to reach more accuracy and overcome overfitting. About 83% of the listings are categorized, having revenue less than $10000, and 17% made more in 2016. So, I applied resampling techniques, like SMOTE, to be sure the classifier can capture all unusual listings. The feature important plot represents that if a host becomes a super host, respond between 1 or a couple of hours and increase the capacity of fitting more people would make more money than others.</p><p>Out of 520 low performer listings about 466 predicted accurately by this classifier. The area under the ROC curve shows the model accuracy in classifying these two groups for a new dataset is 89%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*cOsV9Y0RjmwSG1SXq4fwVA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*jwAWzSvXq8rRzZ2fbu0nKA.png" /></figure><p>This model can be improved by defining a more resealable trade-off between precision and recall. Also, another revenue metric or threshold can be used to enhance the evaluation metrics.</p><p>In the end, I want to note that this project is part of Udacity’s Data Science Nanodegree course. If you are interested in reviewing the code or improving it, the link to my Github is available <a href="https://github.com/RahilBA/Seattle-Airbnb-Open-Data">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=39749cc43ae" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/using-data-to-help-seattle-airbnb-hosts-and-guests-make-smarter-decisions-39749cc43ae">Using data to help Seattle Airbnb hosts and guests make smarter decisions:</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>