<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Abhishek Das on Medium]]></title>
        <description><![CDATA[Stories by Abhishek Das on Medium]]></description>
        <link>https://medium.com/@iamabhishekdas?source=rss-3ea76459494a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*jo3or1ILHZZoxGF3WuNzog.jpeg</url>
            <title>Stories by Abhishek Das on Medium</title>
            <link>https://medium.com/@iamabhishekdas?source=rss-3ea76459494a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 14:25:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@iamabhishekdas/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to build a Convolutional Neural Network from scratch using tf.keras for Image Classification]]></title>
            <link>https://medium.com/data-science/how-to-build-a-convolutional-neural-network-from-scratch-using-tf-keras-for-image-classification-ee4482de8453?source=rss-3ea76459494a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ee4482de8453</guid>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[image-classification]]></category>
            <category><![CDATA[convolutional-network]]></category>
            <dc:creator><![CDATA[Abhishek Das]]></dc:creator>
            <pubDate>Sun, 17 Jan 2021 15:36:49 GMT</pubDate>
            <atom:updated>2021-01-17T21:20:25.097Z</atom:updated>
            <content:encoded><![CDATA[<h4>A hands-on tutorial for image classification in Keras</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hIcLZbhl8bO8GqaJvGSHMg.png" /><figcaption>Visualization of the CIFAR-10 classes by author</figcaption></figure><blockquote>In this article, we will build a Convolutional Neural Network layer-by-layer using the Functional API of tf.keras. Next, we will explore and integrate the Data Augmentation techniques provided by ImageDataGenerator class in Keras.</blockquote><p><strong><em>What is tf.keras?</em></strong></p><ul><li><a href="https://www.tensorflow.org/api_docs/python/tf/keras"><strong>tf.keras</strong></a> is the TensorFlow’s implementation of the Keras API.</li><li>Keras requires a backend to train custom neural networks. It used <a href="https://github.com/Theano/Theano">Theano </a>as its default backend, before switching to <a href="https://www.tensorflow.org/">TensorFlow</a> starting from v1.1.0.</li></ul><p>This tutorial is also available as a Google Colab notebook for a hands-on experience.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/84930ddf9f8e0700b816f54446b6717a/href">https://medium.com/media/84930ddf9f8e0700b816f54446b6717a/href</a></iframe><p>We will use the <a href="https://www.cs.toronto.edu/~kriz/cifar.html"><strong>CIFAR-10</strong></a> dataset, which is a popular baseline for image classification tasks. The CIFAR-10 dataset is a collection of 60000 RGB images of size 32x32. It comprises of 10 classes, with 6000 images per class. There are 50000 images available for training, and 10000 test images.</p><p><em>Let&#39;s Begin….</em></p><p>Importing the necessary libraries-</p><pre>import numpy as np<br>from tensorflow.keras import *<br>from keras.preprocessing.image import ImageDataGenerator<br>import matplotlib.pyplot as plt<br>import seaborn as sns</pre><h3>1. Data Preparation</h3><p>Downloading CIFAR-10 Dataset-</p><pre>(X, y), (test_X, test_y) = datasets.cifar10.load_data()</pre><p>Out of the 50000 images available for training, we will create a validation set using 3000 images and use the rest 47000 images for training.</p><pre>train_X = X[:47000]<br>train_y = y[:47000]<br>val_X = X[-3000:]<br>val_y = y[-3000:]</pre><h3>2. Model Architecture</h3><p>We will use the TensorFlow <a href="https://www.tensorflow.org/guide/keras/functional"><strong>Functional API</strong></a> since it gives us more control and access to each layers compared to Sequential API.</p><p><strong>(Conv2D → BatchNorm → Conv2D → BatchNorm → Maxpooling2D)* 3 → (Dense → BatchNorm)*3 →Softmax</strong></p><p>The layers which we will use-</p><ul><li><a href="https://www.tensorflow.org/api_docs/python/tf/keras/Input"><strong>Input()</strong></a><strong> </strong>instantiates a symbolic tensor object</li><li><a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D"><strong>Conv2D()</strong></a> creates a kernel which is convolved with the layer input. We will use it to perform a Spatial Convolution over images.</li><li><a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization"><strong>BatchNormalization()</strong></a><strong> </strong>layer<strong>, </strong>during training uses the mean and std of the current batch of inputs, while during inference uses a moving average of the mean and std of the batches seen during training.</li><li><strong>MaxPooling2D()</strong> is used for the pooling operation for 2D spatial data</li><li><a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Flatten"><strong>Flatten()</strong></a> layer simply flattens the input so that it can be fed to a Dense layer</li><li><a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense"><strong>Dense()</strong></a> layer is a regular NN layer, with options available for adding an activation and regularization within the method itself.</li><li><a href="https://www.tensorflow.org/api_docs/python/tf/keras/activations/softmax"><strong>softmax()</strong></a> applies the softmax activation function</li></ul><p>Selecting appropriate kernel size, stride and padding are important parameters to design a Convolutional Neural Network. The <a href="https://cs231n.github.io/convolutional-networks/"><strong>CS231n</strong></a> guide by Stanford University provides a beautiful intuition behind choosing appropriate values for these parameters and explains how to determine the output dimensions after each layer.</p><pre>input = Input(shape=(32, 32, 3))</pre><pre>x = layers.Conv2D(32, (3,3), activation=&#39;relu&#39;,padding=&#39;same&#39;)(input)</pre><pre>x = layers.BatchNormalization()(x)<br>x = layers.Conv2D(64, (3,3), activation=&#39;relu&#39;,padding=&#39;same&#39;)(x)<br>x = layers.BatchNormalization()(x)<br>x = layers.MaxPooling2D(2)(x)</pre><pre>x = layers.Conv2D(128, (5,5), activation=&#39;relu&#39;,padding=&#39;same&#39;)(x)<br>x = layers.BatchNormalization()(x)<br>x = layers.Conv2D(256, (5,5), activation=&#39;relu&#39;,padding=&#39;same&#39;)(x)<br>x = layers.BatchNormalization()(x)</pre><pre>x = layers.Conv2D(512, (3,3), activation=&#39;relu&#39;,padding=&#39;same&#39;)(x)<br>x = layers.BatchNormalization()(x)<br>x = layers.Conv2D(512, (5,5), activation=&#39;relu&#39;,padding=&#39;same&#39;)(x)<br>x = layers.BatchNormalization()(x)<br>x = layers.MaxPooling2D(2)(x)</pre><pre>x = layers.Flatten()(x)<br>x = layers.Dense(1024, activation=&#39;relu&#39;)(x)<br>x = layers.BatchNormalization()(x)<br>x = layers.Dense(512, activation=&#39;relu&#39;)(x)<br>x = layers.BatchNormalization()(x)<br>x = layers.Dense(128, activation=&#39;relu&#39;)(x)<br>x= layers.Dense(10)(x)</pre><pre>output= activations.softmax(x)</pre><pre>model = Model(inputs=input, outputs=output,name=&quot;TF_Functional_API&quot;)</pre><pre>model.summary()</pre><p>The number of parameters involved with each layer and the dimension of the layer outputs can be viewed with the <a href="https://www.tensorflow.org/api_docs/python/tf/keras/Model#summary"><strong>summary</strong></a><strong>()</strong> method.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ae8KRKn4aF7XGvWffn7WIA.png" /></figure><p>The 3 primary methods used in Keras for model training and evaluation are <strong><em>compile</em></strong>, <strong><em>fit,</em></strong> and <strong><em>evaluate</em></strong>. TensorFlow provides a <a href="https://www.tensorflow.org/api_docs/python/tf/keras/Model#methods_2"><strong>comprehensive guide</strong></a> for each of these methods.</p><p><em>Some Hyperparameters -</em></p><ul><li>Loss Function → <a href="https://www.tensorflow.org/api_docs/python/tf/keras/losses/SparseCategoricalCrossentropy">SparseCategoricalCrossEntropy</a></li><li>Optimizer → <a href="https://keras.io/api/optimizers/adam/">Adam</a></li><li>Learning Rate → 0.001</li><li>Batch size → 64</li><li>Epochs → 30</li></ul><h3>3. Running our model (without Data Augmentation)</h3><pre><strong>model.compile</strong>(loss=losses.SparseCategoricalCrossentropy(),optimizer=optimizers.Adam(learning_rate=0.001),metrics=[&#39;accuracy&#39;])</pre><pre><strong>model.fit</strong>(train_X, train_y.flatten(), batch_size=64, epochs=30, validation_data=(val_X,val_y.flatten()))</pre><pre><strong>model.evaluate</strong>(test_X, test_y.flatten(), verbose=2)</pre><p>The model gives an accuracy of 81.59 % on the test set.</p><p>The fit method returns a ‘History’ object where the attribute ‘History.history’ is a dictionary of the training and validation metrics (loss and accuracy) over the epochs. Thus it maintains a log which can be used to visualize these metrics as we will see later.</p><h3>4. Data Augmentation using ImageDataGenerator class</h3><p>Now, we will see how using Data Augmentation techniques can boost the accuracy of our model. Data Augmentation helps in preventing <em>overfitting</em> and helps the neural network model to generalize better on unseen variations of test images. Normalization, horizontal flipping, minor rotation, resizing are the most frequently used augmentation techniques.</p><p>We will use the <a href="https://keras.io/api/preprocessing/image/"><strong>ImageDataGenerator</strong></a> class in Keras. It provides a range of transformations for the input images which we can incorporate easily into our training.</p><pre>datagen_train=ImageDataGenerator(featurewise_center=True,<br>                                featurewise_std_normalization=True, <br>                                rotation_range=20,<br>                                width_shift_range=0.2,<br>                                height_shift_range=0.2,<br>                                horizontal_flip=True)</pre><pre>% For testing and validation set, we just normalize the images<br>datagen_test=ImageDataGenerator(featurewise_center=True,<br>                               featurewise_std_normalization=True)</pre><pre>datagen_train.fit(train_X)<br>datagen_test.fit(val_X)<br>datagen_test.fit(test_X)</pre><p>Train and evaluate-</p><pre>model.compile(loss=losses.SparseCategoricalCrossentropy(),optimizer=optimizers.Adam(learning_rate=0.001),metrics=[&#39;accuracy&#39;])</pre><pre>model.fit(datagen_train.flow(train_X, train_y.flatten(), batch_size=64), steps_per_epoch=len(train_X) / 64, epochs=30, validation_data=datagen_test.flow(val_X, val_y.flatten(), batch_size=64))</pre><pre>model.evaluate(datagen_test.flow(test_X, test_y.flatten(), batch_size=1), verbose=2)</pre><p>We reach a test accuracy of <strong>88.30 %</strong>, already achieving a boost of ~7%</p><h3>5. Results</h3><p>Here are the visualizations of how the training progressed-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*Y9KkRXO0Ktahacr7BrIaBg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/509/1*7lq3lPJaP_nLG7CTKqpkiw.png" /></figure><h3>6. Conclusion</h3><ul><li>We saw how a general image classification pipeline looks like, starting from data preparation and visualization, model designing, hyperparameter selection, training, and evaluation.</li><li>Keras API in TensorFlow, especially the Functional API makes it very convenient for an user to design a Neural Network.</li><li>Introducing just data augmentation in training provided the model a boost of ~7 % in test accuracy, thus demonstrating the benefits of the technique as discussed.</li><li>We have used CIFAR-10 dataset for demonstration, however, this template can be extended to any image dataset.</li><li><a href="https://www.pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/"><strong>Here</strong></a> is an article which provides an extensive discussion on the key differences between Keras and tf.keras</li></ul><blockquote>The codes are available <a href="https://gist.github.com/Abhishek0697/147bcbde2eb7a8fba1841169a6435aa4"><strong>here</strong></a><strong> </strong>for further development.</blockquote><h3>7. Further exploration</h3><p>Some tricks to improve accuracy:</p><ol><li>Using architectures with skip connection, like <a href="https://arxiv.org/abs/1512.03385"><strong>ResNet</strong></a>, that have proven to perform well for Images.</li><li>We can use <strong>Transfer learning</strong>. for eg. Load a model pretrained on larger and related datasets like <a href="http://www.image-net.org/"><strong>ImageNet</strong> </a>and then fine-tune on our dataset. A pretrained CIFAR-100 model could be more beneficial on CIFAR-10 dataset because of the similarity in size and classes.</li><li>Tweaking the layers and hyperparameters used. For eg. An SGD optimizer, higher batch size, changing filter size and number of filters in each layer.</li><li>Use <a href="https://keras.io/api/optimizers/learning_rate_schedules/"><strong>Learning Rate Scheduling</strong></a><strong> </strong>to decay the learning rate with epochs as the model learns.</li><li>Use regularization techniques like <a href="https://keras.io/api/layers/regularization_layers/dropout/"><strong>Dropout</strong></a><strong> </strong>and advanced strategies like <a href="https://arxiv.org/abs/1905.04899"><strong>CutMix</strong></a><strong>, </strong><a href="https://www.pyimagesearch.com/2019/12/30/label-smoothing-with-keras-tensorflow-and-deep-learning/"><strong>Label Smoothing</strong></a><strong>, </strong><a href="https://arxiv.org/pdf/1710.09412.pdf"><strong>Mixup training</strong></a><strong> </strong>etc.</li></ol><blockquote>Note:</blockquote><blockquote>We should note that the ResNet models are originally designed for images of much larger size. i.e. 224x224 from ImageNet. Therefore, it uses MaxPooling operations and higher convolution strides to reduce the dimension of images between layers. Thus, for CIFAR-10 images, we should Implement the model layers ourselves and change the filter size and stride values accordingly.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ee4482de8453" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-to-build-a-convolutional-neural-network-from-scratch-using-tf-keras-for-image-classification-ee4482de8453">How to build a Convolutional Neural Network from scratch using tf.keras for Image Classification</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kaggle — Predict survival on the Titanic challenge in MATLAB]]></title>
            <link>https://medium.com/data-science/kaggle-predict-survival-on-the-titanic-challenge-in-matlab-56f6ad3bab78?source=rss-3ea76459494a------2</link>
            <guid isPermaLink="false">https://medium.com/p/56f6ad3bab78</guid>
            <category><![CDATA[matlab]]></category>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Abhishek Das]]></dc:creator>
            <pubDate>Wed, 13 Jan 2021 06:54:49 GMT</pubDate>
            <atom:updated>2021-01-16T03:32:57.314Z</atom:updated>
            <content:encoded><![CDATA[<h3>Kaggle — Predict survival on the Titanic challenge in MATLAB</h3><h4>In this tutorial, I will demonstrate how to use MATLAB to solve this popular Machine Learning challenge</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9JWQTzQXXa83a3mSK7VEcw.png" /><figcaption>Photo of the RMS Titanic departing Southampton on April 10, 1912 by F.G.O. Stuart, <a href="https://commons.wikimedia.org/wiki/RMS_Titanic#/media/File:Titanic_in_color.png">Public Domain</a></figcaption></figure><p>The objective of this <a href="https://www.kaggle.com/c/titanic/overview">Kaggle</a> challenge is to create a Machine Learning model which is able to predict the survival of a passenger on the Titanic, given their features like age, sex, fare, ticket class etc.</p><p>The outline of this tutorial is as follows:</p><ol><li>Data Analysis</li><li>Feature Engineering</li><li>Model Fitting</li><li>Prediction on the test set</li><li>Conclusion</li><li>Future Directions</li></ol><blockquote><strong>Why MATLAB?</strong></blockquote><blockquote>In the past few months, I have explored the Machine Learning Toolbox in MATLAB for solving various tasks like Regression, Clustering, PCA etc. I liked its modular interface and that every function is well documented with examples, which becomes very handy. Thus, in this post, I have shared some of my findings , which I hope is helpful to the academic community.</blockquote><p><em>Let’s begin …</em></p><h3>1. Exploratory Data Analysis</h3><p>Reading the dataset</p><pre>&gt;&gt; Titanic_table = readtable(&#39;train.csv&#39;);<br>&gt;&gt; Titanic_data = (table2cell(Titanic_table));</pre><p>The train set has 891 passenger entries and 12 columns.</p><p>Now, lets take a look into our data. The <a href="https://www.mathworks.com/help/matlab/ref/table.head.html"><strong>head </strong></a>function displays the top rows of a table, similar to that in Pandas.</p><pre>&gt;&gt; head(Titanic_table)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UNbEKi52-vkLU-OMfGBtuQ.png" /></figure><p>The ‘Survived’ column is our binary <em>target</em> variable which we need to predict; where 0- Not Survived, 1- Survived.</p><p>The predictor variables i.e. features are as follows-</p><ul><li>Pclass: Ticket class</li><li>Sex</li><li>Age</li><li>SibSp: number of siblings, spouses along with the passenger</li><li>Parch: number of parents, children along with the passenger</li><li>Ticket: Ticket number</li><li>Fare</li><li>Cabin: Cabin number</li><li>Embarked: Port of Embarkation</li></ul><p>‘Pclass’ variable is indicative of the socio-economic status. It has 3 possible values ‘1’, ‘2’ , ‘3’ representing the Upper, Middle and Lower class respectively.</p><blockquote>The probability of survival for passengers in the training set is around <strong>38 %</strong></blockquote><p>Now, Let’s visualize survival based on different features-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*w1Ex75SAnCiygicQYwHX6w.png" /></figure><ul><li>There are 577 males and 314 female passengers.</li><li>Females have a much higher survival probability of <strong>74.20 %</strong> as compared to males which is <strong>18.90 %</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*oXxBn0l9RyBcDXavOUqK4Q.png" /></figure><ul><li>We see that the passengers with a higher ticket class are more likely to have survived.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/1*IotUVA9h6bQxRWVNqcbFbg.png" /></figure><ul><li>Children have a very high survival rate, while the rate drops to around <strong>10 %</strong> for people above 65 years of age.</li><li>The initial statistics are in lines with our intuition that the strategy must have been to save the lives of women and children first.</li></ul><h3>2. Feature Engineering</h3><p>To transform the ‘<strong><em>sex</em></strong><em>’</em> feature, I have used <a href="https://www.mathworks.com/help/stats/grp2idx.html"><strong>grp2idx</strong></a> to convert the categorical values ‘male’ and ‘female’ to numerical indices ‘1’ and ‘2’ respectively.</p><p>The transformation of the ‘<strong>age</strong>’ feature has 2 steps-</p><ul><li>Filling missing entries-</li></ul><pre>% Check number of missing entries in the &#39;age&#39; feature</pre><pre>&gt;&gt; sum(isnan(age))<br>ans = <br>     177</pre><p>We will impute the missing entries in the ‘age’ feature with the <strong><em>mean</em></strong> age of all other passengers. i.e. 29.61 years. There are some other ways in which we can handle these missing entries, discussed in section 6.</p><ul><li>Categorizing into bins-</li></ul><p>MATLAB provides a <a href="https://www.mathworks.com/help/matlab/ref/double.discretize.html"><strong>discretize</strong></a> method which can be used to group data into bins or categories. I have used this method for categorizing age into the following 4 age groups — ‘Under 15’, ‘15–30’, ‘30–65’ and ‘Above 65’.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c698c17840f38d407d131154e84e2bfd/href">https://medium.com/media/c698c17840f38d407d131154e84e2bfd/href</a></iframe><h3>3. Model Fitting</h3><p>We will see 4 classifiers namely-</p><ul><li>Logistic Regression</li><li>Decision Trees</li><li>K Nearest Neighbor</li><li>Random Forest</li></ul><p><em>Let’s begin with…</em></p><h4>Logistic Regression</h4><p>In MATLAB, we can implement a logistic regression model using the <a href="https://www.mathworks.com/help/stats/fitglm.html"><strong>fitglm </strong></a>method.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6e3e119d656b81ece6f80d85f1a799cb/href">https://medium.com/media/6e3e119d656b81ece6f80d85f1a799cb/href</a></iframe><h4>Decision Trees</h4><p><a href="https://www.mathworks.com/help/stats/fitctree.html"><strong>fitctree</strong></a><strong> </strong>function<strong> </strong>returns a fitted binary classification decision tree for a given set of predictor and response variables. We can visualize our decision tree using the view method, thus providing an easy interpretation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*Wjmv7baTu6ZwTSeYS5a-FQ.png" /></figure><p><em>Pruning</em> decision trees is an efficient strategy to combat overfitting. I have used the <a href="https://www.mathworks.com/help/stats/classificationtree.cvloss.html"><strong>cvloss</strong></a> function to find the ‘BestLevel’ for pruning. The <a href="https://www.mathworks.com/help/stats/classificationtree.prune.html"><strong>prune</strong></a><strong> </strong>function returns the tree pruned to this level.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c69e849b1a1b584f0c0fc1eb20d3b807/href">https://medium.com/media/c69e849b1a1b584f0c0fc1eb20d3b807/href</a></iframe><h4>K Nearest Neighbors</h4><p>The <a href="https://www.mathworks.com/help/stats/classificationknn.fit.html"><strong>ClassificationKNN.fit</strong></a><strong> </strong>function is used to fit a <em>k</em>-nearest neighbor classifier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*zWjW8h06oFXm7Kbsar8ceQ.png" /></figure><ul><li>I have performed a search to find the optimal number of neighbors (N).</li><li>The above graph shows that N=17 gives the lowest Cross Validation Loss. Thus, we may select N=17. However, for N=5 and N=11, we get similar loss. So, we can experiment with N=5, 11, and 17 keeping in mind the model complexity and the chances of overfitting with a large value of N.</li><li>MATLAB has a range of distance metric to choose from. for e.g. ‘cityblock’, ‘chebychev’, ‘hamming’, etc. The default is the ‘euclidean’ distance metric.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/26ebe22cd81c5b0221db19110962fe01/href">https://medium.com/media/26ebe22cd81c5b0221db19110962fe01/href</a></iframe><h4>Random Forest Classifier</h4><p>Individual decision trees tend to overfit. The random forest approach is a bagging method where deep trees, fitted on bootstrap samples, are combined to produce an output with lower variance.</p><p>We will use the <a href="http://mathworks.com/help/stats/treebagger.html"><strong>TreeBagger</strong></a> class to implement a random forest classifier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*49Bnc9gGrPAdbF2z4iq18Q.png" /></figure><ul><li>The above plot shows the out-of-bag classification error as we increase the number of trees in consideration. This allows us to select the optimal number of decision trees to create an ensemble.</li><li>We can visualize the importance of each feature using the <em>OOBPermutedPredictorDeltaError</em><strong><em> </em></strong>property.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*AabInMV1tKrx0V0MpayUZg.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3707611fd356703c0f931b713770f40a/href">https://medium.com/media/3707611fd356703c0f931b713770f40a/href</a></iframe><p>The training accuracies for our models are as follows-</p><ul><li>Logistic Regression: 78.34 %</li><li>Decision Trees: 80.47 %</li><li>K Nearest Neighbor: 79.80 %</li><li>Random Forest: 78.11 %</li></ul><p>We see that the pruned decision tree gives the highest accuracy in this case, however it is prone to overfitting. Let’s find out the accuracy on the test set.</p><h3>4. Generate Test Predictions</h3><p>The test set comprises of 418 passenger entries and 11 columns without the ‘Survived’ variable, which we will predict using the trained models.</p><pre>&gt;&gt; head(Titanic_table_test)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*orv-l-ZCAVpP6mFFD8rctQ.png" /></figure><p>We apply the same feature engineering steps on the test dataset and feed it through the models described above to generate the predictions. Finally, we create a .csv file for submission to Kaggle. The <a href="https://gist.github.com/953f29d900194fe01c7eefc57a5d5bee.git">snippet</a> shows an example for the same.</p><p>Decision Tree gives the highest accuracy of <strong>78.947 %</strong> on the test set.</p><h3>5. Conclusion</h3><p>We saw a step-by-step process in MATLAB to solve a Machine Learning task starting from visualizing the patterns in the dataset, selecting and engineering features, training multiple classifiers to performing prediction using the trained models.</p><p>Some observations-</p><ul><li>Increasing number of bins in the ‘age’ feature led to overfitting.</li><li>Adding a new feature calculated as (fare/ticket frequency) did not give a significant improvement in the test accuracy.</li></ul><blockquote>The codes are available for future development <a href="https://github.com/Abhishek0697/Kaggle---Predict-survival-on-the-Titanic-challenge-in-MATLAB">here</a>.</blockquote><h3>6. Further Exploration</h3><ul><li>Other strategies can be applied to handle the <strong>missing entries</strong> in the ‘age’ feature such as using <em>Median</em> age, or <em>Family relations</em>. for e.g. A child’s age can be approximated to be say (age of father-25).</li><li>Including more combination of features is naturally the next step which I look forward to analyze.</li><li>One can <strong>engineer new features</strong> based on initial data statistics and intuition. <a href="https://www.kaggle.com/cdeotte/titanic-wcg-xgboost-0-84688/notebook">Here </a>is an interesting analysis to read.</li><li>Since we have multiple models, model ensembles is worth giving a try.</li></ul><p>There are many such ideas discussed on this challenge by the Kaggle community. This <a href="https://www.kaggle.com/pliptor/how-am-i-doing-with-my-score/report">notebook</a> provides a great summary for the same.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56f6ad3bab78" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/kaggle-predict-survival-on-the-titanic-challenge-in-matlab-56f6ad3bab78">Kaggle — Predict survival on the Titanic challenge in MATLAB</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>