Car Classification with Resnet 50
When we walk down the street, we observe many cars. And some of the people, like car maniacs, can identify cars even by a glance of a look. Some of the car brands are sometimes very distinguishable. For example, the grill of the BMW is a signature design element of BMW cars.
Inspired by that, I was curious, if people can distinguish cars very well, can a machine also do that. I have learned deep learning and image classification from Jovian.ml, therefore, a perfect opportunity to dig deep into. For this multivariate classification problem, I decided to use Car Dataset provided by Standford CS Department. It has total of 16185 images with 196 classes. Out of 16185 images, 8144 images are train dataset and 8041 images are test dataset. Here, I will explain step by step what approach I took to solve this problem.
- Use custom Dataset function to attach labels to the images and load up images.
- DataLoader to load the data in
- Move model and data to GPU
- Train using ReduceLROnPlateau
For first step to workout, the data has been already moved into two different folders: train and test. In my custom dataset, we are going to use classes which is in form of Pandas DataFrame and the image matching up with the class in numerical format. But before that, let’s check how many models are there in the dataset.
As we can see, there are 196 classes in the dataset and each class represents different car models.
We can see that the image is matching up with the class on column 1. Column 2 has been used to distinguish weather the image is for testing or not. If we also take a look at how many testing and training values there are, we can see below that testing set has 8144 images and training set has 8044 images.
Now, let’s define our transformation for training and testing. As the images have the different size, we are going to resize it in to 400by 400 pixels. We are going to apply random rotation by 15° and random horizontal flip. Normalization also applied. For validation set, only resizing and normalization has been applied. Additionally, both sets have been converted into tensor.
This is the custom Dataset CarDatasetLoader . The reason I defined this data loader is because that it is not in shape that ImageFolder requires. What it does is that opens the image up as PIL, convert it into RGB, which are three channels, and return a tuple of image combined with the correct label.
Checking the number of test dataset and train dataset, it perfectly matches with the number we expect it to be. Using batch size of 50, I have used DataLoader.
Let’s take a look at the batch of images from both training and test set.
You can clearly see that for validation dataset, only normalization has been applied. Apart from that, we cannot see any transformation unlike the training dataset. Let’s take a look at individual images.
This is the image on index 50 in training dataset and we can check its model name, which is ‘Spyker C8 Coupe 2009’.
This is the image on index 1624 in training dataset and we can check its model name, which is ‘Ford Freestar Minivan 2007’.
Lastly, we will take a look at the image on index 5000, which has model name of ‘Dodge Durango SUV 2007’.
Note that all the data and the model will be moved to GPU for faster training. Now, I am going to define some helper functions. Firstly, I will show fit function, which directly trains the data and record the result.
This functions trains the model and as an output, it evaluates the model on accuracy and loss function that has been passed from the model, which I will introduce right now.
This class is extending from nn.Module . On top of this class, we are going to put on transfer learning algorithm, which will be in this case Resnet 50. Here, you can see that cross_entropy will be used as a loss function and accuracy as evaluation metrics. The combination of function fit and MultilabelImageClassificationBase prints out the model metrics and train the model. Based on class MultilabelImageClassificationBase , I have put resnet 50 structure on top.
Notice that last layer has been replaced with linear layer so that it outputs 196 classes. Also, we are using cross entropy as a loss function, therefore, we will be able to pick the label that has highest percentage. This is possible as cross entropy has softmax internally.
Get JaeKyung Sung’s stories in your inbox
Join Medium for free to get updates from this writer.
For the accuracy, I obviously defined function for it. This function gets maximum value — as I said previously — and counts how many times did the model get right in one epoch, then it divides by the number of predictions, so that it becomes percentage.
We evaluate initial performance, and we can see that cross entropy loss is around 5.3 and accuracy is around 0.4, as from 0.004, we multiply 100 to it. It is reasonable as if the model is able to get 1 right out of 196 by picking randomly, it is around 0.5 percent, which means that this initial accuracy is reasonable.
Now, let’s train and how it gets better.
We are going to run for 10 epochs and apply weight decay and gradient cliping to prevent any kind of interferance by large gradient value or weight value. We are going to use Adam as optimizer which is known to perform better than SGD. ReduceLROnPlateau reduces learning rate when a metrics are not improving. Using the fit function, we are going to train our model on those conditions. Here, we set patience to 3 and mode on max, meaning that if the metric does not improve in 3 epochs, it is going to reduce the learning rate.
As you can see, over 10 epochs, it has reached the accuracy of around 86%. Let’s plot the graph.
Loss is constantly decreasing and accuracy is therefore keep on increasing, which is a good sign.
Now, let’s predict some images .
The model is indeed able to predict pretty accurately. Therefore, it is now clear that why car maniacs can memorize and identify the cars so easily. There is definitely some pattern that clearly distinguishes between model, therefore, the model successfully learnt. Obviously, there are many room for improvements or different experiment. For example, we can test it by seperating the models by years or makers. Like I showed in the beginning, BMW has its signiture design element, which is its grill. Therefore, there can be varieties of model and application based on the dataset. Additionally, notice that the model got one wrong in prediction. Maybe, it is still hard for this model to predict those have different brand but has simillar shape. Therefore, I could try other losses such as SGD with momentum or SWATS, which is the mixture between SGD and Adam, solving the problem of Adam’s convergence issue .
