Deep Learning Part 6: Into to Object Detection.
Object detection is one of areas that’s exploding and working so much better than just a couple years ago.
In classification, we’ve one dominant instance or in the image that occupies the central image area. Then, our goal is to detect what it is in the image.
Object detection can be seen as an extension of the classification problem. But first of all, let’s get a better intuition about the difference between image classification, localization & object detection.
- Image classification task is to assign a label/ class to a given image. Or in other word, look at a picture and say, “there’s a specific object or not”.
- Localization means not only to label an object, but also to put a “bounding box” around the position of the object to figure out “where does exactly the object locate in the picture”.
- Object Detection is where might be “multiple objects of different categories within a single picture” and we’ve to detect & localize them all.
To clarify, for an autonomous driving application, we might need to detect not just other cars but maybe pedestrians, motorcycles or even other objects.
We’re already familiar with image classification problem, where we might input a picture into a convnet resulting in a vector of features that is fed to maybe a “softmax unit” that outputs the predicted class.
But, how to localize the car in the image as well?!
We need to change our neural network (NN) to have “more output units” in order to estimate “the bounding box”. There’re 4 numbers (b_x , b_y , b_h, b_w) to parameterize the bounding box of the detected object.
Our training set should contain not just the object class label, which our NN is trying to predict up here, but also should contain 4 additional numbers (the bounding box).
what exactly do we mean by those 4 numbers (b_x , b_y , b_h, b_w)?!
Specifying the bounding box (red rectangle) requires:
1- The midpoints (b_x , b_y).
2- The height (b_h) & the width (b_w).
Next, if there’s an object then we’ve to output:
2- b_x , b_y , b_h and b_w (i.e., specifying the bounding box).
3- C1 , C2 and C3 which tells is it the class1, class 2 or class 3 , in other words is it a pedestrian, a car or a motorcycle.
We can use supervised learning to make our algorithm to make that.
In this example, b_x might be 0.5, because of being half way to the right, b_y might be 0.7, as it is 70% of the way down to the image, b_h might be 0.3 because the height of this red square is about 30% of the overall height of the image, and b_w might be 0.4 because the width of the red box is about 4% of the overall width of the entire image.
Defining the target label y
Let’s formalize this a bit more in terms of how we define the target label Y as a supervised learning task. Y is going to be a vector as follows:
y = [ p_c, c_1, c_2, c_3, b_x, b_y, b_h, b_w ], where:
1- The 1st component is p_c, stands for the probability that one of the classes we’re trying to detect exists (p_c will be 1), and if there’s no (No objects to detected), then p_c will be 0. We can think p_c, something other than the background class.
What if there’re no objects in the image?!
if there’s no object in the image (p_c =0) and the rest of the elements of this vector can be any number (e.g., ?). AS “we don’t care” what bounding box nor which of the objects.
Detecting Multiple Objects.
The loss function.
The loss function of object detectors is a combination of 2 individual loss functions: Bounding Box Regression & Cross Entropy Loss.
The former measures how well predicted bounding boxes capture ground truth bounding boxes while the latter measures how good a job the detector did in predicting the correct class1.
It goes from sum of the squares of the difference of the elements, and that’s the loss if y_1 =1 (there is an object in the image), then the loss can be the sum of squares over all the different elements.
Get AI & Data Science 604’s stories in your inbox
Join Medium for free to get updates from this writer.
The other case is if y_1 =0, in that case the loss can be just (^y1 — y1) ². because in that 2nd case all the rest of the components are not important. All we care about is how accurately is the NN to output p_c in that case..
Landmark Detection
In more cases, we could have a NN, which outputs (x, y) coordinates of important points in the image.
To illustrate, suppose we’re building a face recognition application, and for some reason we want the algorithm to tell us “where is the corner of someone’s eye”.
Every point has an (x, y) coordinate. So, we’ve a NN with that outputs 2 numbers to specify the coordinates of a point.
Now, what if we wanted the NN to tell us all 4 corners of the eye.
Then, we can modify the NN to the estimate the position of those 4 points. (x1, y1) for the 1st point, and (x2, y2) for the 2nd point and so on.
To output different positions a person’s face or shape of the mouth to see whether the person is smiling or not. We could define some more numbers, for the sake of argument, let’s say 64 facial landmarks to help us define the edge of the face.
So, what we are going to do is feeding this image as input to go through a convent to some sets of features maybe have it output 0 or 1 (i.e., there a face in this or not) and it also should output (l_{1x}, l_{1y}) and so on to (l_{64x}, l_{64y}).
So, this example would have 129 output units,
- 1 for where a face or not.
- 128 output units for landmarks (64 *2).
This can tell us if there’s a face, as well as where are all facials on the face.
Pose detection
To get a person’s pose detection, we need to define a few key positions (i.e., the midpoint of the chest, left shoulder, left elbow, and wrist and so on). To do that, we need to train a NN to annotate key positions in the person’s pose.
