Performing Face Recognition using KNN

10 min readMay 8, 2023

Introduction

Facial recognition serves as a vital security purpose. It can be seen everywhere. It can be found in the security camera in front of your house and the sensor on your phone. Moreover, the face is a crucial entity for human identity. It distinguishes a person, so face recognition is an important technique. Face recognition systems are used to unlock software on mobile devices. However, face recognition struggles to perform under certain conditions, such as poor lighting, long hair, sunglasses, or when other objects partially cover the face.

Face recognition is a technology that involves the identification of an individual’s face using various techniques, including machine learning algorithms. K-Nearest Neighbors (KNN) is one of the popular algorithms used in face recognition. It’s a supervised learning approach that is used for classification purposes. It operates by locating the nearest neighbors of a data point and classifying it according to the majority of its neighbors. In face recognition, KNN uses the distance between facial feature points to classify a face.

In this article, we will implement a basic form of face recognition using the Haar cascades classifier and k- Nearest Neighbors algorithm.

What is Haar cascade?

It is a machine learning-based object detection algorithm used to find faces in an image or real-time video. The algorithm works by comparing rectangular features at different positions in an image and at different scales. The research article “Rapid Object Detection using a Boosted Cascade of Simple Features,” written by Paul Viola and Michael Jones in 2001, contains edge or line detection features that are used by the technique. The algorithm is given a large number of positive photos with faces and a large number of negative images without any faces to train.

Why are we using KNN for face recognition?

KNN algorithm uses the neighbourhood classification as the predictive value of a good instance value. In KNN, the input data is classified based on the closest k data points. For face recognition, KNN can be used to compare the facial features of an input image to those of the previously stored image. KNN is also simple to implement and computationally efficient. Also, the experimental results demonstrate that KNN has a greater capacity to recognize frontal faces that aren’t covered, with a success rate of 90%.

Getting started

We will be using VS Code for this project, and we will be running two (2) Python scripts;

Script for adding a new face
Script for live face recognition

Script for adding a new face

import os
import numpy as np
import cv2
import pickle

face_data = []
i = 0

camera = cv2.VideoCapture(0)

facecascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

name = input('Enter your name --> ')
ret = True


while(ret):
    ret, frame = camera.read()
    if ret == True:
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

        face_coordinates = facecascade.detectMultiScale(gray, 1.3, 4)

        for (a, b, w, h) in face_coordinates:
            faces = frame[b:b+h, a:a+w, :]
            resized_faces = cv2.resize(faces, (50, 50))
            
            if i % 10 == 0 and len(face_data) < 10:
                face_data.append(resized_faces)
            cv2.rectangle(frame, (a, b), (a+w, b+h), (255, 0, 0), 2)
        i += 1

        cv2.imshow('frames', frame)

        if cv2.waitKey(1) == 27 or len(face_data) >= 10:
            break
    else:
        print('error')
        break

cv2.destroyAllWindows()
camera.release()


face_data = np.asarray(face_data)
face_data = face_data.reshape(10, -1)

if 'names.pkl' not in os.listdir('data/'):
    names = [name]*10
    with open('data/names.pkl', 'wb') as file:
        pickle.dump(names, file)
else:
    with open('data/names.pkl', 'rb') as file:
        names = pickle.load(file)

    names = names + [name]*10
    with open('data/names.pkl', 'wb') as file:
        pickle.dump(names, file)


if 'faces.pkl' not in os.listdir('data/'):
    with open('data/faces.pkl', 'wb') as w:
        pickle.dump(face_data, w)
else:
    with open('data/faces.pkl', 'rb') as w:
        faces = pickle.load(w)

    faces = np.append(faces, face_data, axis=0)
    with open('data/faces.pkl', 'wb') as w:
        pickle.dump(faces, w)

Let’s go over what the code block above does, line by line.

Line 1–4: Importing libraries

import os
import numpy as np
import cv2
import pickle

Here we import the numpy library because it is used to represent images as arrays, perform operations on these arrays, and manipulate feature vectors extracted from face images. We import the OS library because it provides a way to interact with the operating system in a platform-independent manner. We import the cv2 library because it is used to perform tasks such as face detection and alignment. Also, we import the pickle library because it is used to save and load trained models.

Line 6–7: Initializing variables.

face_data = []
i = 0

Line 9: Here, we create a video capture object to access the webcam. Argument 0 is passed because we want to use the inbuilt webcam of our Laptop. Use 1 if you are using your external camera.

camera = cv2.VideoCapture(0)

Line 11: Creating a Haar cascade object to detect faces in the frame.

facecascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

Line 13: Asking for the name of the person adding their face.

name = input('Enter your name --> ')

Line 14: Set ret = True (To start the infinite loop).

ret = True

The ret variable is generally used in face recognition projects to show if the picture stream is being read correctly or not. It is important to verify that the frames are being read correctly when reading an image from a camera, as improper frame reading can result in errors with the face recognition algorithm. Therefore, when we set ret = True, we indicate that the image stream is being read correctly and that the frames are being captured successfully. If ret is set to False, it means that there was an error in reading the image, and the face recognition algorithm may not work as expected.

Let’s begin the loop for facial recognition using KNN.

while(ret):
    ret, frame = camera.read()
    if ret == True:
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

        face_coordinates = facecascade.detectMultiScale(gray, 1.3, 4)

        for (a, b, w, h) in face_coordinates:
            faces = frame[b:b+h, a:a+w, :]
            resized_faces = cv2.resize(faces, (50, 50))
            
            if i % 10 == 0 and len(face_data) < 10:
                face_data.append(resized_faces)
            cv2.rectangle(frame, (a, b), (a+w, b+h), (255, 0, 0), 2)
        i += 1

        cv2.imshow('frames', frame)

        if cv2.waitKey(1) == 27 or len(face_data) >= 10:
            break
    else:
        print('error')
        break

cv2.destroyAllWindows()
camera.release()


face_data = np.asarray(face_data)
face_data = face_data.reshape(10, -1)

if 'names.pkl' not in os.listdir('data/'):
    names = [name]*10
    with open('data/names.pkl', 'wb') as file:
        pickle.dump(names, file)
else:
    with open('data/names.pkl', 'rb') as file:
        names = pickle.load(file)

    names = names + [name]*10
    with open('data/names.pkl', 'wb') as file:
        pickle.dump(names, file)


if 'faces.pkl' not in os.listdir('data/'):
    with open('data/faces.pkl', 'wb') as w:
        pickle.dump(face_data, w)
else:
    with open('data/faces.pkl', 'rb') as w:
        faces = pickle.load(w)

    faces = np.append(faces, face_data, axis=0)
    with open('data/faces.pkl', 'wb') as w:
        pickle.dump(faces, w)

This code reads the image from the camera and detects faces in each frame using the Haar cascade classifier provided by OpenCV. It does this by looping over the frames while ret is true, indicating that the video stream is being read correctly. For each frame, it converts the image to grayscale, detects the face coordinates using the cascade classifier, crops the face region, resizes it to 50x50 pixels, and then adds the resized face to a list face_data every 10 frames. Once 10 faces have been collected, or if the user presses the ‘Esc’ key, the program ends the loop and saves the collected face data in the face_data list as a NumPy array. The names.pkl and faces.pkl files are then checked to see if they exist. If they do not exist, new files are created and the name variable is saved to names.pkl and the face data is saved to faces.pkl. If the files already exist, the code loads the existing data, appends the new face data and names, and saves them back to the files. This creates a dataset of face images and corresponding names that is used for the project.

Let’s see some output:

Get Omale Happiness’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

After running the code on your terminal, something like this will come up for you to save your name.

Image by Author

Immediately after that, your camera comes on to save your face. It will take your image 10 times, and the camera will turn off.

Let’s try saving another person’s face: Remember to rerun the code to save a new name and face.

Image by Author

Script for live face recognition using knn

import cv2
from sklearn.neighbors import KNeighborsClassifier
import pickle
import numpy as np

facecascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

with open('data/faces.pkl', 'rb') as w:
    faces = pickle.load(w)

with open('data/names.pkl', 'rb') as file:
    labels = pickle.load(file)

camera = cv2.VideoCapture(0)

print('Shape of Faces matrix --> ', faces.shape)
knn = KNeighborsClassifier(n_neighbors=4)
knn.fit(faces,labels)

while True:
    ret, frame = camera.read()
    if ret == True:
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        face_coordinates = facecascade.detectMultiScale(gray, 1.3, 5)

        for (a, b, w, h) in face_coordinates:
            fc = frame[b:b + h, a:a + w, :]
            r = cv2.resize(fc, (50, 50)).flatten().reshape(1,-1)
            text = knn.predict(r)
            cv2.putText(frame, text[0], (a, b-10), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 0), 2)
            cv2.rectangle(frame, (a, b), (a + w, b + w), (0, 0, 255), 2)

        cv2.imshow('livetime face recognition', frame)
        if cv2.waitKey(1) == 27:
            break
    else:
        print("error")
        break

cv2.destroyAllWindows()
camera.release()

Let’ go over what the code block above does, line-by-line.

Line 1–4: Importing required libraries.

import cv2
from sklearn.neighbors import KNeighborsClassifier
import pickle
import numpy as np

Here we import the numpy library because it is used to represent images as arrays and perform operations on these arrays and to manipulate feature vectors extracted from face images.

We import OS library because it provides a way to interact with the operating system in a platform independent manner.

We import the cv2 library because it is used to perform the actual tasks such as face detection and face alignment.

Also, we import the the pickle library because it is used to save and load trained models, such as face recognition classifiers.

Line 6–16:

facecascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

with open('data/faces.pkl', 'rb') as w:
    faces = pickle.load(w)

with open('data/names.pkl', 'rb') as file:
    labels = pickle.load(file)

camera = cv2.VideoCapture(0)

print('Shape of Faces matrix --> ', faces.shape)

In the code above, the Haar cascade classifier for face detection is loaded using ‘cv2.CascadeClassifier()’ function, and the path of the classifier XML file is provided as an argument. The Haar cascade classifier is a pre-trained classifier provided by OpenCV that can detect faces in an image or video stream.

Next, the code loads the previously saved face data and corresponding labels (names) from the ‘faces.pkl’ and ‘names.pkl’ files using the ‘pickle.load()’ function. These files were previously created by the code that collected the face data from the camera. The face data is stored in a NumPy array called ‘faces’, and the labels (names) are stored in a list called labels.

Finally, the code initializes the camera using ‘cv2.VideoCapture()’ function with argument 0, which means it will use the default camera. It then prints the shape of the faces matrix to check the size of the face data that was previously collected.

Check the shape of the faces data/matrix.

Image by Author

Line 17–42:

knn = KNeighborsClassifier(n_neighbors=4)
knn.fit(faces,labels)

while True:
    ret, frame = camera.read()
    if ret == True:
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        face_coordinates = facecascade.detectMultiScale(gray, 1.3, 5)

        for (a, b, w, h) in face_coordinates:
            fc = frame[b:b + h, a:a + w, :]
            r = cv2.resize(fc, (50, 50)).flatten().reshape(1,-1)
            text = knn.predict(r)
            cv2.putText(frame, text[0], (a, b-10), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 0), 2)
            cv2.rectangle(frame, (a, b), (a + w, b + w), (0, 0, 255), 2)

        cv2.imshow('livetime face recognition', frame)
        if cv2.waitKey(1) == 27:
            break
    else:
        print("error")
        break

cv2.destroyAllWindows()
camera.release()

In the code above, a K-Nearest Neighbors (KNN) classifier is trained using the previously collected face data (faces) and corresponding labels (labels). The classifier is initialized with ‘n_neighbors=4’ as the number of neighbors to consider in the classification.

Then, the code reads frames from the camera using ‘cv2.VideoCapture()’ function, converts each frame to grayscale using ‘cv2.cvtColor()’ function, and uses the Haar cascade classifier to detect faces in the frame. For each detected face, the code resizes the face to 50x50 pixels using ‘cv2.resize()’, flattens it using ‘flatten()’, and reshapes it to a 2D array using ‘reshape(1, -1)’. The flattened and reshaped face is then passed to the KNN classifier’s ‘predict()’ method to predict the label of the face.

The predicted label is displayed on the frame using ‘cv2.putText()’ and a rectangle is drawn around the detected face using ‘cv2.rectangle()’. The frame with the predicted labels and rectangles is displayed on the screen using ‘cv2.imshow()’.

The code runs in an infinite loop until the user presses the ‘Esc’ key ‘(cv2.waitKey(1) == 27)’, after which the code destroys all windows and releases the camera using ‘cv2.destroyALLWindows()’ and ‘camera.release()’ functions, respectively.

In summary, this code performs live face recognition using the previously trained KNN classifier and the Haar cascade classifier to detect faces in the image stream from the camera and displays the predicted labels and rectangles around the detected faces in real-time.

Now let’s see the result for the Faces saved

I used my sister’s face and my face.

To get more accurate results, we did it together.

You can always add more faces to improve the result.

Conclusion

K-Nearest Neighbors (KNN) is a powerful algorithm used for face recognition due to its simplicity, effectiveness, and flexibility.

Its ability to learn from training data and accurately classify unknown instances makes it a popular choice for face recognition tasks. It allows easy customization and parameter tuning to optimize performance for specific applications.

KNN is also robust to noise and can handle large amounts of data, making it suited for real-world applications.

You can view the complete code for this article here