BEHIND THE SCENES OF IMAGE PROCESSING (4 OF 9)
Image Processing using Python — Blob Detection
In this article, we will embark on an exciting journey of image analysis and delve into the implementation of blob detection and connected components. To illustrate these techniques, we will use an image I captured a few months ago — a photo of my very first attempt at microwaving a hotdog. Our goal is to explore how we can detect the hotdog within the image using these powerful algorithms.
The Challenge: Spotting the Hotdog on the Plate
As we begin this demonstration, we are faced with the challenge of identifying the hotdog amidst the other elements present on the plate. Blob detection provides us with the means to locate regions in the image that differ significantly from their surroundings, enabling us to pinpoint the hotdog’s distinctive features.
To begin, let us import all necessary libraries.
import os
import matplotlib.pyplot as plt
import numpy as np
from math import sqrt
from skimage.io import imread, imshow
from skimage.color import rgb2gray, label2rgb
from skimage.feature import blob_dog, blob_log, blob_doh
from skimage.morphology import erosion, dilation, opening, closing
from skimage.measure import label, regionprops
from skimage.feature import blob_dog, blob_log, blob_doh
from math import sqrt
import matplotlib.pyplot as plt
import numpy as npNow, let us see the image!
hotdog = imread('hotdog.png')
imshow(hotdog);
Will now make the hotdog image into greyscale, since converting the RGB image to grayscale before performing blob detection simplifies the computation, eliminates color variations, and enhances the detection of blob shapes and structures.
hotdog = rgb2gray(imread('hotdog.png'))
imshow(hotdog);
Implementing Blob Detection: Steps to Success
To successfully implement blob detection, we need to follow a series of steps. First, we will preprocess the image by applying appropriate filters and adjustments to enhance the hotdog’s features and suppress background noise. This preprocessing step plays a crucial role in improving the accuracy of blob detection algorithms.
Let us now binarize the image.
im_bw = (hotdog<0.4)*1
im_mask = hotdog < 0.4
imshow(im_bw)
One popular technique we can utilize is the Laplacian of Gaussian (LoG). This method involves taking the Laplacian operator of a Gaussian-smoothed image. The LoG operation enhances the edges and highlights regions of significant intensity variations, making it effective in identifying blobs or objects with distinct boundaries.
blobs = blob_log(im_mask, max_sigma=30, num_sigma=10, threshold=0.1)
fig, ax = plt.subplots()
ax.imshow(im_bw, cmap='gray')
for blob in blobs:
y, x, area = blob
ax.add_patch(plt.Circle((x, y), area*np.sqrt(2), color='r',
fill=False))
plt.show()
Another approach is the Difference of Gaussians (DoG) method. In this technique, we subtract one Gaussian-smoothed image from another. By subtracting the blurred versions of the image, we emphasize the regions with high-frequency content, such as edges and blobs. The resulting DoG image can reveal the presence of blobs, including the hotdog in our case.
blobs = blob_dog(im_mask, max_sigma=30, threshold=0.1)
fig, ax = plt.subplots()
ax.imshow(im_bw, cmap='gray')
for blob in blobs:
y, x, area = blob
ax.add_patch(plt.Circle((x, y), area * np.sqrt(2), color='g', fill=False))
plt.show()
Furthermore, the Determinant of Hessian (DoH) can also be utilized as an alternative approach for blob detection. This technique involves calculating the determinant of the Hessian matrix for each pixel in the image. The Hessian matrix represents the local curvature of the image intensity and the determinant highlights regions with significant curvature changes. By identifying local maxima in the determinant of the Hessian matrix, we can detect and localize blobs effectively.
blobs = blob_doh(im_mask, max_sigma=30, threshold=0.01)
fig, ax = plt.subplots()
ax.imshow(im_bw, cmap='gray')
for blob in blobs:
y, x, radius = blob
ax.add_patch(plt.Circle((x, y), radius, color='b', fill=False))
plt.show()
By incorporating the Laplacian of Gaussian (LoG), Difference of Gaussians (DoG), and Determinant of Hessian (DoH) techniques into our blob detection workflow, we can leverage the power of these algorithms to detect and extract the hotdog region from the image. These methods provide us with valuable tools to analyze and identify objects of interest, opening up a wide range of possibilities for image analysis, computer vision, and beyond.
The Power of Connected Components: Enhancing Hotdog Detection
Get Francis Camarao’s stories in your inbox
Join Medium for free to get updates from this writer.
Connected components analysis comes into play to further refine our hotdog detection. By grouping pixels or regions that are connected or share similar properties, we can segment the image into distinct objects. Applying connected components algorithms to our image will enable us to isolate the hotdog as a separate entity from the plate and other elements. This step enhances the accuracy and reliability of our hotdog detection process.
Let us clean our image
def multi_dil(im, num):
for _ in range(num):
im = dilation(im)
return im
def multi_ero(im, num):
for _ in range(num):
im = erosion(im)
return im
im_cleaned = multi_ero(multi_dil(im_bw, 5), 5)
plt.imshow(im_cleaned, cmap='gray')
plt.show()
Now, let us label it and then get the region properties of our connected components.
label_im = label(im_cleaned)
plt.imshow(label_im)
plt.show()
props = regionprops(label_im)
plt.imshow(props[15].convex_image)
plt.show()
Observing the Results: Unveiling the Hotdog
hotdog_hsv = rgb2hsv(image)
plt.figure(num=None, figsize=(8, 6), dpi=80)
plt.imshow(hotdog_hsv[:,:,0], cmap='hsv')
plt.colorbar()
plt.show()
lower_threshold = 0.90
upper_threshold = 1.00
# Create the mask
mask = (hotdog_hsv[:, :, 0] > lower_threshold) & (
hotdog_hsv[:, :, 0] <= upper_threshold)
# Apply the mask to the tree image channels
red = tree[:, :, 0] * mask
green = tree[:, :, 1] * mask
blue = tree[:, :, 2] * mask
# Create the hotdog mask
hotdog_mask = np.dstack((red, green, blue))
# Display the hotdog mask
plt.figure(figsize=(8, 8))
plt.imshow(hotdog_mask)
plt.show()
hotdog_blobs = label(rgb2gray(hotdog_mask) > 0)
imshow(hotdog_blobs, cmap = 'tab10');
properties =['area','bbox','convex_area','bbox_area', 'solidity',
'major_axis_length', 'minor_axis_length',
'eccentricity']
df = pd.DataFrame(regionprops_table(hotdog_blobs, properties = properties))
dfblob_coordinates = [(row['bbox-0'],row['bbox-1'],
row['bbox-2'],row['bbox-3'] )for
index, row in df.iterrows()]
fig, ax = plt.subplots(1,1, figsize=(8, 6), dpi = 80)
for blob in tqdm(blob_coordinates):
width = blob[3] - blob[1]
height = blob[2] - blob[0]
patch = Rectangle((blob[1],blob[0]), width, height,
edgecolor='r', facecolor='none')
ax.add_patch(patch)
ax.imshow(tree);
ax.set_axis_off()
Sadly, I wasn’t able to implement the blob detection cleanly on this specific image.
Closing Thoughts
Through this practical demonstration, we have explored the implementation of blob detection and connected components to detect a hotdog in an image. While the results are promising, it is important to acknowledge the challenges encountered during the process. Due to the complexity of the image and the variations in lighting, additional preprocessing and fine-tuning of parameters may be required to improve the accuracy of the detection. We must recognize that implementing blob detection and connected components can be a demanding task, and it highlights the need for further exploration and development of alternative processing techniques to overcome the difficulties encountered in this specific image. By continuously refining our approach and exploring new methods, we can unlock the full potential of blob detection and connected components in image analysis and pave the way for more accurate and robust hotdog detection and beyond.
References
Benjur Emmanuel L. Borja. 2023. Image Processing(MSDS2023). Asian Institute of Management.
