Canvas-based JavaScript Blur detection
Introduction
While developing the Know Your Customer (KYC) module in the banking sector, we faced a specific challenge. KYC is a process where banks verify the identity and assess the risk profile of their clients. As part of this module’s development, we needed a JavaScript library that was capable of analysing images — specifically, we required it to detect if an image is blurred and, importantly, to determine the degree of that blur. This functionality was crucial because clear images are essential for accurate identity verification in KYC processes so a dedicated library was needed. Unfortunately, there were no suitable solutions in opensource that fulfilled criteria satisfactorily.
Left with no alternative, we embarked on creating an algorithm on our own. The outcome was a user-friendly algorithm that effortlessly checks any image for the presence and degree of blur. From the voluminous inquiries we observed on platforms like StackOverflow, it became evident that our solution could greatly benefit the community.
This utility has significant potential for diverse applications, such as image quality analysis on the client side. Yet, surprisingly, a modern, uncomplicated, resource-friendly library to tackle this common problem remains absent in today’s software landscape. Our experience, therefore, not only serves as a case study, but also underscores the urgent need for such a library in the JavaScript/TypeScript ecosystem.
Relevance
In an era where the visual web dominates, the quality of images plays a significant role in user engagement and experience. Blurry images can tarnish user perceptions and limit the effectiveness of visual communication. Yet, our research has shown a glaring gap in the JavaScript/TypeScript ecosystem: the lack of a simple, accessible tool to detect and quantify image blur. This absence impacts developers, businesses, and users alike, leaving a critical need unmet.
In the digital age, the quality of visuals is paramount across a broad spectrum of platforms. From e-commerce sites, where product images are pivotal in influencing purchasing decisions, to online galleries showcasing artists’ portfolios, news portals highlighting photojournalism, and even social media platforms where the clarity of a shared image can dictate its virality. Consider for example a retailer on Aliexpress. Take a look at the images below. It is impossible to determine how the product looks and what is written in the explanation to it because images are blurry.
Even a minor degradation in an image can turn potential buyers away. Although there are some guidelines for file formats and sizes, maintaining the intrinsic quality of an image, ensuring aspects such as clarity, remains a daunting task. Among various determinants of image quality, blur stands out as a predominant issue, impacting both amateur snapshots and occasionally even professionally curated images.
Addressing this glaring need, the proposed open-source blur detection tool for the JavaScript/TypeScript ecosystem goes beyond just identifying blurs. It aims to provide real-time feedback to users like shop owners, photographers, or journalists, prompting immediate rectification. Such a tool not only ensures that images meet the highest visual standards but also enhances user trust and engagement, reinforcing the idea that in the modern digital landscape, the clarity of an image is often synonymous with its value. The goal of the proposed blur detector is to introduce automated quality checks in the browser. This could provide feedback to users and potentially prompt them to upload images of higher quality.
Current solutions
This part spotlights other solutions and a crucial need in the software landscape: an accessible and efficient blur-detection library in JavaScript/TypeScript.
1. OpenCV approach
Solutions such as js-blur-image-detection, based on the heavy OpenCV library makes it an impractical choice for many applications for the multiple reasons:
Performance Overhead with OpenCV:
- Size: OpenCV is a comprehensive computer vision library. While this means it’s feature-rich, it also means it’s heavy. For web applications, using such a bulky library can significantly increase the size of your bundled JavaScript, leading to longer load times.
- Complexity: OpenCV offers a myriad of functionalities, many of which may not be needed for a simple blur detection task. This can add unnecessary complexity to the application.
Deployment Challenges:
- Browser Compatibility: OpenCV, given its origins and extensive functionality, might not be entirely optimised for all browsers. This can lead to potential compatibility issues.
- Version Maintenance: With a complex library like OpenCV, staying updated with the latest versions or dealing with potential security vulnerabilities can become a challenge.
Runtime Efficiency:
- Processing Time: Due to its comprehensive nature, operations in OpenCV might be more resource-intensive, potentially leading to slower runtimes, especially on less powerful devices (eg. mobile devices) or in browsers with multiple tabs and operations.
Dependency Concerns:
- External Dependencies: Relying on large external libraries can sometimes introduce bugs or conflicts with other packages, making debugging more challenging.
On the other hand, a zero-dependency JS/TS solution offers:
- Lightweight & Fast: With a dedicated focus on blur detection, a zero-dependency solution would be optimised for that sole task, ensuring swift performance without unnecessary overhead.
- Simplified Maintenance: A specialised JS/TS library would likely have fewer updates, and when updates do occur, they would be directly relevant to the blur detection task.
- Better Integration & Compatibility: Being designed specifically for the JS/TS ecosystem ensures better compatibility with other JS libraries and frameworks. It’s also more likely to be optimised for a wide range of browsers.
- Enhanced Security: Fewer dependencies mean a reduced surface area for potential security vulnerabilities.
In summary, while OpenCV-based solutions like the one mentioned provide a vast array of features, they might be akin to using a sledgehammer to crack a nut when it comes to specific tasks like blur detection in web applications. A zero-dependency JS/TS solution, tailored for this particular problem, offers a more efficient, maintainable, and user-friendly approach.
2. Plain JS approach
The best solution found so far is inspector-bokeh: Canvas-based blur detection with JavaScript. The approach here is based on this article. The provided algorithm showcases a detailed method for blur detection in images. It begins with a preprocessing step to blur the image slightly with the intention to enhance edge detection. This is followed by the utilisation of a Sobel kernel convolution to detect edges in the luminance-converted image. The core of the blur detection leverages these edges to calculate the average edge width, which is then used to infer the level of blur in the image. Some points of consideration:
- Typescript Compatibility: This algorithm, while detailed, does not cater to TypeScript-specific applications. Absence of type annotations and interfaces makes its direct adoption into TypeScript projects challenging.
- Usability & Dependency: The algorithm has a dependency on canvasfilters, which is sourced from an arbitrary GitHub repository. This presents two primary concerns: firstly, the potential for usability issues if the library is not well-maintained or lacks comprehensive documentation. Secondly, there’s a security risk associated with depending on unverified repositories.
- Size-Dependent Metric: The algorithm’s blur metric, which takes the average edge width as a proportion of the image’s width (avgEdgeWidth / width * 100), may yield inconsistent results for images of varying dimensions. This relationship could result in misleading blur assessments for either very large or small images.
- Complexity: The algorithm, though comprehensive, presents a steep learning curve with its multiple stages of processing and a myriad of parameters. Its intricacy could pose challenges for developers seeking a plug-and-play solution.
In conclusion, this algorithm offers a methodical approach to blur detection. However, issues related to TypeScript compatibility, dependency clarity, and a questionable size-driven metric, compounded by its inherent complexity, could limit its applicability and accuracy in diverse scenarios.
Proposed solution
The solution is a lean and focused library designed to tackle one specific problem: analysing an image and returning a quantified blur score. This library takes a minimalist approach, intentionally avoiding dependencies, style exports, or unnecessary frameworks. What it does export is a simple TypeScript function, embodying the essence of “do one thing and do it well”.
Get Eremingeorgy’s stories in your inbox
Join Medium for free to get updates from this writer.
Within this package, all the necessary mathematical computations are internalised, thus keeping the interface clean and straightforward. Most importantly, the underlying blur detection algorithm has been carefully crafted and thoroughly vetted by our data science team. This ensures that not only is it effective and reliable, but it’s also optimised to deliver accurate results swiftly. This library, therefore, represents a streamlined, specialised solution to the image blur detection problem, designed to integrate seamlessly into modern development workflows.
Approach description
Like many blur detection algorithms, this solution is fundamentally based on an edge detection algorithm. The process begins by passing the input image through a sequence of filters. Initially, the image is blurred to reduce high-frequency noise. Subsequently, it is transformed into grayscale to simplify the image structure, which is then convolved using the Laplacian aperture, further enhancing the image’s features.
To expedite the blur detection process, an image reduction technique is employed to decrease the number of pixels in the image without compromising on the critical details. Finally, the principal algorithm is executed, which locates the edges within the processed image. The density and clarity of these edges form the basis of the blur score returned by our function.
This series of operations forms a robust and efficient pipeline for blur detection. Our library, therefore, stands as a streamlined and effective tool for image analysis, offering an accurate blur score as a measure of image clarity.
Helper functions
While developing this blur-detection solution, I chose not to use external libraries for image filtering due to the lack of quality TypeScript libraries. Instead, it was decided to craft custom helper functions, to reduce library’s size and complexity. This choice results in a streamlined, performance-optimised library that could be easily integrated into other projects.
1. Grayscalling
Images often use the RGBA format: Red, Green, Blue, and Alpha (for transparency). If an image is grayscaled then Red, Green, and Blue values for any pixel will be the same. Storing all three identical values is redundant. The code provided optimises this by keeping only one channel — in essence, transforming the image into a grayscale representation. The implementation of the algorithm is shown below.
export const greyScale = (pixels: ImageData): ImageData => {
const output = createImageData(pixels.width, pixels.height)
const dst = output.data
const d = pixels.data
for (let i = 0; i < d.length; i += 4) {
const r = d[i]
const g = d[i + 1]
const b = d[i + 2]
// CIE luminance for the RGB
const v = 0.299 * r + 0.587 * g + 0.114 * b
dst[i] = v
dst[i + 1] = v
dst[i + 2] = v
}
return output
}2. Reducing Image data for efficiency
When working with image data, especially in the context of web applications, it’s often crucial to reduce memory usage and improve processing speed. One common approach is to reduce image data. In the given code snippet, it is achieved by transforming an image with RGBA channels into a single-channel grayscale representation.
const reducedPixels = (imageData: ImageData): Uint8ClampedArray[] => {
const { data: pixels, width } = imageData
const rowLen = width * 4
let i
let x
let y
let row
const rows = []
for (y = 0; y < pixels.length; y += rowLen) {
row = new Uint8ClampedArray(imageData.width)
x = 0
for (i = y; i < y + rowLen; i += 4) {
row[x] = pixels[i]
x += 1
}
rows.push(row)
}
return rows
}3. Initial edge smoothing
A Gaussian blur is then applied to smooth out any high-frequency noise that could disrupt edge detection. The implementation of the algorithm is shown below.
export const gaussianBlur = (pixels: ImageData, diameter: number): ImageData => {
diameter = Math.abs(diameter)
if (diameter <= 1) return pixels
const radius = diameter / 2
const len = Math.ceil(diameter) + (1 - (Math.ceil(diameter) % 2))
const weights = getFloat32Array(len)
const rho = (radius + 0.5) / 3
const gaussianFactor = 1 / Math.sqrt(2 * Math.PI * rho * rho)
const rhoFactor = -1 / (2 * rho * rho)
let wsum = 0
const middle = Math.floor(len / 2)
for (let i = 0; i < len; i++) {
const x = i - middle
const gx = gaussianFactor * Math.exp(x * x * rhoFactor)
weights[i] = gx
wsum += gx
}
for (let i = 0; i < weights.length; i++) {
weights[i] /= wsum
}
return separableConvolve(pixels, weights, weights)
}4. Edge enhancement
The final step uses a Laplacian aperture to enhance the image’s edges, readying it for the core edge-detection process in our blur algorithm. These functions help the tool remain compact, reliable, and independent, providing accurate blur scores without the need for external dependencies.
Core algorithm
Blur check is done by calculating a variance matrix of Laplacian operator applied to the input. The implementation follows that post.
Steps:
- Remove noise by applying a Gaussian blur and then convert the original image to grayscale
- Apply a Laplacian operator to the grayscale image
That produces a matrix of second derivatives and variance of second derivatives will be a blurriness score. If we want to find blurry images we should adjust the threshold, e.g. 5. Of course threshold adjustment should be done on real data.
The basic edge analysis algorithm is given below.
const detectBlur = (pixels: Uint8ClampedArray[]) => {
const width = pixels[0].length
const height = pixels.length
const getMean = () => {
let sum = 0
for (let i = 0; i < height; i++) {
for (let j = 0; j < width; j++) {
sum += pixels[i][j]
}
}
return sum / (width * height)
}
const getVariance = (mean: number) => {
let sum = 0
for (let i = 0; i < height; i++) {
for (let j = 0; j < width; j++) {
const normalized = pixels[i][j] - mean
sum += normalized * normalized
}
}
return sum / (width * height)
}
return getVariance(getMean())
}In order to determine the blur score using the approach described above, the code below could be used.
const BLUR_BEFORE_EDGE_DETECTION_DIAMETER = 3.0 // pixels
const LAPLACIAN_APERTURE = [0, 1, 0, 1, -4, 1, 0, 1, 0]
const prepareImage = (imageData: ImageData) => {
const preBlurredImageData = gaussianBlur(imageData, BLUR_BEFORE_EDGE_DETECTION_DIAMETER)
const greyScaled = greyScale(preBlurredImageData)
return convolve(greyScaled, LAPLACIAN_APERTURE)
}
export const measureBlur = (imageData: ImageData): number =>
detectBlur(reducedPixels(prepareImage(imageData)))Conclusions
Proposed approach is more streamlined then existing ones because it has no dependencies and focuses on variance as a measure of blur. It consists of fewer operations and has a more straightforward process which might lead to better performance, especially when applied iteratively or on numerous images.
Advantages:
- Simplicity makes it easy to understand, modify, and maintain.
- The Laplacian-based approach is a classic method for blur detection, relying on the principle that a sharp image will have a high variance of intensities in its edges.
- It does not depend on external libraries like canvasfilters or openCV.
Disadvantages:
- May not be as nuanced or detailed in its evaluation as existing solutions.
- Could potentially miss subtle details or nuances in certain images.
In summary, the proposed algorithm’s simplicity could suffice for most general blur detection tasks. It makes it easier to integrate, adapt, or extend. The existing solutions, due to its dependence on other libraries, might have some constraints. If speed, simplicity, and independence from external libraries are paramount, then this solution would be the way to go.
