Image Processing using Python: Understanding Images, Pixels, Arrays & OpenCV
1. Digital Images & How Computers See Them
To human vision, an image is a continuous field of light and color. Light reflects off an object, enters the eye, stimulates photoreceptors on the retina, and sends electrical signals via the optic nerve to the visual cortex of the brain to be recognized.
Computers have no retina or visual cortex. A digital image inside computer memory is simply a grid of numbers (matrices). Each entry in the grid represents a discrete measurement of light intensity or color sampled across a 2D coordinate space.
2. Pixels, Dimensions, and Color Spaces
What is a Pixel?
The term Pixel is an abbreviation of Picture + Element. It represents the smallest individual building block or unit of a digital image.
- MegaPixel (MP): 1{ MP} = 1,000,000 pixels. For instance, a sensor capturing 4000 * 3000 pixels produces 12,000,000 pixels, or a 12 MP image.
Pixel Values & Dynamic Range
Get Tarunsai’s stories in your inbox
Join Medium for free to get updates from this writer.
In standard 8-bit digital imaging, each color component per pixel is represented by an integer in the closed range $[0, 255]$ ($2⁸ = 256$ discrete levels):
- 0: Absence of light (pure black)
- 255: Maximum intensity of that specific light component
Image Dimensions vs. Matrix Shapes
- Resolution / Image Size: Stated conventionally as Width * Height.
- Width: Count of pixels from left to right along horizontal axis.
- Height: Count of pixels from top to bottom along vertical axis.
- Array / Matrix Dimension: In NumPy and OpenCV, coordinates follow standard matrix notation:
(Height, Width, Channels).
3. Channels & Color Formats
A channel is an independent 2D matrix storing intensity values for a single primary component.
4. Images as NumPy Arrays
When an image is loaded using an image processing library, its numerical grid is converted directly into an ndarray.
Multi-Dimensional Structure
- Grayscale Image: 2D array of shape
(Height, Width). - Color Image (RGB / BGR): 3D array of shape
(Height, Width, Channels).
Flattening for Classical Machine Learning
Classical algorithms (e.g., Logistic Regression, SVMs, Decision Trees, Random Forests) require 2D tabular feature inputs (samples, features).
5. Introduction to OpenCV
OpenCV (Open Source Computer Vision Library) is an optimized, open-source library built in C++ with bindings for Python, C++, and Java.
- Execution Speed: Implements hardware acceleration routines (SIMD, CUDA, OpenCL) for real-time video stream manipulation (e.g., 24+ FPS processing).
- Array-Native Integration: Automatically maps loaded images and video frames directly into native NumPy arrays without auxiliary wrappers.
6. Core OpenCV Operations & Syntax
1. Reading an Image (cv2.imread)
Loads an image file from a directory into a NumPy array.
2. Displaying an Image (cv2.imshow, cv2.waitKey, cv2.destroyAllWindows)
Opens an OS-level window rendering the image matrix.
3. Writing / Saving an Image (cv2.imwrite)
Serializes a NumPy array back into an encoded file format (.png, .jpg).
4. Color Conversion (cv2.cvtColor)
Changes color representations across color spaces.
5. Resizing & Interpolation (cv2.resize)
A computer cannot discard or duplicate pixels arbitrarily when altering dimensions. Instead, it uses interpolation algorithms (mathematical approximation methods) to compute intermediate pixel values:
cv2.INTER_AREA: Preferred for downsampling (shrinking) images.cv2.INTER_LINEAR/cv2.INTER_CUBIC: Preferred for upsampling (enlarging) images.
6. Cropping via Array Slicing
Because OpenCV images are native NumPy arrays, spatial regions of interest (ROI) are extracted using standard slice syntax: array[y_start:y_end, x_start:x_end].
7. End-to-End Project: MNIST Image Classification Pipeline
This pipeline implements the end-to-end workflow shown in the lecture notes: ingesting image folders, transforming them via OpenCV/NumPy, building a structured tabular dataset, and training a classifier.
8. Common Interview Q&A
Q1: Why does OpenCV load images in BGR instead of standard RGB format?
A: When OpenCV was initially developed in the late 1990s, early camera hardware manufacturers and frame grabbers delivered image data in BGR memory layouts. OpenCV maintained this standard for backward compatibility across its C/C++ runtime.
Q2: Why must images be resized to uniform dimensions before feeding into classical ML or Deep Learning networks?
A: Classical machine learning algorithms and fully connected neural network layers require fixed-length feature vectors. If images possess variable heights and widths, flattening them results in mismatched feature sizes (W_1 * H_1 != W_2 * H_2), which breaks batch matrix multiplications (X .W).
Q3: What is the computational difference between image downsampling using cv2.INTER_AREA versus cv2.INTER_NEAREST?
A: INTER_NEAREST simply picks the value of the nearest neighboring pixel, which introduces severe pixelation and aliasing artifacts when downsampling. INTER_AREA resamples pixels using pixel-area relations, effectively averaging over pixel neighborhoods to produce smoother results with fewer artifacts.
Q4: Why divide pixel values by 255.0 before training?
A: Raw pixel values span [0, 255]. Normalizing them to [0.0, 1.0] prevents gradient explosion in optimization routines, speeds up convergence, and balances feature weight dominance across inputs.
