<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Riwaj Neupane on Medium]]></title>
        <description><![CDATA[Stories by Riwaj Neupane on Medium]]></description>
        <link>https://medium.com/@RiwajNeupane?source=rss-ab66ffa56064------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Riwaj Neupane on Medium</title>
            <link>https://medium.com/@RiwajNeupane?source=rss-ab66ffa56064------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 15:27:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@RiwajNeupane/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[YOLO Algorithm]]></title>
            <link>https://medium.com/@RiwajNeupane/yolo-algorithm-c4f4bb1cdcd8?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/c4f4bb1cdcd8</guid>
            <category><![CDATA[basics]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[object-detection]]></category>
            <category><![CDATA[yolo]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Mon, 15 Jan 2024 07:43:05 GMT</pubDate>
            <atom:updated>2024-01-15T07:43:05.257Z</atom:updated>
            <content:encoded><![CDATA[<p>YOLO (You Only Look Once) is a real-time object detection algorithm developed by Joseph Redmon and Ali Farhadi in 2015. It is a single-stage object detector that uses a convolutional neural network (CNN) to predict the bounding boxes and class probabilities of objects in input images. YOLO was first implemented using the Darkent framework.</p><p>The YOLO algorithm divides the input image into a grid of cells, and for each cell, it predicts the probability of the presence of an object and the bounding box coordinates of the object. It also predicts the class of the object. Unlike two-stage object detectors such as R-CNN and its variants, YOLO processes the entire image in one pass, making it faster and more efficient.</p><p>YOLO has been developed in several versions, such as YOLOv1, YOLOv2, YOLOv3, YOLOv4, YOLOv5, YOLOv6, and YOLOv7. Each version has been built on top of the previous version with enhanced features such as improved accuracy, faster processing, and better handling of small objects.</p><p>YOLO is widely used in various applications such as self-driving cars and surveillance systems. It is also widely used for real-time object detection tasks like in real-time video analytics and real-time video surveillance.`</p><p><strong>YOLO Algorithm: How Does it Works?</strong></p><p>The basic idea behind YOLO is to divide the input image into a grid of cells and, for each cell, predict the probability of the presence of an object and the bounding box coordinates of the object. The process of YOLO can be broken down into several steps:</p><ol><li>Input image is passed through a CNN to extract features from the image.</li><li>The features are then passed through a series of fully connected layers, which predict ‌class probabilities and bounding box coordinates.</li><li>The image is divided into a grid of cells, and each cell is responsible for predicting a set of bounding boxes and class probabilities.</li><li>The output of the network is a set of bounding boxes and class probabilities for each cell.</li><li>The bounding boxes are then filtered using a post-processing algorithm called non-max suppression to remove overlapping boxes and choose the box with the highest probability.</li><li>The final output is a set of predicted bounding boxes and class labels for each object in the image.</li><li>One of the key advantages of YOLO is that it processes the entire image in one pass, making it faster and more efficient than two-stage object detectors such as R-CNN and its variants.</li></ol><p><strong>YOLOV2</strong></p><p>YOLO v2, also known as YOLO 9000, is an improved version of the original YOLO object detection algorithm. It builds upon the concepts and architecture of YOLO, but addresses some of the limitations of the original version.</p><p>One of the main differences between YOLO v2 and the original YOLO is the use of anchor boxes. In YOLO v2, ‌CNN predicts not only the bounding box coordinates but also the anchor boxes. Anchor boxes are pre-defined boxes of different aspect ratios and scales, which are used to match the predicted bounding boxes with the actual objects in the image. This allows YOLO v2 to handle objects of different shapes and sizes better.</p><p>Another key difference is the use of a multi-scale approach. In YOLO v2, the input image is fed through ‌CNN at multiple scales, which allows the model to detect objects at different sizes. This is achieved by using a feature pyramid network (FPN), which allows the model to extract features at different scales from the same image.</p><p>Additionally, YOLO v2 uses a different loss function than the original YOLO, called the sum-squared error (SSE) loss function. The SSE loss function is more robust and helps the model to converge faster.</p><p>In terms of architecture, YOLO v2 uses a slightly deeper CNN than YOLO, which allows it to extract more powerful features from the image. The CNN is followed by several fully connected layers, which predict ‌class probabilities and bounding box coordinates.</p><p><strong>YOLO v3</strong></p><p>YOLO v3 is the third version of the YOLO object detection algorithm. The first difference between YOLO v3 and previous versions is the use of multiple scales in the input image. YOLO v3 uses a technique called “feature pyramid network” (FPN) to extract features from the image at different scales. This allows the model to detect objects of different sizes in the image.</p><p>Another important difference is the use of anchor boxes. In YOLO v3, anchor boxes are used to match the predicted bounding boxes with the actual objects in the image. Anchor boxes are pre-defined boxes of different aspect ratios and scales, and the model predicts the offset of the anchor boxes relative to the bounding boxes. This helps the model to handle objects of different shapes and sizes better.</p><p>In terms of architecture, YOLO v3 is built on a deep convolutional neural network (CNN) that is composed of many layers of filters. The CNN is followed by several fully connected layers, which predict ‌class probabilities and bounding box coordinates.</p><p>YOLO v3 also uses a different loss function than previous versions. It uses a combination of classification loss and localization loss, which allows the model to learn both the class probabilities and the bounding box coordinates.</p><p><strong>YOLO v4</strong></p><p>Now, let’s take a look at one of the most important versions of YOLO. A key distinction between YOLO v4 and previous versions is using a more advanced neural network architecture. YOLO v4 uses a technique called “Spatial Pyramid Processing” (SPP) to extract features from the image at different scales and resolutions. This allows the model to detect objects of different sizes in the image.</p><p>Additionally, YOLO v4 also uses a technique called “Cross-stage partial connection” (CSP) to improve the model’s accuracy. It uses a combination of multiple models with different architectures and scales and combines their predictions to achieve better accuracy.</p><p><strong>YOLO v5</strong></p><p>YOLO v5 was introduced in 2020 with a key difference from the previous versions, which is the use of a more efficient neural network architecture called EfficientDet, which is based on the EfficientNet architecture. EfficientDet is a family of image classification models that have achieved state-of-the-art performance on a number of benchmark datasets. The EfficientDet architecture is designed to be efficient in terms of computation and memory usage while also achieving high accuracy.</p><p>Another important difference is the use of anchor-free detection, which eliminates the need for anchor boxes used in previous versions of YOLO. Instead of anchor boxes, YOLO v5 uses a single convolutional layer to predict the bounding box coordinates directly, which allows the model to be more flexible and adaptable to different object shapes and sizes.</p><p>YOLO v5 also uses a technique called “Cross mini-batch normalization” (CmBN) to improve the model’s accuracy. CmBN is a variant of the standard batch normalization technique that is used to normalize the activations of the neural network.</p><p>Regarding training, YOLO v5 uses transfer learning, which allows it to be pre-trained on a large dataset and then fine-tuned on a smaller dataset. This allows the model to learn from a wide range of data and generalize better to new data.</p><p><strong>YOLO v6</strong></p><p>A notable contrast between YOLO v6 and previous versions is the use of a more efficient and lightweight neural network architecture; this allows YOLO v6 to run faster and with fewer computational resources. The architecture of YOLO v6 is based on the “EfficientNet-Lite” family, which is a set of lightweight models that can be run on various devices with limited computational resources.</p><p>YOLO v6 also incorporates data augmentation techniques to improve the robustness and generalization of the model. This is done by applying random transformations to the input images during training, such as rotation, scaling, and flipping.</p><p><strong>YOLO v7</strong></p><p>YOLO v7, on the most recent stable iterations of the YOLO algorithm. It boasts a number of enhancements compared to ‌previous versions. A key enhancement is the implementation of anchor boxes. These anchor boxes, which come in various aspect ratios, are utilized to identify objects of various shapes. The use of nine anchor boxes in YOLO v7 enables it to detect a wider range of object shapes and sizes, leading to a decrease in false positives.</p><p>In YOLO v7, a new loss function called “focal loss” is implemented to enhance performance. Unlike the standard cross-entropy loss function used in previous versions of YOLO, focal loss addresses the difficulty in detecting small objects by adjusting the weight of the loss on well-classified examples and placing more emphasis on challenging examples to detect.</p><p><strong>YOLO Algorithms: Fortes</strong></p><p>YOLO is widely used in real-world projects because of its accuracy and speed; its main powerful sides can be listed like the following:</p><ol><li>Real-time object detection: YOLO is able to detect objects in real-time, making it suitable for applications such as video surveillance or self-driving cars.</li><li>High accuracy: YOLO achieves high accuracy by using a convolutional neural network (CNN) to predict both the class and location of objects in an image.</li><li>Single-shot detection: YOLO can detect objects in an image with just one forward pass through the network, making it more efficient than other object detection methods that require multiple passes.</li><li>Good performance on small objects: YOLO is able to detect small objects in an image because of its grid-based approach.</li><li>Efficient use of GPUs: YOLO uses a fully convolutional network architecture, which allows for the efficient use of GPUs during training and inference.</li><li>Ability to handle multiple scales: YOLO uses anchor boxes, which allows the model to handle objects of different scales, thus allowing the model to detect objects of different sizes in the same image.</li></ol><h3>YOLO Algorithm: Limitations</h3><p>Even though YOLO is a powerful object detection algorithm, it also has some limitations. Some of these limitations include:</p><ol><li>Limited to object detection: YOLO is primarily designed for object detection and may not perform as well on other tasks such as image segmentation or instance segmentation.</li><li>Less accurate than some other methods: While YOLO is accurate, it may not be as accurate as two-shot object detection methods, such as RetinaNet or Mask R-CNN.</li><li>Struggles with very small objects: YOLO’s grid-based approach can make it difficult to detect tiny objects, especially if they are located close to other objects.</li><li>No tracking capability: YOLO does not provide any tracking capability, so it may not be suitable for video surveillance applications that require tracking of objects over time.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c4f4bb1cdcd8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OCR with PyTesseract and EasyOCR]]></title>
            <link>https://medium.com/@RiwajNeupane/ocr-with-pytesseract-and-easyocr-2747180e8b66?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/2747180e8b66</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[ocr]]></category>
            <category><![CDATA[object-detection]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Mon, 15 Jan 2024 06:54:25 GMT</pubDate>
            <atom:updated>2024-01-15T06:54:25.555Z</atom:updated>
            <content:encoded><![CDATA[<p>Tesseract is an open source text recognition (OCR) Engine, available under the Apache 2.0 license. It can be used directly, or (for programmers) using an API to extract printed text from images. It supports a wide variety of languages. Tesseract doesn’t have a built-in GUI, but there are several available from the 3rdParty page. Tesseract is compatible with many programming languages and frameworks through wrappers that can be found here. It can be used with the existing layout analysis to recognize text within a large document, or it can be used in conjunction with an external text detector to recognize text from an image of a single text line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oPfo5OeWzt3E0gSI.png" /></figure><p>Tesseract 4.00 includes a new neural network subsystem configured as a text line recognizer. It has its origins in OCRopus’ Python-based LSTM implementation but has been redesigned for Tesseract in C++. The neural network system in Tesseract pre-dates TensorFlow but is compatible with it, as there is a network description language called Variable Graph Specification Language (VGSL), that is also available for TensorFlow.</p><p>To recognize an image containing a single character, we typically use a Convolutional Neural Network (CNN). Text of arbitrary length is a sequence of characters, and such problems are solved using RNNs and LSTM is a popular form of RNN. Read this post to learn more about LSTM.</p><p><strong>Technology — How it works</strong></p><p>LSTMs are great at learning sequences but slow down a lot when the number of states is too large. There are empirical results that suggest it is better to ask an LSTM to learn a long sequence than a short sequence of many classes. Tesseract developed from OCRopus model in Python which was a fork of a LSMT in C++, called CLSTM. CLSTM is an implementation of the LSTM recurrent neural network model in C++, using the Eigen library for numerical computations</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PXkUyaTIporpN-aV.png" /></figure><p>Legacy Tesseract 3.x was dependant on the multi-stage process where we can differentiate steps:</p><ul><li>Word finding</li><li>Line finding</li><li>Character classification</li></ul><p>Word finding was done by organizing text lines into blobs, and the lines and regions are analyzed for fixed pitch or proportional text. Text lines are broken into words differently according to the kind of character spacing. Recognition then proceeds as a two-pass process. In the first pass, an attempt is made to recognize each word in turn. Each word that is satisfactory is passed to an adaptive classifier as training data. The adaptive classifier then gets a chance to more accurately recognize text lower down the page.</p><p>Modernization of the Tesseract tool was an effort on code cleaning and adding a new LSTM model. The input image is processed in boxes (rectangle) line by line feeding into the LSTM model and giving output. In the image below we can visualize how it works.</p><p><strong>OCR with Pytesseract and OpenCV</strong></p><p>Pytesseract or Python-tesseract is an OCR tool for python that also serves as a wrapper for the Tesseract-OCR Engine. It can read and recognize text in images and is commonly used in python ocr image to text use cases.</p><p>It is also useful as a stand-alone invocation script to tesseract, as it can read all image types supported by the Pillow and Leptonica imaging libraries, including jpeg, png, gif, bmp, tiff, and others.</p><p>Code Implementation:</p><pre>from skimage.filters import threshold_local<br><br>image = cv2.imread(&#39;Receipt-woolworth.jpg&#39;)<br><br># We get the Value component from the HSV color space <br># then we apply adaptive thresholdingto <br>V = cv2.split(cv2.cvtColor(image, cv2.COLOR_BGR2HSV))[2]<br>T = threshold_local(V, 25, offset=15, method=&quot;gaussian&quot;)<br><br># Apply the threshold operation <br>thresh = (V &gt; T).astype(&quot;uint8&quot;) * 255<br>imshow(&quot;threshold_local&quot;, thresh)<br><br>output_txt = pytesseract.image_to_string(thresh)<br>print(&quot;PyTesseract Extracted: {}&quot;.format(output_txt))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*oa_xjAxf-37K1dLywOhfRg.jpeg" /></figure><p><strong>EasyOCR</strong></p><p>EasyOCR, as the name suggests, is a Python package that allows computer vision developers to effortlessly perform Optical Character Recognition.</p><p>A varied dataset of text images is fundamental for getting started with EasyOCR. It helps the OCR system to handle a wide range of text styles, fonts, and orientations, enhancing the system’s overall effectiveness.</p><p>Roboflow has free tools for each stage of the computer vision pipeline that will streamline your workflows and supercharge your productivity.</p><p>he <a href="https://github.com/JaidedAI/EasyOCR"><strong>EasyOCR package</strong></a> is created and maintained by <a href="https://jaided.ai/"><strong>Jaided AI</strong></a>, a company that specializes in Optical Character Recognition services.</p><p>EasyOCR is implemented using Python and the PyTorch library. If you have a CUDA-capable GPU, the underlying PyTorch deep learning library can speed up your text detection and OCR speed <em>tremendously.</em></p><p>As of this writing, EasyOCR can OCR text in <strong>58 languages,</strong> including English, German, Hindi, Russian, <em>and more!</em> The EasyOCR maintainers plan to add additional languages in the future. You can find the full list of languages EasyOCR supports <strong>on the following page</strong>.</p><p>Currently, EasyOCR only supports OCR’ing typed text. Later in 2020 they plan on releasing a <em>handwriting recognition</em> model as well!</p><p><strong>Code Implementation:</strong></p><pre># import the necessary packages<br>from matplotlib import pyplot as plt<br>from easyocr import Reader<br>import pandas as pd<br>import cv2<br>import time<br><br># Define our imshow function <br>def imshow(title = &quot;Image&quot;, image = None, size = 10):<br>    w, h = image.shape[0], image.shape[1]<br>    aspect_ratio = w/h<br>    plt.figure(figsize=(size * aspect_ratio,size))<br>    plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))<br>    plt.title(title)<br>    plt.show()<br><br># load the input image from disk<br>image = cv2.imread(&quot;whatsapp_conv.jpeg&quot;)<br>imshow(&quot;Original Image&quot;, image, size = 12)<br><br># OCR the input image using EasyOCR<br>print(&quot;Detecting and OCR&#39;ing text from input image...&quot;)<br>reader = Reader([&#39;en&#39;], gpu = False)<br><br>ts = time.time()<br>results = reader.readtext(image)<br>te = time.time()<br>td = te - ts<br>print(f&#39;Completed in {td} seconds&#39;)</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2747180e8b66" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GrabCut Algorithm for Background Removal]]></title>
            <link>https://medium.com/@RiwajNeupane/grabcut-algorithm-for-background-removal-497129944bd6?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/497129944bd6</guid>
            <category><![CDATA[object-detection]]></category>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[background-removal]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Sun, 14 Jan 2024 10:05:28 GMT</pubDate>
            <atom:updated>2024-01-14T10:05:28.149Z</atom:updated>
            <content:encoded><![CDATA[<p>GrabCut algorithm was designed by Carsten Rother, Vladimir Kolmogorov &amp; Andrew Blake from Microsoft Research Cambridge, UK. in their paper, “GrabCut”: interactive foreground extraction using iterated graph cuts . An algorithm was needed for foreground extraction with minimal user interaction, and the result was GrabCut.</p><p>How it works from user point of view ? Initially user draws a rectangle around the foreground region (foreground region should be completely inside the rectangle). Then algorithm segments it iteratively to get the best result. Done. But in some cases, the segmentation won’t be fine, like, it may have marked some foreground region as background and vice versa. In that case, user need to do fine touch-ups. Just give some strokes on the images where some faulty results are there. Strokes basically says *”Hey, this region should be foreground, you marked it background, correct it in next iteration”* or its opposite for background. Then in the next iteration, you get better results.</p><p>See the image below. First player and football is enclosed in a blue rectangle. Then some final touchups with white strokes (denoting foreground) and black strokes (denoting background) is made. And we get a nice result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*VtPpNILb_-cvD07A.jpg" /></figure><p>So what happens in background ?</p><ul><li>User inputs the rectangle. Everything outside this rectangle will be taken as sure background (That is the reason it is mentioned before that your rectangle should include all the objects). Everything inside rectangle is unknown. Similarly any user input specifying foreground and background are considered as hard-labelling which means they won’t change in the process.</li><li>Computer does an initial labelling depending on the data we gave. It labels the foreground and background pixels (or it hard-labels)</li><li>Now a Gaussian Mixture Model(GMM) is used to model the foreground and background.</li><li>Depending on the data we gave, GMM learns and create new pixel distribution. That is, the unknown pixels are labelled either probable foreground or probable background depending on its relation with the other hard-labelled pixels in terms of color statistics (It is just like clustering).</li><li>A graph is built from this pixel distribution. Nodes in the graphs are pixels. Additional two nodes are added, <strong>Source node</strong> and <strong>Sink node</strong>. Every foreground pixel is connected to Source node and every background pixel is connected to Sink node.</li><li>The weights of edges connecting pixels to source node/end node are defined by the probability of a pixel being foreground/background. The weights between the pixels are defined by the edge information or pixel similarity. If there is a large difference in pixel color, the edge between them will get a low weight.</li><li>Then a mincut algorithm is used to segment the graph. It cuts the graph into two separating source node and sink node with minimum cost function. The cost function is the sum of all weights of the edges that are cut. After the cut, all the pixels connected to Source node become foreground and those connected to Sink node become background.</li><li>The process is continued until the classification converges.</li></ul><p>It is illustrated in below image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*wHMO0dHwo_RJy97X.jpg" /></figure><p>Now we go for grabcut algorithm with OpenCV. OpenCV has the function, <strong>cv.grabCut()</strong> for this. We will see its arguments first:</p><ul><li><em>img</em> — Input image</li><li><em>mask</em> — It is a mask image where we specify which areas are background, foreground or probable background/foreground etc. It is done by the following flags, <strong>cv.GC_BGD, cv.GC_FGD, cv.GC_PR_BGD, cv.GC_PR_FGD</strong>, or simply pass 0,1,2,3 to image.</li><li><em>rect</em> — It is the coordinates of a rectangle which includes the foreground object in the format (x,y,w,h)</li><li><em>bdgModel</em>, <em>fgdModel</em> — These are arrays used by the algorithm internally. You just create two np.float64 type zero arrays of size (1,65).</li><li><em>iterCount</em> — Number of iterations the algorithm should run.</li><li><em>mode</em> — It should be <strong>cv.GC_INIT_WITH_RECT</strong> or <strong>cv.GC_INIT_WITH_MASK</strong> or combined which decides whether we are drawing rectangle or final touchup strokes.</li></ul><p>Code Implementation</p><pre>image = cv2.imread(&#39;/content/h.jpg&#39;)<br>copy = image.copy()<br># Create a mask (of zeros uint8 datatype) that is the same size (width, height) as our original image <br>mask = np.zeros(image.shape[:2], np.uint8)<br><br>bgdModel = np.zeros((1,65), np.float64)<br>fgdModel = np.zeros((1,65), np.float64)<br><br># Needs to be set Manually or selected with cv2.selectROI()<br>x1, y1, x2, y2 = 60, 30, 120, 150<br><br>start = (x1, y1)<br>end = (x2, y2)<br><br># Format is X,Y,W,H<br>rect = (x1,y1,x2-x1,y2-y1)<br><br># Show Rectangle<br>cv2.rectangle(copy, start, end, (0,0,255), 1)<br>imshow(&quot;Input Image&quot;, copy)<br></pre><pre># Let the algorithm run for 5 iterations. Mode should be cv.GC_INIT_WITH_RECT since we are using rectangle. <br># Grabcut modifies the mask image. <br># In the new mask image, pixels will be marked with four flags denoting background/foreground as specified above. <br># So we modify the mask such that all 0-pixels and 2-pixels are put to 0 (i.e. background) &amp; all 1-pixels and 3-pixels are put to 1(i.e. foreground pixels). <br># Now our final mask is ready. Just multiply it with input image to get the segmented image.<br>cv2.grabCut(image, mask, rect, bgdModel, fgdModel, 5, cv2.GC_INIT_WITH_RECT)<br><br>mask2 = np.where((mask==2)|(mask==0),0,1).astype(&#39;uint8&#39;)<br>image = image * mask2[:,:,np.newaxis]<br><br>imshow(&quot;Mask&quot;, mask * 80)<br>imshow(&quot;Mask2&quot;, mask2 * 255)<br>imshow(&quot;Image&quot;, image)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/178/1*jh0z2x4mi35_sff5jkIuVg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*2vxSn1cc2Wxs09MOmLVm-A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*Ia-yemiZzXvnlzHqWV6fYA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*pC-2DzG2yCLclglSuJg6mg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*MtPh31VPnB5Tn0khuy_54A.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=497129944bd6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Facial Landmark Detection]]></title>
            <link>https://medium.com/@RiwajNeupane/facial-landmark-detection-a6b3e29eac5b?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6b3e29eac5b</guid>
            <category><![CDATA[facial-recognition]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[image-processing]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Sun, 14 Jan 2024 07:27:40 GMT</pubDate>
            <atom:updated>2024-01-14T07:27:40.758Z</atom:updated>
            <content:encoded><![CDATA[<p>Today we are going to use dlib and OpenCV to detect <strong><em>facial landmarks</em></strong> in an image.</p><p>Facial landmarks are used to localize and represent salient regions of the face, such as:</p><ul><li>Eyes</li><li>Eyebrows</li><li>Nose</li><li>Mouth</li><li>Jawline</li></ul><p>Facial landmarks have been successfully applied to face alignment, head pose estimation, face swapping, blink detection and much more.</p><p>In today’s blog post we’ll be focusing on the <strong>basics of facial landmarks</strong>, including:</p><ol><li>Exactly <em>what</em> facial landmarks are and <em>how</em> they work.</li><li>How to detect and extract facial landmarks from an image using dlib, OpenCV, and Python.</li></ol><p><strong>Introduction to Facial Landmarks</strong></p><p>Detecting facial landmarks is a <em>subset</em> of the <em>shape prediction</em> problem. Given an input image (and normally an ROI that specifies the object of interest), a shape predictor attempts to localize key points of interest along the shape.</p><p>In the context of facial landmarks, our goal is detect important facial structures on the face using shape prediction methods.</p><p>Detecting facial landmarks is therefore a two step process:</p><ul><li><strong>Step #1:</strong> Localize the face in the image.</li><li><strong>Step #2:</strong> Detect the key facial structures on the face ROI.</li></ul><p><strong>Understanding dlib’s facial landmark detector</strong></p><p>The pre-trained facial landmark detector inside the dlib library is used to estimate the location of <strong><em>68 (x, y)-coordinates</em></strong> that map to facial structures on the face.</p><p>The indexes of the 68 coordinates can be visualized on the image below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LMsomWjiTKW3hgeE" /></figure><p>These annotations are part of the 68 point <a href="https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/"><strong>iBUG 300-W dataset</strong></a> which the dlib facial landmark predictor was trained on.</p><p>It’s important to note that other flavors of facial landmark detectors exist, including the 194 point model that can be trained on the <strong>HELEN dataset</strong>.</p><p>Regardless of which dataset is used, the same dlib framework can be leveraged to train a shape predictor on the input training data — this is useful if you would like to train facial landmark detectors or custom shape predictors of your own.</p><p>In the remaining of this blog post I’ll demonstrate how to detect these facial landmarks in images.</p><p>Future blog posts in this series will use these facial landmarks to extract <em>specific</em> regions of the face, apply face alignment, and even build a blink detection system.</p><p><strong>Code Implementation:</strong></p><pre>PREDICTOR_PATH = &quot;shape_predictor_68_face_landmarks.dat&quot;<br>predictor = dlib.shape_predictor(PREDICTOR_PATH)<br>detector = dlib.get_frontal_face_detector()<br><br>class TooManyFaces(Exception):<br>    pass<br><br>class NoFaces(Exception):<br>    pass<br><br>def get_landmarks(im):<br>    rects = detector(im, 1)<br><br>    if len(rects) &gt; 1:<br>        raise TooManyFaces<br>    if len(rects) == 0:<br>        raise NoFaces<br><br>    return np.matrix([[p.x, p.y] for p in predictor(im, rects[0]).parts()])<br><br>def annotate_landmarks(im, landmarks):<br>    im = im.copy()<br>    for idx, point in enumerate(landmarks):<br>        pos = (point[0, 0], point[0, 1])<br>        cv2.putText(im, str(idx), pos,<br>                    fontFace=cv2.FONT_HERSHEY_SCRIPT_SIMPLEX,<br>                    fontScale=0.4,<br>                    <br>                    color=(0, 0, 255))<br>        cv2.circle(im, pos, 3, color=(0, 255, 255))<br>    return im<br><br>image = cv2.imread(&#39;images/Trump.jpg&#39;)<br>imshow(&#39;Original&#39;, image)<br>landmarks = get_landmarks(image)<br>image_with_landmarks = annotate_landmarks(image, landmarks)<br>imshow(&#39;Result&#39;, image_with_landmarks)</pre><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UVXQvCOck8xcL5lVKeH5XA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6b3e29eac5b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Motion and Object Tracking]]></title>
            <link>https://medium.com/@RiwajNeupane/motion-and-object-tracking-42eb1e6a5443?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/42eb1e6a5443</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[tracking-algorit]]></category>
            <category><![CDATA[object-tracking]]></category>
            <category><![CDATA[video-processing]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Sun, 14 Jan 2024 06:56:58 GMT</pubDate>
            <atom:updated>2024-01-14T06:56:58.826Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Simple Object Tracking by color:</strong></p><p>Color-based object tracking using the HSV (Hue, Saturation, Value) color space is a popular technique in computer vision and image processing. HSV is a color representation that separates the color information into three components: Hue, Saturation, and Value. This representation is often preferred for color-based image processing tasks because it better aligns with human perception of colors.</p><p>Here’s a brief explanation of each component in the HSV color space:</p><ol><li>Hue (H): Represents the type of color. It is measured in degrees (0 to 360), with red at 0, green at 120, and blue at 240. This circular representation is helpful for color-based segmentation.</li><li>Saturation (S): Represents the vividness or purity of the color. It is a percentage ranging from 0% (gray) to 100% (fully saturated color)</li><li>Value (V): Represents the brightness of the color. It also ranges from 0% (black) to 100% (fully bright or white).</li></ol><p>Code Implementation:</p><pre>#Object Tracking<br>import cv2<br>import numpy as np<br><br># Initalize camera<br>#cap = cv2.VideoCapture(0)<br><br># define range of color in HSV<br>lower = np.array([20,50,90])<br>upper = np.array([40,255,255])<br><br># Create empty points array<br>points = []<br><br># Get default camera window size<br><br># Load video stream, long clip<br>cap = cv2.VideoCapture(&#39;bmwm4.mp4&#39;)<br><br># Get the height and width of the frame (required to be an interger)<br>width = int(cap.get(3)) <br>height = int(cap.get(4))<br><br># Define the codec and create VideoWriter object. The output is stored in &#39;*.avi&#39; file.<br>out = cv2.VideoWriter(&#39;bmwm4_output.avi&#39;, cv2.VideoWriter_fourcc(&#39;M&#39;,&#39;J&#39;,&#39;P&#39;,&#39;G&#39;), 30, (width, height))<br><br>ret, frame = cap.read()<br>Height, Width = frame.shape[:2]<br>frame_count = 0<br>radius = 0<br><br>while True:<br>  <br>    # Capture webcame frame<br>    ret, frame = cap.read()<br>    if ret:<br>      hsv_img = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)<br><br>      # Threshold the HSV image to get only green colors<br>      mask = cv2.inRange(hsv_img, lower, upper)<br>      #mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)<br>      <br>      contours, _ = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)<br>      <br>      # Create empty centre array to store centroid center of mass<br>      center =   int(Height/2), int(Width/2)<br><br>      if len(contours) &gt; 0:<br>          <br>          # Get the largest contour and its center <br>          c = max(contours, key=cv2.contourArea)<br>          (x, y), radius = cv2.minEnclosingCircle(c)<br>          M = cv2.moments(c)<br>          <br>          # Sometimes small contours of a point will cause a divison by zero error<br>          try:<br>              center = (int(M[&quot;m10&quot;] / M[&quot;m00&quot;]), int(M[&quot;m01&quot;] / M[&quot;m00&quot;]))<br><br>          except:<br>              center =   int(Height/2), int(Width/2)<br><br>          # Allow only countors that have a larger than 25 pixel radius<br>          if radius &gt; 25:<br>              <br>              # Draw cirlce and leave the last center creating a trail<br>              cv2.circle(frame, (int(x), int(y)), int(radius),(0, 0, 255), 2)<br>              cv2.circle(frame, center, 5, (0, 255, 0), -1)<br>              <br>          # Log center points <br>          points.append(center)<br>      <br>      # If radius large enough, we use 25 pixels<br>      if radius &gt; 25:<br>          <br>          # loop over the set of tracked points<br>          for i in range(1, len(points)):<br>              try:<br>                  cv2.line(frame, points[i - 1], points[i], (0, 255, 0), 2)<br>              except:<br>                  pass<br>              <br>          # Make frame count zero<br>          frame_count = 0<br>              <br>      out.write(frame)<br>    else:<br>      break<br><br># Release camera and close any open windows<br>cap.release()<br>out.release()</pre><p>Motion Tracking by MeanShift and CAMSHIFT:</p><p>The intuition behind the meanshift is simple. Consider you have a set of points. (It can be a pixel distribution like histogram backprojection). You are given a small window (may be a circle) and you have to move that window to the area of maximum pixel density (or maximum number of points). It is illustrated in the simple image given below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*A6skKOgEouWLdyyZ.jpg" /></figure><p>The initial window is shown in blue circle with the name “C1”. Its original center is marked in blue rectangle, named “C1_o”. But if you find the centroid of the points inside that window, you will get the point “C1_r” (marked in small blue circle) which is the real centroid of the window. Surely they don’t match. So move your window such that the circle of the new window matches with the previous centroid. Again find the new centroid. Most probably, it won’t match. So move it again, and continue the iterations such that the center of window and its centroid falls on the same location (or within a small desired error). So finally what you obtain is a window with maximum pixel distribution. It is marked with a green circle, named “C2”. As you can see in the image, it has maximum number of points.</p><p><strong>Code Implementation</strong>:</p><pre>import numpy as np<br>import cv2 as cv<br>import argparse<br>parser = argparse.ArgumentParser(description=&#39;This sample demonstrates the meanshift algorithm. \<br>                                              The example file can be downloaded from: \<br>                                              https://www.bogotobogo.com/python/OpenCV_Python/images/mean_shift_tracking/slow_traffic_small.mp4&#39;)<br>parser.add_argument(&#39;image&#39;, type=str, help=&#39;path to image file&#39;)<br>args = parser.parse_args()<br>cap = cv.VideoCapture(args.image)<br># take first frame of the video<br>ret,frame = cap.read()<br># setup initial location of window<br>x, y, w, h = 300, 200, 100, 50 # simply hardcoded the values<br>track_window = (x, y, w, h)<br># set up the ROI for tracking<br>roi = frame[y:y+h, x:x+w]<br>hsv_roi =  cv.cvtColor(roi, cv.COLOR_BGR2HSV)<br>mask = cv.inRange(hsv_roi, np.array((0., 60.,32.)), np.array((180.,255.,255.)))<br>roi_hist = cv.calcHist([hsv_roi],[0],mask,[180],[0,180])<br>cv.normalize(roi_hist,roi_hist,0,255,cv.NORM_MINMAX)<br># Setup the termination criteria, either 10 iteration or move by at least 1 pt<br>term_crit = ( cv.TERM_CRITERIA_EPS | cv.TERM_CRITERIA_COUNT, 10, 1 )<br>while(1):<br>    ret, frame = cap.read()<br>    if ret == True:<br>        hsv = cv.cvtColor(frame, cv.COLOR_BGR2HSV)<br>        dst = cv.calcBackProject([hsv],[0],roi_hist,[0,180],1)<br>        # apply meanshift to get the new location<br>        ret, track_window = cv.meanShift(dst, track_window, term_crit)<br>        # Draw it on image<br>        x,y,w,h = track_window<br>        img2 = cv.rectangle(frame, (x,y), (x+w,y+h), 255,2)<br>        cv.imshow(&#39;img2&#39;,img2)<br>        k = cv.waitKey(30) &amp; 0xff<br>        if k == 27:<br>            break<br>    else:<br>        break</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/303/0*VA_AGZSrO7ksnumi.jpg" /></figure><p><strong>CamShift Motion Tracking:</strong></p><p>Our window always has the same size whether the car is very far or very close to the camera. That is not good. We need to adapt the window size with size and rotation of the target. Once again, the solution came from “OpenCV Labs” and it is called CAMshift (Continuously Adaptive Meanshift) published by Gary Bradsky in his paper “Computer Vision Face Tracking for Use in a Perceptual User Interface” in 1998 <a href="https://docs.opencv.org/3.4/d0/de3/citelist.html#CITEREF_Bradski98"><strong>[33]</strong></a> .</p><p>It applies meanshift first. Once meanshift converges, it updates the size of the window as, it also calculates the orientation of the best fitting ellipse to it. Again it applies the meanshift with new scaled search window and previous window location. The process continues until the required accuracy is met.</p><p><strong>Code Implementation</strong></p><pre>import numpy as np<br>import cv2 as cv<br>import argparse<br>parser = argparse.ArgumentParser(description=&#39;This sample demonstrates the camshift algorithm. \<br>                                              The example file can be downloaded from: \<br>                                              https://www.bogotobogo.com/python/OpenCV_Python/images/mean_shift_tracking/slow_traffic_small.mp4&#39;)<br>parser.add_argument(&#39;image&#39;, type=str, help=&#39;path to image file&#39;)<br>args = parser.parse_args()<br>cap = cv.VideoCapture(args.image)<br># take first frame of the video<br>ret,frame = cap.read()<br># setup initial location of window<br>x, y, w, h = 300, 200, 100, 50 # simply hardcoded the values<br>track_window = (x, y, w, h)<br># set up the ROI for tracking<br>roi = frame[y:y+h, x:x+w]<br>hsv_roi =  cv.cvtColor(roi, cv.COLOR_BGR2HSV)<br>mask = cv.inRange(hsv_roi, np.array((0., 60.,32.)), np.array((180.,255.,255.)))<br>roi_hist = cv.calcHist([hsv_roi],[0],mask,[180],[0,180])<br>cv.normalize(roi_hist,roi_hist,0,255,cv.NORM_MINMAX)<br># Setup the termination criteria, either 10 iteration or move by at least 1 pt<br>term_crit = ( cv.TERM_CRITERIA_EPS | cv.TERM_CRITERIA_COUNT, 10, 1 )<br>while(1):<br>    ret, frame = cap.read()<br>    if ret == True:<br>        hsv = cv.cvtColor(frame, cv.COLOR_BGR2HSV)<br>        dst = cv.calcBackProject([hsv],[0],roi_hist,[0,180],1)<br>        # apply camshift to get the new location<br>        ret, track_window = cv.CamShift(dst, track_window, term_crit)<br>        # Draw it on image<br>        pts = cv.boxPoints(ret)<br>        pts = np.int0(pts)<br>        img2 = cv.polylines(frame,[pts],True, 255,2)<br>        cv.imshow(&#39;img2&#39;,img2)<br>        k = cv.waitKey(30) &amp; 0xff<br>        if k == 27:<br>            break<br>    else:<br>        break</pre><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/362/0*HyaRiSyvWG8y_bcj.jpg" /></figure><p><strong>Object Tracking with Optical Flow:</strong></p><p>Optical flow is the pattern of apparent motion of image objects between two consecutive frames caused by the movement of object or camera. It is 2D vector field where each vector is a displacement vector showing the movement of points from first frame to second. Consider the image below (Image Courtesy: <a href="http://en.wikipedia.org/wiki/Optical_flow">Wikipedia article on Optical Flow</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/437/0*wutq8SqvSOdMUMD-.jpg" /></figure><p>It shows a ball moving in 5 consecutive frames. The arrow shows its displacement vector. Optical flow has many applications in areas like :</p><ul><li>Structure from Motion</li><li>Video Compression</li><li>Video Stabilization …</li></ul><p>Optical flow works on several assumptions:</p><ol><li>The pixel intensities of an object do not change between consecutive frames.</li><li>Neighbouring pixels have similar motion.</li></ol><p>Consider a pixel I(x,y,t) in first frame (Check a new dimension, time, is added here. Earlier we were working with images only, so no need of time). It moves by distance (dx,dy) in next frame taken after dt time. So since those pixels are the same and intensity does not change, we can say,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wHwKQyLdmvKwpw31mFiF5g.png" /></figure><p>Code Implementation:</p><pre># Load video stream, short clip<br>#cap = cv2.VideoCapture(&#39;walking_short_clip.mp4&#39;)<br><br># Load video stream, long clip<br>cap = cv2.VideoCapture(&#39;walking.avi&#39;)<br><br># Get the height and width of the frame (required to be an interger)<br>width = int(cap.get(3)) <br>height = int(cap.get(4))<br><br># Define the codec and create VideoWriter object. The output is stored in &#39;*.avi&#39; file.<br>out = cv2.VideoWriter(&#39;optical_flow_walking.avi&#39;, cv2.VideoWriter_fourcc(&#39;M&#39;,&#39;J&#39;,&#39;P&#39;,&#39;G&#39;), 30, (width, height))<br><br># Set parameters for ShiTomasi corner detection<br>feature_params = dict( maxCorners = 100,<br>                       qualityLevel = 0.3,<br>                       minDistance = 7,<br>                       blockSize = 7 )<br><br># Set parameters for lucas kanade optical flow<br>lucas_kanade_params = dict( winSize  = (15,15),<br>                  maxLevel = 2,<br>                  criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))<br><br># Create some random colors<br># Used to create our trails for object movement in the image <br>color = np.random.randint(0,255,(100,3))<br><br># Take first frame and find corners in it<br>ret, prev_frame = cap.read()<br>prev_gray = cv2.cvtColor(prev_frame, cv2.COLOR_BGR2GRAY)<br><br># Find inital corner locations<br>prev_corners = cv2.goodFeaturesToTrack(prev_gray, mask = None, **feature_params)<br><br># Create a mask image for drawing purposes<br>mask = np.zeros_like(prev_frame)<br><br>while(1):<br>    ret, frame = cap.read()<br><br>    if ret == True:<br>      frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)<br><br>      # calculate optical flow<br>      new_corners, status, errors = cv2.calcOpticalFlowPyrLK(prev_gray, <br>                                                            frame_gray, <br>                                                            prev_corners, <br>                                                            None, <br>                                                            **lucas_kanade_params)<br><br>      # Select and store good points<br>      good_new = new_corners[status==1]<br>      good_old = prev_corners[status==1]<br><br>      # Draw the tracks<br>      for i,(new,old) in enumerate(zip(good_new, good_old)):<br>          a, b = new.ravel()<br>          c, d = old.ravel()<br>          mask = cv2.line(mask, (a,b),(c,d), color[i].tolist(), 2)<br>          frame = cv2.circle(frame, (a,b), 5, color[i].tolist(),-1)<br>          <br>      img = cv2.add(frame,mask)<br><br>      # Save Video<br>      out.write(img)<br>      # Show Optical Flow<br>      #imshow(&#39;Optical Flow - Lucas-Kanade&#39;,img)<br><br>      # Now update the previous frame and previous points<br>      prev_gray = frame_gray.copy()<br>      prev_corners = good_new.reshape(-1,1,2)<br><br>    else:<br>      break<br>    <br>cap.release()<br>out.release()</pre><p><strong>Dense Optical Flow</strong></p><p>Lucas-Kanade method computes optical flow for a sparse feature set (in our example, corners detected using Shi-Tomasi algorithm). OpenCV.js provides another algorithm to find the dense optical flow. It computes the optical flow for all the points in the frame. It is based on Gunnar Farneback’s algorithm which is explained in “Two-Frame Motion Estimation Based on Polynomial Expansion” by Gunnar Farneback in 2003.</p><p>Code Implementation:</p><pre># Load video stream, short clip<br>#cap = cv2.VideoCapture(&#39;walking_short_clip.mp4&#39;)<br><br># Load video stream, long clip<br>cap = cv2.VideoCapture(&#39;walking.avi&#39;)<br><br># Get the height and width of the frame (required to be an interger)<br>width = int(cap.get(3)) <br>height = int(cap.get(4))<br><br># Define the codec and create VideoWriter object. The output is stored in &#39;*.avi&#39; file.<br>out = cv2.VideoWriter(&#39;dense_optical_flow_walking.avi&#39;, cv2.VideoWriter_fourcc(&#39;M&#39;,&#39;J&#39;,&#39;P&#39;,&#39;G&#39;), 30, (width, height))<br><br># Get first frame<br>ret, first_frame = cap.read()<br>previous_gray = cv2.cvtColor(first_frame, cv2.COLOR_BGR2GRAY)<br>hsv = np.zeros_like(first_frame)<br>hsv[...,1] = 255<br><br>while True:<br>    <br>    # Read of video file<br>    ret, frame2 = cap.read()<br><br>    if ret == True:<br>      next = cv2.cvtColor(frame2,cv2.COLOR_BGR2GRAY)<br><br>      # Computes the dense optical flow using the Gunnar Farneback’s algorithm<br>      flow = cv2.calcOpticalFlowFarneback(previous_gray, next, <br>                                          None, 0.5, 3, 15, 3, 5, 1.2, 0)<br><br>      # use flow to calculate the magnitude (speed) and angle of motion<br>      # use these values to calculate the color to reflect speed and angle<br>      magnitude, angle = cv2.cartToPolar(flow[...,0], flow[...,1])<br>      hsv[...,0] = angle * (180 / (np.pi/2))<br>      hsv[...,2] = cv2.normalize(magnitude, None, 0, 255, cv2.NORM_MINMAX)<br>      final = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)<br><br>      # Save Video<br>      out.write(final)<br>      # Show our demo of Dense Optical Flow<br>      #imshow(&#39;Dense Optical Flow&#39;, final)<br>      <br>      # Store current image as previous image<br>      previous_gray = next<br><br>    else:<br>      break<br>    <br>cap.release()<br>out.release()</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42eb1e6a5443" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Analysis and Transformations]]></title>
            <link>https://medium.com/@RiwajNeupane/image-analysis-and-transformations-38664e582fe9?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/38664e582fe9</guid>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[image-analysis]]></category>
            <category><![CDATA[image-processing]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Thu, 11 Jan 2024 11:45:52 GMT</pubDate>
            <atom:updated>2024-01-11T11:45:52.648Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Perspective Transformations:</strong></p><p>In <strong>Perspective Transformation, </strong>we can change the perspective of a given image or video for getting better insights into the required information. In Perspective Transformation, we need to provide the points on the image from which want to gather information by changing the perspective. We also need to provide the points inside which we want to display our image. Then, we get the perspective transform from the two given sets of points and wrap it with the original image.</p><p><strong><em>cv2.getPerspectiveTransform method</em></strong></p><p><strong><em>Syntax</em></strong><em>: cv2.getPerspectiveTransform(src, dst)</em></p><p><strong><em>Parameters</em></strong><em>:</em></p><ul><li><strong><em>src</em></strong><em>: Coordinates of quadrangle vertices in the source image.</em></li><li><strong><em>dst</em></strong><em>: Coordinates of the corresponding quadrangle vertices in the destination image.</em></li></ul><p><strong><em>cv2.wrapPerspective method</em></strong></p><p><strong><em>Syntax</em></strong><em>: cv2.warpPerspective(src, dst, dsize)</em></p><p><strong><em>Parameters</em></strong><em>:</em></p><ul><li><strong><em>src</em></strong><em>: Source Image</em></li><li><strong><em>dst</em></strong><em>: output image that has the size dsize and the same type as src.</em></li><li><strong><em>dsize</em></strong><em>: size of output image</em></li></ul><p>Code Implementation:</p><pre># import necessary libraries <br><br>import cv2 <br>import numpy as np <br><br># Turn on Laptop&#39;s webcam<br>cap = cv2.VideoCapture(0)<br><br>while True:<br> <br> ret, frame = cap.read()<br><br> # Locate points of the documents<br> # or object which you want to transform<br> pts1 = np.float32([[0, 260], [640, 260],<br>     [0, 400], [640, 400]])<br> pts2 = np.float32([[0, 0], [400, 0],<br>     [0, 640], [400, 640]])<br> <br> # Apply Perspective Transform Algorithm<br> matrix = cv2.getPerspectiveTransform(pts1, pts2)<br> result = cv2.warpPerspective(frame, matrix, (500, 600))<br> <br> # Wrap the transformed image<br> cv2.imshow(&#39;frame&#39;, frame) # Initial Capture<br> cv2.imshow(&#39;frame1&#39;, result) # Transformed Capture<br><br> if cv2.waitKey(24) == 27:<br>  break<br><br>cap.release()<br>cv2.destroyAllWindows()</pre><p><strong>Result:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ip7Yigp-Cta2Ev_L.JPG" /></figure><p><strong>K Means Clustering for dominant colors:</strong></p><p>The goal is to partition <em>n</em> data points into <em>k</em> clusters. Each of the <em>n</em> data points will be assigned to a cluster with the nearest mean. The mean of each cluster is called its “centroid” or “center”.</p><p>Overall, applying k-means yields <em>k</em> separate clusters of the original <em>n</em> data points. Data points inside a particular cluster are considered to be “more similar” to each other than data points that belong to other clusters.</p><p>In our case, we will be clustering the pixel intensities of a RGB image. Given a <em>MxN</em> size image, we thus have <em>MxN</em> pixels, each consisting of three components: Red, Green, and Blue respectively.</p><p>We will treat these <em>MxN</em> pixels as our data points and cluster them using k-means.</p><p>Pixels that belong to a given cluster will be more similar in color than pixels belonging to a separate cluster.</p><p><strong>Code Implementation:</strong></p><pre>from sklearn.cluster import KMeans<br><br>image = cv2.imread(&#39;images/tobago.jpg&#39;)<br>imshow(&quot;Input&quot;, image)<br><br># We reshape our image into a list of RGB pixels<br>image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)<br>print(image.shape)<br>image = image.reshape((image.shape[0] * image.shape[1], 3))<br>print(image.shape)<br><br>number_of_clusters = 5<br>clt = KMeans(number_of_clusters)<br>clt.fit(image)<br><br>hist = centroidHistogram(clt)<br>bar = plotColors(hist, clt.cluster_centers_)<br><br># show our color bart<br>plt.figure()<br>plt.axis(&quot;off&quot;)<br>plt.imshow(bar)<br>plt.show()</pre><h4>Result:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*scmOsRsxXyG9i1mZ" /></figure><p><strong>Figure: </strong>Applying OpenCV and k-means clustering to find the five most dominant colors in a RGB image.</p><p><strong>Comparing images using MSE and Structural Similarity:</strong></p><p><strong>Mean Squared Error (MSE)</strong></p><p>The MSE between the two images is the sum of the squared difference between the two images. This can easily be implemented with numpy.</p><p>The lower the MSE the more similar the images are.</p><pre>def mse(image1, image2):<br>    # Images must be of the same dimension<br>    error = np.sum((image1.astype(&quot;float&quot;) - image2.astype(&quot;float&quot;)) ** 2)<br>    error /= float(image1.shape[0] * image1.shape[1])<br><br>    return error</pre><p><strong>Structural Similarity Index:</strong></p><p>In order to compute the difference between two images we’ll be utilizing the Structural Similarity Index, first introduced by Wang et al. in their 2004 paper, <strong><em>Image Quality Assessment: From Error Visibility to Structural Similarity</em></strong>. This method is already implemented in the <strong>scikit-image</strong> library for image processing.</p><p><strong>Computing image difference</strong></p><p>Can you spot the difference between these two images?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*s-OF5vCOv335w5CR" /></figure><p>If you take a second to study the two credit cards, you’ll notice that the MasterCard logo is present on the <em>left</em> image but has been Photoshopped out from the <em>right</em> image.</p><p>You may have noticed this difference immediately, or it may have taken you a few seconds. Either way, this demonstrates an important aspect of comparing image differences — <em>sometimes image differences are subtle — </em>so subtle that the naked eye struggles to immediately comprehend the difference</p><p>Code Implementation:</p><pre>def compare(image1, image2):<br>  image1 = cv2.cvtColor(image1, cv2.COLOR_BGR2GRAY)<br>  image2 = cv2.cvtColor(image2, cv2.COLOR_BGR2GRAY)<br>  print(&#39;MSE = {:.2f}&#39;.format(mse(image1, image2)))<br>  print(&#39;SS = {:.2f}&#39;.format(structural_similarity(image1, image2)))</pre><p><strong>Filtering Colors:</strong></p><p>Colour segmentation or color filtering is widely used in OpenCV for identifying specific objects/regions having a specific color. The most widely used color space is RGB color space, it is called an <strong>additive color space</strong> as the three color shades add up to give color to the image. To identify a region of a specific color, put the threshold and create a mask to separate the different colors. HSV color space is much more useful for this purpose as the colors in HSV space are much more localized thus can be easily separated. Color Filtering has many applications and uses cases such as in Cryptography, infrared analysis, food preservation of perishable foods, etc. In such cases, the concepts of Image processing can be used to find out or extract out regions of a particular color. <br>For color segmentation, all we need is the threshold values or the knowledge of the lower bound and upper bound range of colors in one of the color spaces. It works best in the Hue-Saturation-Value color space. <br>After specifying the range of color to be segmented, it is needed to create a mask accordingly and by using it, a particular region of interest can be separated out.</p><p><strong>Code Implementation:</strong></p><pre>image = cv2.imread(&#39;images/truck.jpg&#39;)<br><br># define range of BLUE color in HSV<br>lower = np.array([90,0,0])<br>upper = np.array([135,255,255])<br><br># Convert image from RBG/BGR to HSV so we easily filter<br>hsv_img = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)<br><br># Use inRange to capture only the values between lower &amp; upper <br>mask = cv2.inRange(hsv_img, lower, upper)<br><br># Perform Bitwise AND on mask and our original frame<br>res = cv2.bitwise_and(image, image, mask=mask)<br><br>imshow(&#39;Original&#39;, image)  <br>imshow(&#39;mask&#39;, mask)<br>imshow(&#39;Filtered Color Only&#39;, res)</pre><p><strong>Original Image-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*MQVc3_CC6R_0IN68.png" /></figure><p><strong>Masked Image-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*XSvUOXIov8H2gZt0.png" /></figure><p><strong>Blue Color segmented regions-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*J7rcyBW_Ez0njY3U.png" /></figure><p><strong>Watershed Algorithm:</strong></p><p><strong>Image segmentation</strong> is the process of dividing an image into several disjoint small local areas or cluster sets according to certain rules and principles. The<strong> watershed algorithm</strong> is a computer vision technique used for image region segmentation. The segmentation process will take the similarity with adjacent pixels of the image as an important reference to connect pixels with similar spatial positions and gray values. Constitute a closed contour(outline), and this closure is an important feature of the watershed algorithm. In short, it is an algorithm that correctly determines the “<strong>outline of an object</strong>“. In this article, you will learn Image Segmentation with Watershed Algorithm in OpenCV using Python.</p><p><strong>Working of Watershed Algorithm</strong></p><p>The watershed algorithm uses topographic information to divide an image into multiple segments or regions. The algorithm views an image as a topographic surface, each pixel representing a different height. The watershed algorithm uses this information to identify catchment basins, similar to how water would collect in valleys in a real topographic map.</p><p>The watershed algorithm identifies the local minima, or the lowest points, in the image. These points are then marked as markers. The algorithm then floods the image with different colors, starting from these marked markers. As the color spreads, it fills up the catchment basins until it reaches the boundaries of the objects or regions in the image.</p><p>The <strong>catchment basin</strong> in the watershed algorithm refers to a region in the image that is filled by the spreading color starting from a marker. The catchment basin is defined by the boundaries of the object or region in the image and the local minima in the intensity values of the pixels. The algorithm uses the catchment basins to divide the image into separate regions and then identifies the boundaries between the basins to create a segmentation of the image for object recognition, image analysis, and feature extraction tasks.</p><p><strong>Image Segmentation with Watershed Algorithm — OpenCV Python</strong></p><p><strong>Image segmentation</strong> is the process of dividing an image into several disjoint small local areas or cluster sets according to certain rules and principles. The<strong> watershed algorithm</strong> is a computer vision technique used for image region segmentation. The segmentation process will take the similarity with adjacent pixels of the image as an important reference to connect pixels with similar spatial positions and gray values. Constitute a closed contour(outline), and this closure is an important feature of the watershed algorithm. In short, it is an algorithm that correctly determines the “<strong>outline of an object</strong>“. In this article, you will learn Image Segmentation with Watershed Algorithm in OpenCV using Python.</p><p><strong>Working of Watershed Algorithm</strong></p><p>The watershed algorithm uses topographic information to divide an image into multiple segments or regions. The algorithm views an image as a topographic surface, each pixel representing a different height. The watershed algorithm uses this information to identify catchment basins, similar to how water would collect in valleys in a real topographic map.</p><p>The watershed algorithm identifies the local minima, or the lowest points, in the image. These points are then marked as markers. The algorithm then floods the image with different colors, starting from these marked markers. As the color spreads, it fills up the catchment basins until it reaches the boundaries of the objects or regions in the image.</p><p>The <strong>catchment basin</strong> in the watershed algorithm refers to a region in the image that is filled by the spreading color starting from a marker. The catchment basin is defined by the boundaries of the object or region in the image and the local minima in the intensity values of the pixels. The algorithm uses the catchment basins to divide the image into separate regions and then identifies the boundaries between the basins to create a segmentation of the image for object recognition, image analysis, and feature extraction tasks.</p><p>The flooding process continues until all catchment basins have been filled with different colors, creating a segmentation of the image. The resulting segments or regions are assigned unique colors, which can then be used to identify different objects or features in the image.</p><p>The whole process of the watershed algorithm can be summarized in the following steps:</p><ul><li><strong>Marker placement:</strong> The first step is to place markers on the local minima, or the lowest points, in the image. These markers serve as the starting points for the flooding process.</li><li><strong>Flooding</strong>: The algorithm then floods the image with different colors, starting from the markers. As the color spreads, it fills up the catchment basins until it reaches the boundaries of the objects or regions in the image.</li><li><strong>Catchment basin formation</strong>: As the color spreads, the catchment basins are gradually filled, creating a segmentation of the image. The resulting segments or regions are assigned unique colors, which can then be used to identify different objects or features in the image.</li><li><strong>Boundary identification</strong>: The watershed algorithm uses the boundaries between the different colored regions to identify the objects or regions in the image. The resulting segmentation can be used for object recognition, image analysis, and feature extraction tasks.</li></ul><p>Implementation:</p><p>Step1: Thresholding Image</p><pre># Load image<br>img = cv2.imread(&#39;water_coins.jpg&#39;)<br>imshow(&quot;Original image&quot;, img)<br><br># Grayscale<br>gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)<br><br># Threshold using OTSU<br>ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)<br><br>imshow(&quot;Thresholded&quot;, thresh)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/219/0*A8QL9lfjCCa4FRV1.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/252/0*-G0PtQWnl3LnXCQz.png" /></figure><p>Step2: Removing the Touching Masks</p><pre># noise removal<br>kernel = np.ones((3,3), np.uint8)<br>opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN,kernel, iterations = 2)<br><br># sure background area<br>sure_bg = cv2.dilate(opening, kernel, iterations=3)<br><br># Finding sure foreground area<br>dist_transform = cv2.distanceTransform(opening, cv2.DIST_L2,5)<br>ret, sure_fg = cv2.threshold(dist_transform, 0.7 * dist_transform.max(), 255, 0)<br><br># Finding unknown region<br>sure_fg = np.uint8(sure_fg)<br>unknown = cv2.subtract(sure_bg, sure_fg)<br><br>imshow(&quot;SureFG&quot;, sure_fg)<br>imshow(&quot;SureBG&quot;, sure_bg)<br>imshow(&quot;unknown&quot;, unknown)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/0*hB1N5Har2A9xrcI2.png" /></figure><p>Step3: Marker Labelling</p><pre># Marker labelling<br># Connected Components determines the connectivity of blob-like regions in a binary image.<br>ret, markers = cv2.connectedComponents(sure_fg)<br><br># Add one to all labels so that sure background is not 0, but 1<br>markers = markers+1<br><br># Now, mark the region of unknown with zero<br>markers[unknown==255] = 0<br><br>markers = cv2.watershed(img,markers)<br>img[markers == -1] = [255,0,0]<br><br>imshow(&quot;img&quot;, img)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/252/0*ntmBJbOtZd4fxIrR.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=38664e582fe9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Haar Cascade Classifier]]></title>
            <link>https://medium.com/@RiwajNeupane/haar-cascade-classifier-5092ec02ee8d?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/5092ec02ee8d</guid>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[haar-cascades]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Tue, 09 Jan 2024 10:22:06 GMT</pubDate>
            <atom:updated>2024-01-09T10:22:06.845Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Theory</strong></p><p>Object Detection using Haar feature-based cascade classifiers is an effective object detection method proposed by Paul Viola and Michael Jones in their paper, “Rapid Object Detection using a Boosted Cascade of Simple Features” in 2001. It is a machine learning based approach where a cascade function is trained from a lot of positive and negative images. It is then used to detect objects in other images.</p><p>Here we will work with face detection. Initially, the algorithm needs a lot of positive images (images of faces) and negative images (images without faces) to train the classifier. Then we need to extract features from it. For this, Haar features shown in the below image are used. They are just like our convolutional kernel. Each feature is a single value obtained by subtracting sum of pixels under the white rectangle from sum of pixels under the black rectangle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/0*yL9CJuF-hHgL-pUP.jpg" /></figure><p>Now, all possible sizes and locations of each kernel are used to calculate lots of features. (Just imagine how much computation it needs? Even a 24x24 window results over 160000 features). For each feature calculation, we need to find the sum of the pixels under white and black rectangles. To solve this, they introduced the integral image. However large your image, it reduces the calculations for a given pixel to an operation involving just four pixels. Nice, isn’t it? It makes things super-fast.</p><p>But among all these features we calculated, most of them are irrelevant. For example, consider the image below. The top row shows two good features. The first feature selected seems to focus on the property that the region of the eyes is often darker than the region of the nose and cheeks. The second feature selected relies on the property that the eyes are darker than the bridge of the nose. But the same windows applied to cheeks or any other place is irrelevant. So how do we select the best features out of 160000+ features? It is achieved by <strong>Adaboost</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/334/0*MjY8IoZQkqNcK633.png" /></figure><p>For this, we apply each and every feature on all the training images. For each feature, it finds the best threshold which will classify the faces to positive and negative. Obviously, there will be errors or misclassifications. We select the features with minimum error rate, which means they are the features that most accurately classify the face and non-face images. (The process is not as simple as this. Each image is given an equal weight in the beginning. After each classification, weights of misclassified images are increased. Then the same process is done. New error rates are calculated. Also new weights. The process is continued until the required accuracy or error rate is achieved or the required number of features are found).</p><p>The final classifier is a weighted sum of these weak classifiers. It is called weak because it alone can’t classify the image, but together with others forms a strong classifier. The paper says even 200 features provide detection with 95% accuracy. Their final setup had around 6000 features. (Imagine a reduction from 160000+ features to 6000 features. That is a big gain).</p><p>So now you take an image. Take each 24x24 window. Apply 6000 features to it. Check if it is face or not. Wow.. Isn’t it a little inefficient and time consuming? Yes, it is. The authors have a good solution for that.</p><p>In an image, most of the image is non-face region. So it is a better idea to have a simple method to check if a window is not a face region. If it is not, discard it in a single shot, and don’t process it again. Instead, focus on regions where there can be a face. This way, we spend more time checking possible face regions.</p><p>For this they introduced the concept of <strong>Cascade of Classifiers</strong>. Instead of applying all 6000 features on a window, the features are grouped into different stages of classifiers and applied one-by-one. (Normally the first few stages will contain very many fewer features). If a window fails the first stage, discard it. We don’t consider the remaining features on it. If it passes, apply the second stage of features and continue the process. The window which passes all stages is a face region. How is that plan!</p><p>The authors’ detector had 6000+ features with 38 stages with 1, 10, 25, 25 and 50 features in the first five stages. (The two features in the above image are actually obtained as the best two features from Adaboost). According to the authors, on average 10 features out of 6000+ are evaluated per sub-window.</p><p>So this is a simple intuitive explanation of how Viola-Jones face detection works. Read the paper for more details or check out the references in the Additional Resources section.</p><p><strong>OpenCV Implementation:</strong></p><p>Face Detection:</p><pre>face_classifier = cv2.CascadeClassifier(&#39;Haarcascades/haarcascade_frontalface_default.xml&#39;)<br><br>image = cv2.imread(&#39;/content/trump.jpg&#39;)<br>gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)<br>faces = face_classifier.detectMultiScale(gray, scaleFactor = 1.1, minNeighbors = 5)<br><br># When no faces detected, face_classifier returns and empty tuple<br>if faces is ():<br>    print(&quot;No faces found&quot;)<br><br># We iterate through our faces array and draw a rectangle<br># over each face in faces<br>for (x,y,w,h) in faces:<br>    cv2.rectangle(image, (x,y), (x+w,y+h), (127,0,255), 2)<br><br>imshow(&#39;Face Detection&#39;, image)</pre><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*RW8q-jM4-KNjMVIbSjMfOg.png" /></figure><p><strong>Face and Eye Detection:</strong></p><pre>import numpy as np<br>import cv2<br> <br>face_classifier = cv2.CascadeClassifier(&#39;Haarcascades/haarcascade_frontalface_default.xml&#39;)<br>eye_classifier = cv2.CascadeClassifier(&#39;Haarcascades/haarcascade_eye.xml&#39;)<br> <br>img =  cv2.imread(&#39;/content/trump.jpg&#39;)<br><br>gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)<br><br>faces = face_classifier.detectMultiScale(gray,1.1,3)<br><br># When no faces detected, face_classifier returns and empty tuple<br>if faces is ():<br>    print(&quot;No Face Found&quot;)<br><br>for (x,y,w,h) in faces:<br>    cv2.rectangle(img,(x,y),(x+w,y+h),(127,0,255),2)<br>    roi_gray = gray[y:y+h, x:x+w]<br>    roi_color = img[y:y+h, x:x+w]<br>    eyes = eye_classifier.detectMultiScale(roi_gray,1.2,2<br>                                           <br>                                           )<br>    for (ex,ey,ew,eh) in eyes:<br>        cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(255,255,0),2)<br><br>imshow(&#39;Eye &amp; Face Detection&#39;,img)<br></pre><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*-rMBbrf6aB1SyApoTlY9GQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5092ec02ee8d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Line, Circle and Blob Detection]]></title>
            <link>https://medium.com/@RiwajNeupane/line-circle-and-blob-detection-e5a3487dcb1a?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5a3487dcb1a</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[image-segmentation]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Tue, 09 Jan 2024 08:24:24 GMT</pubDate>
            <atom:updated>2024-01-09T10:24:34.336Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>HoughLineTransform:</strong></p><ol><li>The Hough Line Transform is a transform used to detect straight lines.</li><li>To apply the Transform, first an edge detection pre-processing is desirable.</li></ol><p><strong>How does it work?</strong></p><ol><li>As you know, a line in the image space can be expressed with two variables. For example:</li><li>In the <strong>Cartesian coordinate system:</strong> Parameters: (m,b).</li><li>In the <strong>Polar coordinate system:</strong> Parameters: (r,θ)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*LIRX0zi_HEfhSi_l.jpg" /></figure><p><strong>Standard and Probabilistic Hough Line Transform</strong></p><p>OpenCV implements two kind of Hough Line Transforms:</p><p>a. <strong>The Standard Hough Transform</strong></p><ul><li>It consists in pretty much what we just explained in the previous section. It gives you as result a vector of couples (θ,rθ)</li><li>In OpenCV it is implemented with the function <strong>HoughLines()</strong></li></ul><p>b. <strong>The Probabilistic Hough Line Transform</strong></p><ul><li>A more efficient implementation of the Hough Line Transform. It gives as output the extremes of the detected lines (x0,y0,x1,y1)</li><li>In OpenCV it is implemented with the function <strong>HoughLinesP()</strong></li></ul><p><strong>Code Implementation:</strong></p><p><strong>Standard Hough Line Transform:</strong></p><p>First, you apply the Transform:</p><p># Standard Hough Line Transform</p><p>lines = <a href="https://docs.opencv.org/3.4/dd/d1a/group__imgproc__feature.html#ga46b4e588934f6c8dfd509cc6e0e4545a">cv.HoughLines</a>(dst, 1, np.pi / 180, 150, None, 0, 0)</p><ul><li>with the following arguments:</li><li><em>dst</em>: Output of the edge detector. It should be a grayscale image (although in fact it is a binary one)</li><li><em>lines</em>: A vector that will store the parameters (r,θ) of the detected lines</li><li><em>rho</em> : The resolution of the parameter r in pixels. We use <strong>1</strong> pixel.</li><li><em>theta</em>: The resolution of the parameter θ in radians. We use <strong>1 degree</strong> (CV_PI/180)</li><li><em>threshold</em>: The minimum number of intersections to “*detect*” a line</li><li><em>srn</em> and <em>stn</em>: Default parameters to zero.</li></ul><p>And then you display the result by drawing the lines.</p><pre># Draw the lines<br>if lines is not None:<br>for i in range(0, len(lines)):<br>rho = lines[i][0][0]<br>theta = lines[i][0][1]<br>a = math.cos(theta)<br>b = math.sin(theta)<br>x0 = a * rho<br>y0 = b * rho<br>pt1 = (int(x0 + 1000*(-b)), int(y0 + 1000*(a)))<br>pt2 = (int(x0–1000*(-b)), int(y0–1000*(a)))<br>cv.line(cdst, pt1, pt2, (0,0,255), 3, cv.LINE_AA)<br><br><br>Probabilistic Hough Line Transform:<br># Probabilistic Line Transform<br>linesP = cv.HoughLinesP(dst, 1, np.pi / 180, 50, None, 50, 10)</pre><ul><li>with the arguments:</li><li><em>dst</em>: Output of the edge detector. It should be a grayscale image (although in fact it is a binary one)</li><li><em>lines</em>: A vector that will store the parameters (xstart,ystart,xend,yend) of the detected lines</li><li><em>rho</em> : The resolution of the parameter r in pixels. We use <strong>1</strong> pixel.</li><li><em>theta</em>: The resolution of the parameter θ in radians. We use <strong>1 degree</strong> (CV_PI/180)</li><li><em>threshold</em>: The minimum number of intersections to “*detect*” a line</li><li><em>minLineLength</em>: The minimum number of points that can form a line. Lines with less than this number of points are disregarded.</li><li><em>maxLineGap</em>: The maximum gap between two points to be considered in the same line.</li></ul><p>And then you display the result by drawing the lines.</p><pre># Draw the lines<br>if linesP is not None:<br>for i in range(0, len(linesP)):<br>l = linesP[i][0]<br>cv.line(cdstP, (l[0], l[1]), (l[2], l[3]), (0,0,255), 3, cv.LINE_AA)<br>Display the original image and the detected lines:<br># Show results<br>cv.imshow(&quot;Source&quot;, src)<br>cv.imshow(&quot;Detected Lines (in red) - Standard Hough Line Transform&quot;, cdst)<br>cv.imshow(&quot;Detected Lines (in red) - Probabilistic Line Transform&quot;, cdstP)</pre><p>Using an input image such as a sudoku image. We get the following result by using the Standard Hough Line Transform:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/391/0*M31Iof3TOWL3rwip.png" /></figure><p>And by using the Probabilistic Hough Line Transform:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/0*Yb_-3l2EucPHQSjm.png" /></figure><p>You may observe that the number of lines detected vary while you change the <em>threshold</em>. The explanation is sort of evident: If you establish a higher threshold, fewer lines will be detected (since you will need more points to declare a line detected).</p><p><strong>Circle Detection:</strong></p><p><strong>cv2.HoughCircles</strong>(image, method, dp, MinDist, param1, param2, minRadius, MaxRadius)</p><ul><li>Method — currently only cv2.HOUGH_GRADIENT available</li><li>dp — Inverse ratio of accumulator resolution</li><li>MinDist — the minimum distance between the center of detected circles</li><li>param1 — Gradient value used in the edge detection</li><li>param2 — Accumulator threshold for the HOUGH_GRADIENT method (lower allows more circles to be detected (false positives))</li><li>minRadius — limits the smallest circle to this size (via radius)</li><li>MaxRadius — similarly sets the limit for the largest circles</li></ul><p><strong>Code Implementation:</strong></p><pre>rows = gray.shape[0]<br>circles = cv.HoughCircles(gray, cv.HOUGH_GRADIENT, 1, rows / 8,<br>param1=100, param2=30,<br>minRadius=1, maxRadius=30)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/0*IEkq5E60We4FefKD.png" /></figure><p><strong>Blob Detection:</strong></p><p>Blob detection is an important technique in computer vision and image processing. Blob detection is the process of identifying and localizing these regions in an image.</p><p>Blobs can be useful in a variety of applications, such as object detection, tracking, and recognition. For example, in object recognition, a blob can represent a particular feature of an object, such as an <strong>edge or a corner</strong>.</p><p><strong>Syntax</strong></p><p>cv2.SimpleBlobDetector(src, minThreshold, maxThreshold, thresholdStep)</p><p><strong>Parameters</strong></p><ul><li>src: Input image or source image</li><li>minThreshold: Minimum value for the threshold function</li><li>maxThreshold: Maximum value for the threshold function</li><li>thresholdStep: We increment the value of this parameter and add to the minThreshold parameter till the value of minThreshold is equal to the maxThreshold parameter</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*0t_kaVHeWxnrg5RUt9uUzg.png" /></figure><p><strong>Template Matching:</strong></p><p>Template Matching is a method for searching and finding the location of a template image in a larger image. OpenCV comes with a function <strong>cv.matchTemplate()</strong> for this purpose. It simply slides the template image over the input image (as in 2D convolution) and compares the template and patch of input image under the template image. Several comparison methods are implemented in OpenCV. (You can check docs for more details). It returns a grayscale image, where each pixel denotes how much does the neighbourhood of that pixel match with template.</p><p>If input image is of size (WxH) and template image is of size (wxh), output image will have a size of (W-w+1, H-h+1). Once you got the result, you can use <strong>cv.minMaxLoc()</strong> function to find where is the maximum/minimum value. Take it as the top-left corner of rectangle and take (w,h) as width and height of the rectangle. That rectangle is your region of template.</p><p><strong>Code Implementation:</strong></p><pre>import cv2<br>from matplotlib import pyplot as plt<br># Read the main image<br>image = cv2.imread(&#39;/content/messiki.jpg&#39;)<br># Display the original image<br>plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))<br>plt.title(&#39;Where is Waldo? - Original&#39;)<br>plt.show()<br># Convert the image to grayscale<br>gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)<br># Load the template image<br>template = cv2.imread(&#39;/content/messi.jpg&#39;,0)<br># Resize the template image to a smaller size if needed<br># You can adjust the dimensions based on your requirements<br>template = cv2.resize(template, (50, 50))<br># Match the template with the main image<br>result = cv2.matchTemplate(gray, template, cv2.TM_CCOEFF)<br>min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)<br># Create a bounding box around the detected region<br>top_left = max_loc<br>bottom_right = (top_left[0] + template.shape[1], top_left[1] + template.shape[0])<br>cv2.rectangle(image, top_left, bottom_right, (0, 0, 255), 2)<br># Display the result with the bounding box<br>plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))<br>plt.title(&#39;Where is Waldo? - Detected&#39;)<br>plt.show()</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5a3487dcb1a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Segmentation]]></title>
            <link>https://medium.com/@RiwajNeupane/image-segmentation-4dd7ee29a9d1?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/4dd7ee29a9d1</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[image-segmentation]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Tue, 09 Jan 2024 07:16:09 GMT</pubDate>
            <atom:updated>2024-01-09T10:28:34.537Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Contours:</strong></p><p>Contours can be explained simply as a curve joining all the continuous points (along the boundary), having same color or intensity. The contours are a useful tool for shape analysis and object detection and recognition.</p><ul><li>For better accuracy, use binary images. So before finding contours, apply threshold or canny edge detection.</li><li>Since OpenCV 3.2, <strong>findContours()</strong> no longer modifies the source image.</li><li>In OpenCV, finding contours is like finding white object from black background. So remember, object to be found should be white and background should be black.</li></ul><p><strong>Finding Contours on an image:</strong></p><pre>import numpy as np<br>import cv2 as cv<br># Read an image named &#39;test.jpg&#39; using OpenCV<br>im = cv.imread(&#39;test.jpg&#39;)<br># Convert the BGR image to grayscale<br>imgray = cv.cvtColor(im, cv.COLOR_BGR2GRAY)<br># Apply a binary threshold to the grayscale image<br># Pixels below 127 are set to 0, and pixels above or equal to 127 are set to 255<br>ret, thresh = cv.threshold(imgray, 127, 255, 0)<br># Find contours in the binary image<br># contours: a list of contours found in the image<br># hierarchy: information about the relationship between contours<br>contours, hierarchy = cv.findContours(thresh, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)</pre><p>See, there are three arguments in <strong>cv.findContours()</strong> function, first one is source image, second is contour retrieval mode, third is contour approximation method. And it outputs the contours and hierarchy. Contours is a Python list of all the contours in the image. Each individual contour is a Numpy array of (x,y) coordinates of boundary points of the object.</p><p><strong>Drawing Contours:</strong></p><p>To draw the contours, <strong>cv.drawContours</strong> function is used. It can also be used to draw any shape provided you have its boundary points. Its first argument is source image, second argument is the contours which should be passed as a Python list, third argument is index of contours (useful when drawing individual contour. To draw all contours, pass -1) and remaining arguments are color, thickness etc.</p><ul><li>To draw all the contours in an image: cv.drawContours(img, contours, -1, (0,255,0), 3)</li><li>To draw an individual contour, say 4th contour: <a href="https://docs.opencv.org/4.x/d6/d6e/group__imgproc__draw.html#ga746c0625f1781f1ffc9056259103edbc">cv.drawContours</a>(img, contours, 3, (0,255,0), 3)</li></ul><p><strong>Contour Approximation Method:</strong></p><p>This is the third argument in <a href="https://docs.opencv.org/4.x/d3/dc0/group__imgproc__shape.html#gadf1ad6a0b82947fa1fe3c3d497f260e0"><strong>cv.findContours</strong></a> function. What does it denote actually?</p><p>Above, we told that contours are the boundaries of a shape with same intensity. It stores the (x,y) coordinates of the boundary of a shape. But does it store all the coordinates ? That is specified by this contour approximation method.</p><p>If you pass <strong>cv.CHAIN_APPROX_NONE</strong>, all the boundary points are stored. But actually do we need all the points? For eg, you found the contour of a straight line. Do you need all the points on the line to represent that line? No, we need just two end points of that line. This is what <strong>cv.CHAIN_APPROX_SIMPLE</strong> does. It removes all redundant points and compresses the contour, thereby saving memory.</p><p>Below image of a rectangle demonstrate this technique. Just draw a circle on all the coordinates in the contour array (drawn in blue color). First image shows points I got with <strong>cv.CHAIN_APPROX_NONE</strong> (734 points) and second image shows the one with <strong>cv.CHAIN_APPROX_SIMPLE</strong> (only 4 points). See, how much memory it saves!!!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*AaNyoKbWRMv4gFbI.jpg" /></figure><p><strong>Hierarchy:</strong></p><p>Normally we use the <strong>cv.findContours()</strong> function to detect objects in an image, right ? Sometimes objects are in different locations. But in some cases, some shapes are inside other shapes. Just like nested figures. In this case, we call outer one as <strong>parent</strong> and inner one as <strong>child</strong>. This way, contours in an image has some relationship to each other. And we can specify how one contour is connected to each other, like, is it child of some other contour, or is it a parent etc. Representation of this relationship is called the <strong>Hierarchy</strong>.</p><p>Consider an example image below :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*ihmuF-QOiid4rqJx.png" /></figure><p>In this image, there are a few shapes which I have numbered from <strong>0–5</strong>. <em>2 and 2a</em> denotes the external and internal contours of the outermost box.</p><p>Here, contours 0,1,2 are <strong>external or outermost</strong>. We can say, they are in <strong>hierarchy-0</strong> or simply they are in <strong>same hierarchy level</strong>.</p><p>Next comes <strong>contour-2a</strong>. It can be considered as a <strong>child of contour-2</strong> (or in opposite way, contour-2 is parent of contour-2a). So let it be in <strong>hierarchy-1</strong>. Similarly contour-3 is child of contour-2 and it comes in next hierarchy. Finally contours 4,5 are the children of contour-3a, and they come in the last hierarchy level. From the way I numbered the boxes, I would say contour-4 is the first child of contour-3a (It can be contour-5 also).</p><p>I mentioned these things to understand terms like <strong>same hierarchy level</strong>, <strong>external contour</strong>, <strong>child contour</strong>, <strong>parent contour</strong>, <strong>first child</strong> etc. Now let’s get into OpenCV.</p><p><strong>Contour Retrieval Modes:</strong></p><p><strong>1. RETR_LIST</strong></p><p>This is the simplest of the four flags (from explanation point of view). It simply retrieves all the contours, but doesn’t create any parent-child relationship. <strong>Parents and kids are equal under this rule, and they are just contours</strong>. i.e they all belongs to same hierarchy level.</p><p><strong>2. RETR_EXTERNAL</strong></p><p>If you use this flag, it returns only extreme outer flags. All child contours are left behind. <strong>We can say, under this law, Only the eldest in every family is taken care of. It doesn’t care about other members of the family :)</strong>.</p><p>So, in our image, how many extreme outer contours are there? ie at hierarchy-0 level?. Only 3, ie contours 0,1,2, right? Now try to find the contours using this flag. Here also, values given to each element is same as above. Compare it with above result. Below is what I got :</p><p>hierarchy</p><p>array([[[ 1, -1, -1, -1],</p><p>[ 2, 0, -1, -1],</p><p>[-1, 1, -1, -1]]])</p><p><strong>3. RETR_CCOMP</strong></p><p>This flag retrieves all the contours and arranges them to a 2-level hierarchy. ie external contours of the object (ie its boundary) are placed in hierarchy-1. And the contours of holes inside object (if any) is placed in hierarchy-2. If any object inside it, its contour is placed again in hierarchy-1 only. And its hole in hierarchy-2 and so on.</p><p>Just consider the image of a “big white zero” on a black background. Outer circle of zero belongs to first hierarchy, and inner circle of zero belongs to second hierarchy.</p><p>We can explain it with a simple image. Here I have labelled the order of contours in red color and the hierarchy they belongs to, in green color (either 1 or 2). The order is same as the order OpenCV detects contours.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*CdWwy7ERS2xk_hkp.png" /></figure><p>So consider first contour, ie contour-0. It is hierarchy-1. It has two holes, contours 1&amp;2, and they belong to hierarchy-2. So for contour-0, Next contour in same hierarchy level is contour-3. And there is no previous one. And its first is child is contour-1 in hierarchy-2. It has no parent, because it is in hierarchy-1. So its hierarchy array is [3,-1,1,-1]</p><p>Now take contour-1. It is in hierarchy-2. Next one in same hierarchy (under the parenthood of contour-1) is contour-2. No previous one. No child, but parent is contour-0. So array is [2,-1,-1,0].</p><p>Similarly contour-2 : It is in hierarchy-2. There is not next contour in same hierarchy under contour-0. So no Next. Previous is contour-1. No child, parent is contour-0. So array is [-1,1,-1,0].</p><p>Contour — 3 : Next in hierarchy-1 is contour-5. Previous is contour-0. Child is contour-4 and no parent. So array is [5,0,4,-1].</p><p>Contour — 4 : It is in hierarchy 2 under contour-3 and it has no sibling. So no next, no previous, no child, parent is contour-3. So array is [-1,-1,-1,3].</p><p>Remaining you can fill up. This is the final answer I got:</p><p>&gt;&gt;&gt; hierarchy</p><p>array([[[ 3, -1, 1, -1],</p><p>[ 2, -1, -1, 0],</p><p>[-1, 1, -1, 0],</p><p>[ 5, 0, 4, -1],</p><p>[-1, -1, -1, 3],</p><p>[ 7, 3, 6, -1],</p><p>[-1, -1, -1, 5],</p><p>[ 8, 5, -1, -1],</p><p>[-1, 7, -1, -1]]])</p><p><strong>4. RETR_TREE</strong></p><p>It retrieves all the contours and creates a full family hierarchy list. <strong>It even tells, who is the grandpa, father, son, grandson and even beyond… :)</strong>.</p><p>For example, I took above image, rewrite the code for <a href="https://docs.opencv.org/4.x/d3/dc0/group__imgproc__shape.html#gga819779b9857cc2f8601e6526a3a5bc71ab10df56aed56c89a026580adc9431f58"><strong>cv.RETR_TREE</strong></a>, reorder the contours as per the result given by OpenCV and analyze it. Again, red letters give the contour number and green letters give the hierarchy order.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*A_1GV-vodZUC4O-g.png" /></figure><p>Take contour-0 : It is in hierarchy-0. Next contour in same hierarchy is contour-7. No previous contours. Child is contour-1. And no parent. So array is [7,-1,1,-1].</p><p>Take contour-2 : It is in hierarchy-1. No contour in same level. No previous one. Child is contour-3. Parent is contour-1. So array is [-1,-1,3,1].</p><p>And remaining, try yourself. Below is the full answer:</p><p>&gt;&gt;&gt; hierarchy</p><p>array([[[ 7, -1, 1, -1],</p><p>[-1, -1, 2, 0],</p><p>[-1, -1, 3, 1],</p><p>[-1, -1, 4, 2],</p><p>[-1, -1, 5, 3],</p><p>[ 6, -1, -1, 4],</p><p>[-1, 5, -1, 4],</p><p>[ 8, 0, -1, -1],</p><p>[-1, 7, -1, -1]]])</p><p><strong>Moments</strong>:</p><p>In image processing, computer vision and related fields, an <strong>image moment</strong> is a certain particular weighted average (moment) of the image pixels’ intensities, or a function of such moments, usually chosen to have some attractive property or interpretation.</p><p>Image moments are useful to describe objects after segmentation. Simple properties of the image which are found <em>via</em> image moments include area (or total intensity), its centroid, and information about its orientation.</p><p>The function <strong>cv.moments()</strong> gives a dictionary of all moment values calculated. See below:</p><pre>import numpy as np<br>import cv2 as cv<br>img = cv.imread(&#39;star.jpg&#39;, cv.IMREAD_GRAYSCALE)<br>assert img is not None, &quot;file could not be read, check with os.path.exists()&quot;<br>ret,thresh = cv.threshold(img,127,255,0)<br>contours,hierarchy = cv.findContours(thresh, 1, 2)<br>cnt = contours[0]<br>M = cv.moments(cnt)<br>print( M )<br><br>area = cv.contourArea(cnt)</pre><p><strong>Contour Approximation:</strong></p><p>It approximates a contour shape to another shape with less number of vertices depending upon the precision we specify. It is an implementation of Douglas-Peucker algorithm.</p><p>To understand this, suppose you are trying to find a square in an image, but due to some problems in the image, you didn’t get a perfect square, but a “bad shape” (As shown in first image below). Now you can use this function to approximate the shape. In this, second argument is called epsilon, which is maximum distance from contour to approximated contour. It is an accuracy parameter. A wise selection of epsilon is needed to get the correct output.</p><p>epsilon = 0.1*cv.arcLength(cnt,True)</p><p>approx = cv.approxPolyDP(cnt,epsilon,True)</p><p>Below, in second image, green line shows the approximated curve for epsilon = 10% of arc length. Third image shows the same for epsilon = 1% of the arc length. Third argument specifies whether curve is closed or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*b31hiO4ynbDLRrXWEFF4aQ.png" /></figure><p><strong>Approximating Contours using ApproxPolyDP</strong></p><p>Using ApproxPolyDP to approximate contours as a more defined shape</p><p>It approximates a contour shape to another shape with less number of vertices depending upon the precision we specify.</p><p><strong><em>cv2.approxPolyDP(contour, Approximation Accuracy, Closed)</em></strong></p><ul><li><strong>contour</strong> — is the individual contour we wish to approximate</li><li><strong>Approximation Accuracy</strong> — Important parameter is determining the accuracy of the approximation. Small values give precise- approximations, large values give more generic approximation. A good rule of thumb is less than 5% of the contour perimeter</li><li><strong>Closed</strong> — a Boolean value that states whether the approximate contour should be open or closed</li></ul><p><strong>Convex Hull</strong></p><p>Convex Hull will look similar to contour approximation, but it is not (Both may provide same results in some cases). Here, <a href="https://docs.opencv.org/4.x/d3/dc0/group__imgproc__shape.html#ga014b28e56cb8854c0de4a211cb2be656"><strong>cv.convexHull()</strong></a> function checks a curve for convexity defects and corrects it. Generally speaking, convex curves are the curves which are always bulged out, or at-least flat. And if it is bulged inside, it is called convexity defects. For example, check the below image of hand. Red line shows the convex hull of hand. The double-sided arrow marks shows the convexity defects, which are the local maximum deviations of hull from contours.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/279/0*9ijeUAVwcCTPdJ1w.jpg" /></figure><p>There is a little bit things to discuss about it its syntax:</p><p>hull = <a href="https://docs.opencv.org/4.x/d3/dc0/group__imgproc__shape.html#ga014b28e56cb8854c0de4a211cb2be656">cv.convexHull</a>(points[, hull[, clockwise[, returnPoints]]])</p><p>Arguments details:</p><ul><li><strong>points</strong> are the contours we pass into.</li><li><strong>hull</strong> is the output, normally we avoid it.</li><li><strong>clockwise</strong> : Orientation flag. If it is True, the output convex hull is oriented clockwise. Otherwise, it is oriented counter-clockwise.</li><li><strong>returnPoints</strong> : By default, True. Then it returns the coordinates of the hull points. If False, it returns the indices of contour points corresponding to the hull points.</li></ul><p>So to get a convex hull as in above image, following is sufficient:</p><p>hull = cv.convexHull(cnt)</p><p>But if you want to find convexity defects, you need to pass returnPoints = False. To understand it, we will take the rectangle image above. First I found its contour as cnt. Now I found its convex hull with returnPoints = True, I got following values: [[[234 202]], [[ 51 202]], [[ 51 79]], [[234 79]]] which are the four corner points of rectangle. Now if do the same with returnPoints = False, I get following result: [[129],[ 67],[ 0],[142]]. These are the indices of corresponding points in contours.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4dd7ee29a9d1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Morphological Transformations]]></title>
            <link>https://medium.com/@RiwajNeupane/morphological-transformations-ea57cf922ac4?source=rss-ab66ffa56064------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea57cf922ac4</guid>
            <category><![CDATA[image-processing]]></category>
            <dc:creator><![CDATA[Riwaj Neupane]]></dc:creator>
            <pubDate>Mon, 08 Jan 2024 11:02:00 GMT</pubDate>
            <atom:updated>2024-01-09T10:29:38.048Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Morphological Transformations</strong></p><p>Morphological transformations are some simple operations based on the image shape. It is normally performed on binary images. It needs two inputs, one is our original image, second one is called <strong>structuring element</strong> or <strong>kernel</strong> which decides the nature of operation. Two basic morphological operators are Erosion and Dilation. Then its variant forms like Opening, Closing, Gradient etc also comes into play. We will see them one-by-one with help of following image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/112/1*n_r_ff1pqzcNYTJAe3zx9Q.png" /><figcaption>Original Image</figcaption></figure><p><strong>1. Erosion</strong></p><p>The basic idea of erosion is just like soil erosion only, it erodes away the boundaries of foreground object (Always try to keep foreground in white). So what it does? The kernel slides through the image (as in 2D convolution). A pixel in the original image (either 1 or 0) will be considered 1 only if all the pixels under the kernel is 1, otherwise it is eroded (made to zero).</p><p>So what happens is that, all the pixels near boundary will be discarded depending upon the size of kernel. So the thickness or size of the foreground object decreases or simply white region decreases in the image. It is useful for removing small white noises (as we have seen in colorspace chapter), detach two connected objects etc.</p><p>Here, as an example, I would use a 5x5 kernel with full of ones. Let’s see it how it works:</p><pre>import cv2 as cv<br>import numpy as np<br># Read the input image in grayscale<br>img = cv.imread(&#39;j.png&#39;, cv.IMREAD_GRAYSCALE)<br># Display the original image<br>cv.imshow(&#39;Original Image&#39;, img)<br>cv.waitKey(0)<br>cv.destroyAllWindows()<br># Define a 5x5 square-shaped structuring element (kernel) for erosion<br>kernel = np.ones((5, 5), np.uint8)<br># Perform erosion operation with the defined kernel<br>erosion = cv.erode(img, kernel, iterations=1)<br># Display the eroded image<br>cv.imshow(&#39;Erosion&#39;, erosion)<br>cv.waitKey(0)<br>cv.destroyAllWindows()</pre><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/112/0*lnjl5ES2AyeSHdzA.png" /></figure><p><strong>2. Dilation</strong></p><p>It is just opposite of erosion. Here, a pixel element is ‘1’ if at least one pixel under the kernel is ‘1’. So it increases the white region in the image or size of foreground object increases. Normally, in cases like noise removal, erosion is followed by dilation. Because, erosion removes white noises, but it also shrinks our object. So we dilate it. Since noise is gone, they won’t come back, but our object area increases. It is also useful in joining broken parts of an object.</p><p>dilation = cv.dilate(img,kernel,iterations = 1)</p><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/112/0*5Egt7H-Nf3zl118s.png" /></figure><p>3. Opening</p><p>Opening is just another name of <strong>erosion followed by dilation</strong>. It is useful in removing noise, as we explained above. Here we use the function, <strong>cv.morphologyEx()</strong></p><p>opening = cv.morphologyEx(img, cv.MORPH_OPEN, kernel)</p><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/0*Sq51QUcIxo21s6St.png" /></figure><p><strong>4. Closing</strong></p><p>Closing is reverse of Opening, <strong>Dilation followed by Erosion</strong>. It is useful in closing small holes inside the foreground objects, or small black points on the object.</p><p>closing = <a href="https://docs.opencv.org/3.4/d4/d86/group__imgproc__filter.html#ga67493776e3ad1a3df63883829375201f">cv.morphologyEx</a>(img, cv.MORPH_CLOSE, kernel)</p><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/0*KD46pWyPaQuMDaOe.png" /></figure><p><strong>5. Morphological Gradient</strong></p><p>It is the difference between dilation and erosion of an image.</p><p>The result will look like the outline of the object.</p><p>gradient = <a href="https://docs.opencv.org/3.4/d4/d86/group__imgproc__filter.html#ga67493776e3ad1a3df63883829375201f">cv.morphologyEx</a>(img, cv.MORPH_GRADIENT, kernel)</p><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/0*ILX_TmL-So6XJRnI.png" /></figure><p><strong>6. Top Hat</strong></p><p>It is the difference between input image and Opening of the image. Below example is done for a 9x9 kernel.</p><p>tophat = <a href="https://docs.opencv.org/3.4/d4/d86/group__imgproc__filter.html#ga67493776e3ad1a3df63883829375201f">cv.morphologyEx</a>(img, cv.MORPH_TOPHAT, kernel)</p><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/0*0qxlR3Ik71wjg8d0.png" /></figure><p><strong>7. Black Hat</strong></p><p>It is the difference between the closing of the input image and input image.</p><p>blackhat = <a href="https://docs.opencv.org/3.4/d4/d86/group__imgproc__filter.html#ga67493776e3ad1a3df63883829375201f">cv.morphologyEx</a>(img, cv.MORPH_BLACKHAT, kernel)</p><p>Result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/0*2FmHMidxbT9Bepir.png" /></figure><p><strong>Canny Edge Detection:</strong></p><p>Canny Edge Detection is a popular edge detection algorithm. It was developed by John F. Canny in</p><ol><li>It is a multi-stage algorithm and we will go through each stages.</li><li><strong>Noise Reduction: </strong>Since edge detection is susceptible to noise in the image, first step is to remove the noise in the image with a 5x5 Gaussian filter. We have already seen this in previous chapters.</li><li><strong>Finding Intensity Gradient of the Image: </strong>Smoothened image is then filtered with a Sobel kernel in both horizontal and vertical direction to get first derivative in horizontal direction ( Gx) and vertical direction ( Gy).</li><li><strong>Non-maximum Suppression: </strong>After getting gradient magnitude and direction, a full scan of image is done to remove any unwanted pixels which may not constitute the edge. For this, at every pixel, pixel is checked if it is a local maximum in its neighborhood in the direction of gradient. Check the image below:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*PVZOZer16N6zAxzJ.jpg" /></figure><p>Point A is on the edge ( in vertical direction). Gradient direction is normal to the edge. Point B and C are in gradient directions. So point A is checked with point B and C to see if it forms a local maximum. If so, it is considered for next stage, otherwise, it is suppressed ( put to zero). In short, the result you get is a binary image with “thin edges”.</p><p>5<strong>. Hysteresis Thresholding: </strong>This stage decides which are all edges are really edges and which are not. For this, we need two threshold values, minVal and maxVal. Any edges with intensity gradient more than maxVal are sure to be edges and those below minVal are sure to be non-edges, so discarded. Those who lie between these two thresholds are classified edges or non-edges based on their connectivity. If they are connected to “sure-edge” pixels, they are considered to be part of edges. Otherwise, they are also discarded. See the image below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*0OmI-PnLDH4f7iEA.jpg" /></figure><p>The edge A is above the maxVal, so considered as “sure-edge”. Although edge C is below maxVal, it is connected to edge A, so that also considered as valid edge and we get that full curve. But edge B, although it is above minVal and is in same region as that of edge C, it is not connected to any “sure-edge”, so that is discarded. So it is very important that we have to select minVal and maxVal accordingly to get the correct result.</p><p>This stage also removes small pixels noises on the assumption that edges are long lines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*5NTBFYyS9a9ol_A5gmn0lg.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea57cf922ac4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>