<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by JManansala on Medium]]></title>
        <description><![CDATA[Stories by JManansala on Medium]]></description>
        <link>https://medium.com/@jmanansala?source=rss-8de95334d9ba------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*vnWMosthwROufqKg681Utw.jpeg</url>
            <title>Stories by JManansala on Medium</title>
            <link>https://medium.com/@jmanansala?source=rss-8de95334d9ba------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 20:27:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jmanansala/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Complex Systems, as viewed by an Electrical Engineer]]></title>
            <link>https://medium.com/swlh/complex-systems-as-viewed-by-an-electrical-engineer-bb0be4e25d4d?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb0be4e25d4d</guid>
            <category><![CDATA[network-science]]></category>
            <category><![CDATA[electricity]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[complexity]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Sat, 20 Feb 2021 08:43:37 GMT</pubDate>
            <atom:updated>2021-02-20T08:43:37.274Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6WwjvSDGUbyh4ZSu" /><figcaption>Photo by <a href="https://unsplash.com/@alirexa?utm_source=medium&amp;utm_medium=referral">Alireza Khatami</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>The whole is greater than the sum of its parts.</em></p><p>This is often the quote that is mentioned to encourage teamwork and collaboration between team members. It highlights that through unity, the group can achieve much greater things than when done individually. Interestingly, we can also observe this very same principle across nature and our society.</p><p>Notice in nature, how a single cell can seem minute and insignificant, but it can bring life to an organism when combined with other cells. Or even ourselves, as a consumer in our economy, our spending patterns might seem insignificant, but collectively, our actions can define the national or even global economy. In these examples, the collective interactions between the cells or the consumers can organize and exhibit new observations that would have never been seen when they are looked at individually. <em>But who would have thought that there is a whole new field of science dedicated to this?</em></p><p>When I first heard about <a href="https://complexityexplained.github.io/"><strong>complexity science</strong></a>, I was amazed how this new field of science can change how we look and analyze things. Through the study of the emergence, self-organization, non-linearity, and adaptation between the individual components, complexity science can generate new insights. It can effectively explain a system’s characteristics and properties by studying its components’ relationships and interactions. I am primarily interested because there is universality that applies across multiple disciplines — physics, biology, economics, and even engineering. This is particularly useful because a good understanding of how these complex systems work and operate can significantly benefit our society. More the fact that the world we live in today is VUCA — that is, volatility, uncertainty, complexity, and ambiguity.</p><p>As an Electrical Engineer, I am personally interested in how this emerging field of science can help establish and understand the electrical network’s complexity. In itself, the physical infrastructure of the electric grid is already complicated. The electrical network can be thought of as the <em>largest piece of machinery that humankind has ever created </em>— it spans thousands of kilometers that traverse forests, mountains, and oceans. It has multiple electrical components like conductors, transformers, circuit breakers, and insulators working together to ensure the entire electrical network’s continuous operation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Vk8iWQXsRh5UQtxi" /><figcaption>Photo by <a href="https://unsplash.com/@nasa?utm_source=medium&amp;utm_medium=referral">NASA</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>However, it is not the physical components that make the electric network complex. Power system engineers design and develop the electric grid with full knowledge of how components work and cooperate — and once this structure is built, it will function as it was intended. Thus, an electrical network’s physical components will <em>not </em>exhibit emergence, self-organization, or adaptation, the way a complex system is defined. Additionally, no non-linearity can be observed in electric networks. The power generated is always equal to the power consumed plus the losses incurred.</p><p>What makes the electric grid complex is <em>how humans interact and intervene</em> in the operation of the electrical network. The human-aspect introduces uncertainty and non-linearity that makes the electric network complex. A prime example of how humans can influence how the electric network operates is how the demand profile emerges, self-organizes, and adapts depending on the time of the day, day of the week, or month of a year. As millions of random events happen from the people and industries interacting with each other, trends and patterns quickly arise on the electric network. Another interesting aspect is how the policies and regulations are developed across multiple electrical networks. The best practices, policies, and regulations are shared and interacted with several electric networks from several countries. An example is how the renewable energy movement has become what it is today as people became more self-aware of the consequences of their actions towards the environment. Through these examples, it is apparent that people’s, industries’, and policymakers’ decisions and actions are indeed what makes the electric network complex.</p><p>To sum up, the electrical network, by itself, is <em>complicated</em>. However, it is how the people and the policy are integrated into the electrical network, making it truly complex — <em>and getting even more complex</em>. Contributory factors for this added complexity include the continued growth of load demand, the integration of variable renewable energy sources, the need to improve the security of supply, and the need to lower carbon emissions. It is interesting how complexity science can effectively capture how these factors affect the electrical network’s operation.</p><p>Indeed, complexity science is a <em>new way of seeing the world</em>, and it adds another layer to the analysis and understanding that power system engineers employ to design and develop electrical networks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb0be4e25d4d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/complex-systems-as-viewed-by-an-electrical-engineer-bb0be4e25d4d">Complex Systems, as viewed by an Electrical Engineer</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing with Python: Applications in Machine Learning Part 2]]></title>
            <link>https://medium.com/analytics-vidhya/image-processing-with-python-applications-in-machine-learning-part-2-c13ac25f7f6f?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/c13ac25f7f6f</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Tue, 09 Feb 2021 04:12:53 GMT</pubDate>
            <atom:updated>2021-05-10T09:47:50.989Z</atom:updated>
            <content:encoded><![CDATA[<h4>How can image processing techniques be used to prepare data for a machine learning algorithm?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Kym_jNECp9XOWfqq" /><figcaption>Photo by <a href="https://unsplash.com/@marius?utm_source=medium&amp;utm_medium=referral">Marius Masalar</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In this post, we will learn the step-by-step procedures on how to preprocess and prepare image datasets to extract quantifiable features that can be used for a machine learning algorithm. Techniques such as binarization, template matching, morphological operations, and blob detection will be used. This serves as a supplement to the image processing techniques used in my <a href="https://jephraim-manansala.medium.com/image-processing-with-python-applications-in-machine-learning-17d7aac6bc97">previous article</a>.</p><p>Let’s begin.</p><p>As usual, we import libraries such as numpy, pandas, and matplotlib. Additionally, we import specific functions from the skimage, sklearn, and imblearn library.</p><pre>import numpy as np<br>import pandas as pd<br>import matplotlib.pyplot as plt<br>from matplotlib.patches import Rectangle<br>from skimage.io import imread, imshow<br>from skimage.feature import match_template, peak_local_max<br>from skimage.measure import label, regionprops<br>from skimage.morphology import erosion, dilation<br>from skimage.feature import blob_dog, blob_log, blob_doh<br>from skimage.util import invert<br>from sklearn.model_selection import train_test_split<br>from sklearn.ensemble import RandomForestClassifier<br>from imblearn.over_sampling import RandomOverSampler</pre><p>Our objective is to properly label the pitch and the notes on a music sheet. We will have to extract quantifiable information from each note that will serve as the input for our machine learning algorithm. Let’s tackle this step-by-step!</p><h3><strong>STEP 1: Perform Exploratory Data Analysis (EDA)</strong></h3><p>We should always remember that EDA is an essential part of any data science project. Before developing the machine learning pipeline, we need to familiarize ourselves with the data — this includes its structure, formatting, and nuances. This is crucial to ensure that the project methodology that will be conceptualized is appropriate for the dataset at hand.</p><p>For this project, we are going to use this music sheet of the nursery rhyme ‘Mary Had a Little Lamb’<em> (Original Image by musescore.com)</em></p><pre>music = imread(&#39;music-sheet.png&#39;)<br>imshow(music);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/459/1*ULl6VADYqxzRPsF5BQ5Cvg.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><p>Notice that we are only given one image of a music sheet. The image is a single-channel grayscale image, which means that we only need to binarize the image. Also, notice that each musical note has a specific symbol and position on it. This means that the machine learning pipeline to be developed should classify the musical symbol and the position of the said symbol. Lastly, notice that there is a G-clef on each line on the musical sheet. We can use this as a reference point for each of the lines.</p><h3><strong>STEP 2: Apply Image Binarization</strong></h3><p>From the EDA findings, we can start by performing image thresholding to extract the necessary information in the images. To do this, we will set all values above the midpoint as 255 — meanwhile, all values below the midpoint as 0.</p><pre>music[music&lt;120]=0<br>music[music&gt;=120]=255</pre><h3><strong>STEP 3: Image Segmentation using Template Matching</strong></h3><p>To help ease the image analysis, we should segment the image to every music line and remove unnecessary objects. From the EDA, we have observed that there is a G-clef on each of the music line. We can use this object as a reference point for each music line and use template matching <em>(object detection)</em> techniques. If you are not familiar with this technique, please do check it out in my <a href="https://jephraim-manansala.medium.com/image-processing-with-python-object-detection-using-template-matching-fa82b8c94fbd">previous post</a>!</p><pre>template = music[285:405, 70:140]<br>imshow(template);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/142/1*xRUtf8heW-vGFTaiIF4_bw.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><p>Now, let’s try to locate this template image to the input image!</p><pre>result = match_template(music, template)<br>imshow(result, cmap=&#39;viridis&#39;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*tsSp3tnzaury5VpkJgR8mQ.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><p>Notice that the yellow spots on the resulting cross-correlation image? This signifies the areas where the image template matched the input image. We can see that the technique successfully identified the G-clefs’ position in the input image. From here, we can segment the image using the coordinates of the matched template and add a range of values to encapsulate the entire staff lines.</p><pre>imshow(music)<br>template_height, template_width  = template.shape<br>music_lines = []</pre><pre>for y, x in peak_local_max(result, threshold_abs=0.8):<br>    rect = plt.Rectangle((x+55, y), <br>                         template_width+600, <br>                         template_height, <br>                         color=&#39;r&#39;, <br>                         fc=&#39;none&#39;)<br>    plt.gca().add_patch(rect);<br>    <br>    music_lines.append(music[y:y+template_height, <br>                             x+55:x+template_width+600+55])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*KH1XVgEMVS4la5t_RIfCQg.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><p>Notice how we did not encapsulate the G-clefs on the image segments? This is because we no longer need this symbol to determine each music notes’ note and pitch. It has served its purpose of automatically encapsulating each line of the music sheet by detecting the G-clefs.</p><h3><strong>STEP 4: Apply Morphological Operations</strong></h3><p>The next step is to remove the staff lines from the resulting segmented music lines to focus on the musical notes and their position. We can do this by applying morphological operations to the image. Note that we will use a vertical structuring element to remove the horizontal lines on the image.</p><pre>element = np.array([[0,1],<br>                    [0,1],<br>                    [0,1],<br>                    [0,1]])<br>imshow(element);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/201/1*LTZJtRJeXvlr_pqdRjQG0Q.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><p>Using the vertical structuring element, we can now apply the dilation and erosion operations on the image.</p><pre>def remove_lines(image_used):<br>    dilated_image = dilation(image_used, element)<br>    eroded_image = erosion(dilated_image, element)<br>    <br>    plt.figure(figsize=(15,3))<br>    plt.imshow(eroded_image, cmap=&#39;gray&#39;);<br>    plt.axis(&#39;off&#39;)<br>    return eroded_image</pre><pre>line1 = remove_lines(music_lines[0])<br>line2 = remove_lines(music_lines[1])<br>line3 = remove_lines(music_lines[2])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v0yUxVrPW_NziDoOHAh0_Q.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><p>Notice how the background is white, and the objects are black? This signifies that the background has the maximum value <em>(255)</em>, while the objects have the minimum value <em>(0)</em>. Because of this inversion, we will need to invert the functions of the morphological operations. Instead of applying erosion first, we will have to apply dilation <em>(to remove the staff lines)</em> prior to the erosion <em>(to restore the remaining items’ original shape)</em>.</p><h3><strong>STEP 5: Perform Blob Detection</strong></h3><p>Now that the staff lines are now removed, we can now apply blob detection to identify the notes’ position. Check this <a href="https://jephraim-manansala.medium.com/image-processing-with-python-detecting-blobs-in-digital-images-edebfd22328c">link </a>if you are still not familiar with this technique!</p><pre>def blob_detection(eroded_image):<br>    blobs = blob_dog(invert(eroded_image), <br>                     min_sigma=15, max_sigma = 20, threshold=0.70)</pre><pre>fig, ax = plt.subplots( figsize=(15,3))<br>    ax.imshow(eroded_image, cmap=&#39;gray&#39;)<br>    ax.axis(&#39;off&#39;)<br>    for blob in blobs:<br>        y, x, area = blob<br>        ax.add_patch(plt.Circle((x, y), area*np.sqrt(2), color=&#39;r&#39;, <br>                                fill=False))</pre><pre>    blobs = [(xaxis, yaxis) for yaxis, xaxis, z in blobs]<br>    blobs = sorted(blobs, key=lambda x: x[0])<br>    return blobs<br>blobs1 = blob_detection(line1)<br>blobs2 = blob_detection(line2)<br>blobs3 = blob_detection(line3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z26o3Jr6UOErg_Nm6pxbzw.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><p>For this step, I have opted to use blob detection using Difference of Gaussian rather than connected components because the objects I am trying to identify are not fully connected <em>(specifically the 2-beat notes)</em>. Moreover, the blob detection algorithm will suffice to identify the blobs in this step because we only need the coordinates of each note’s centroid to properly segment the musical symbols individually.</p><h3><strong>STEP 6: Pitch Evaluator</strong></h3><p>We can now determine each musical note’s pitch from its relative position on the image segments’ borders from the coordinates gathered in the blob detection. We do this by creating an if-else statement on the range of the centroids of each note. A deviation range, dev, is also created so that the algorithm has flexibility when the note is not centered correctly on the staff lines.</p><p><em>PRO-TIP: When defining the range of values for each note, it will be easier to estimate this by using the </em><em>axhline function in </em><em>matplotlib to create a horizontal line on the image.</em></p><pre>def define_pitch(number, dev=4):<br>    if int(number) in range(11-dev, 11+dev):<br>        return &quot;A&quot;<br>    elif int(number) in range(19-dev, 19+dev): <br>        return &quot;G&quot;<br>    elif int(number) in range(28-dev, 28+dev): <br>        return &quot;F&quot;<br>    elif int(number) in range(36-dev, 36+dev): <br>        return &quot;E&quot;<br>    elif int(number) in range(45-dev, 45+dev): <br>        return &quot;D&quot;<br>    elif int(number) in range(53-dev, 53+dev): <br>        return &quot;C&quot;<br>    elif int(number) in range(61-dev, 61+dev): <br>        return &quot;B&quot;<br>    elif int(number) in range(69-dev, 69+dev): <br>        return &quot;A&quot;<br>    elif int(number) in range(78-dev, 78+dev): <br>        return &quot;G&quot;<br>    elif int(number) in range(86-dev, 86+dev): <br>        return &quot;F&quot;<br>    elif int(number) in range(95-dev, 95+dev): <br>        return &quot;E&quot;<br>    elif int(number) in range(104-dev, 104+dev): <br>        return &quot;D&quot;<br>    elif int(number) in range(113-dev, 113+dev): <br>        return &quot;C&quot;</pre><p>Testing the created function, we have yielded the following results.</p><pre>imshow(music_lines[1])<br>plt.axhline(69);<br>print([define_pitch(x[1]) for x in blobs2])</pre><pre>&gt;&gt; [&#39;A&#39;, &#39;B&#39;, &#39;D&#39;, &#39;D&#39;, &#39;B&#39;, &#39;A&#39;, &#39;G&#39;, &#39;A&#39;]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*ygFhkzE3HCi85lpS7T015g.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><p>Note that this function works because we have fixed the image segmentation length and width on the algorithm. The pixel coordinate values set in the define_pitch function will not work on other image segmentation settings.</p><h3><strong>STEP 7: Retrieve Object Properties using Connected Components</strong></h3><p>Now that we have determined the centroid of each of the musical symbols on the image, we can now segment each of these symbols individually. We can then apply the regionprops function to extract key features and properties of each of the symbols. We will be extracting the following properties on the symbols:</p><ol><li>area — Number of pixels of the region.</li><li>convex_area — Number of pixels of convex hull image, which is the smallest convex polygon that encloses the region.</li><li>major_axis_length — The length of the ellipse’s major axis has the same normalized second central moments as the region.</li><li>minor_axis_length — The length of the ellipse’s minor axis has the same normalized second central moments as the region.</li><li>perimeter — Perimeter of the object, which approximates the contour as a line through the centers of border pixels using a 4-connectivity.</li></ol><p>These steps are automated on the function below:</p><pre>def properties(regions):<br>    area= np.sum([x.area for x in regions])<br>    convex_area = np.sum([x.convex_area for x in regions]) <br>    perimeter = np.sum([x.perimeter for x in regions]) <br>    major_axis= np.mean([x.major_axis_length for x in regions]) <br>    minor_axis = np.mean([x.minor_axis_length for x in regions]) <br>    return {&#39;area&#39;: area, <br>            &#39;convex_area&#39;: convex_area,<br>            &#39;perimeter&#39;: perimeter,<br>            &#39;major_axis&#39;: major_axis,<br>            &#39;minor_axis&#39;: minor_axis}</pre><pre>def blob_properties(eroded_image, blobs):<br>    fig, ax = plt.subplots(1, len(blobs), figsize=(15,2.5))</pre><pre>props = {}<br>    for num, data in enumerate(blobs):<br>        columns=int(data[0])<br>        rows=int(data[1])<br>        new_image = eroded_image[:, columns-15:columns+20]<br>        <br>        label_im = label(invert(new_image))<br>        regions = regionprops(label_im)<br>        <br>        props[num] = properties(regions)<br>        <br>        ax[num].imshow(new_image, cmap=&#39;gray&#39;)<br>    return pd.DataFrame(props).T</pre><pre>props1 = blob_properties(line1, blobs1);<br>props2 = blob_properties(line2, blobs2);<br>props3 = blob_properties(line3, blobs3);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_fGWg-MWiRLJ9628o2t8dQ.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><p>Now that we have collected each musical symbol’s properties, we need to label each symbol to its appropriate classification <strong><em>manually</em></strong>. Note that we need not label all symbols. We just need enough so that the algorithm can learn which is which.</p><pre>one_beat = pd.concat([props1.iloc[[1,2,3,4,5,6,8,9]], <br>                      props2.iloc[[1,2,4,5,6,7]], <br>                      props3.iloc[[0,1,3,4,5,6]]], axis=0)<br>one_beat[&#39;class&#39;] = &#39;1&#39;<br>two_beat = pd.concat([props1.iloc[[7]], <br>                      props2.iloc[[0,3]], <br>                      props3.iloc[[2,7]]], axis=0)<br>two_beat[&#39;class&#39;] = &#39;2&#39;<br>timesig = props1.iloc[[0]*2]<br>timesig[&#39;class&#39;] = &#39;timesig&#39;<br>end = props3.iloc[[8]*2]<br>end[&#39;class&#39;] = &#39;end&#39;</pre><pre>df = pd.concat([one_beat, two_beat, timesig, end])<br>df.sample(10)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/1*2ASa6V1LT_ZZyC4Qq-rx6A.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice that we multiplied the number of items in the timesig and end objects. We did this to represent all objects in the train set and test set <em>(later in the steps)</em>.</p><h3><strong>STEP 8: Apply Feature Engineering</strong></h3><p>From the extracted features, we can further expand the features by deriving new features. We do this by getting the ratios of the extracted features together. Let’s try this!</p><pre>def feature_engineering(df):<br>    df[&#39;ratio_length&#39;] = df[&#39;minor_axis&#39;] / df[&#39;major_axis&#39;]<br>    df[&#39;perimeter_ratio_major&#39;] = df[&#39;perimeter&#39;] / df[&#39;major_axis&#39;]<br>    df[&#39;perimeter_ratio_minor&#39;] = df[&#39;perimeter&#39;] / df[&#39;minor_axis&#39;]<br>    df[&#39;area_ratio_convex&#39;] = df[&#39;area&#39;] / df[&#39;convex_area&#39;]<br>    df[&#39;area_ratio_major&#39;] = df[&#39;area&#39;] / df[&#39;major_axis&#39;]<br>    df[&#39;area_ratio_minor&#39;] = df[&#39;area&#39;] / df[&#39;minor_axis&#39;]<br>    df[&#39;area_ratio_peri&#39;] = df[&#39;area&#39;] / df[&#39;perimeter&#39;]<br>    df[&#39;convex_ratio_major&#39;] = df[&#39;convex_area&#39;] / df[&#39;major_axis&#39;]<br>    df[&#39;convex_ratio_minor&#39;] = df[&#39;convex_area&#39;] / df[&#39;minor_axis&#39;]<br>    df[&#39;convex_ratio_peri&#39;] = df[&#39;convex_area&#39;] / df[&#39;perimeter&#39;]<br>    return df<br>final_df = feature_engineering(df)<br>final_df[&#39;class&#39;] = df[&#39;class&#39;]</pre><p>By doing so, we were able to expand our dataframe to 15 features!</p><h3><strong>STEP 9: Split the Training and Test Set</strong></h3><p>The next step is to split the data from the dataframe of the extracted and derived features on the image data. This is critical to ensure that the machine learning model will have a set of “unseen” data to ensure that the training is not overfitting or underfitting. We do this using the train_test_split function in the sklearn.model_selection library.</p><pre>X = final_df.drop(&#39;class&#39;, axis=1)<br>y = final_df[&#39;class&#39;]</pre><pre>X_train, X_test, y_train, y_test = train_test_split(X, y, <br>                                                    test_size=0.3, <br>                                                    stratify = y)</pre><h3><strong>STEP 10: Data Imbalance Handling</strong></h3><p>After splitting the dataset, we can notice that the number of object instances in each class is not equal. This can be a problem because this can lead to a bias to the majority class on the machine learning algorithm. To address this, we apply data imbalance handling using RandomOverSampler to ensure that the number of instances in each class is equal. <em>Note that other data imbalance handling techniques can be used in this (such as RandomUnderSampler, SMOTE, or AdaSyn).</em></p><pre>oversample = RandomOverSampler()<br>X_train, y_train = oversample.fit_resample(X_train, y_train)</pre><h3><strong>STEP 11: Train a Random Forest Classifier Model</strong></h3><p>Using the training dataset, we can now train a machine learning classifier model. We will be using a Random Forest Classifier because it is known to yield a high level of accuracy despite the simplicity of its algorithm. Note that since the Random Forest model is a tree-based model, we do not need to scale the dataset. However, if you will be using other machine learning models, you must scale the dataset’s features.</p><pre>RF = RandomForestClassifier(max_depth=5, n_estimators=500)<br>RF.fit(X_train, y_train)<br>y_pred_RF= RF.predict(X_test)</pre><pre>df = pd.DataFrame()</pre><pre>cols = [&#39;Machine Learning Classification Method&#39;,<br>        &#39;Train Accuracy&#39;, <br>        &#39;Test Accuracy&#39;, <br>        &quot;Top 1 Feature Predictor&quot;,<br>        &quot;Top 2 Feature Predictor&quot;,<br>        &quot;Top 3 Feature Predictor&quot;]</pre><pre>df.loc[&#39;Machine Learning Classification Method&#39;, <br>       &#39;Details&#39;] = &#39;Random Forest Classifier&#39;<br>df.loc[&#39;Train Accuracy&#39;, &#39;Details&#39;] = RF.score(X_train, y_train) <br>df.loc[&#39;Test Accuracy&#39;, &#39;Details&#39;] = RF.score(X_test, y_test)</pre><pre>importance = X.columns[np.argsort(RF.feature_importances_)][-3:]<br>df.loc[&#39;Top 1 Feature Predictor&#39;, &#39;Details&#39;] = importance[2]<br>df.loc[&#39;Top 2 Feature Predictor&#39;, &#39;Details&#39;] = importance[1]<br>df.loc[&#39;Top 3 Feature Predictor&#39;, &#39;Details&#39;] = importance[0]</pre><pre>display(df)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/562/1*KqJ-hdUK96wzjE1ifzY1kw.png" /><figcaption>(Image by Author)</figcaption></figure><p>The Random Forest Classifier yielded a test accuracy of 100%. The top features in classifying the different leaves are their perimeter_ratio_minor, convex_area, and convex_ratio_minor. We can see that the convex_area, perimeter, and minor_axis are the primary features because of the musical symbols&#39; distinct shapes. For example, the 1-beat note has a greater convex_area because it covers a greater area, while the 2-beat note has a greater perimeter because it has many regions.</p><p>Awesome, right? Again, it is a model that is <strong>both accurate and interpretable</strong>! Now, the only thing left to do is to display the predicted musical note classification and pitch.</p><pre>def blob_labeling(eroded_image, blobs):<br>    fig, ax = plt.subplots(1, len(blobs), figsize=(15,2))</pre><pre>    for num, data in enumerate(blobs):<br>        columns=int(data[0])<br>        rows=int(data[1])<br>        new_image = eroded_image[:, columns-15:columns+20]<br>        <br>        label_im = label(invert(new_image))<br>        regions = regionprops(label_im)<br>        <br>        dataset = pd.DataFrame([properties(regions)])<br>        final_df = feature_engineering(dataset)<br>        <br>        note = RF.predict(final_df)<br>        if (note[0] == &#39;1&#39;) | (note[0] == &#39;2&#39;):<br>            pitch = define_pitch(rows)<br>        else:<br>            pitch = &#39;&#39;<br>        title = str(pitch + &quot; &quot; + str(note[0]))</pre><pre>        ax[num].imshow(new_image, cmap=&#39;gray&#39;)<br>        ax[num].axis(&#39;off&#39;)<br>        ax[num].set_title(title)</pre><pre>props1 = blob_labeling(line1, blobs1);<br>props2 = blob_labeling(line2, blobs2);<br>props3 = blob_labeling(line3, blobs3);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sNKwB1lIiAwrpkZJQmJ1bw.png" /><figcaption><em>(Original Image by musescore.com)</em></figcaption></figure><h3><strong>In summary</strong></h3><p>We have explored how to use image processing techniques to prepare and preprocess image datasets to implement a machine learning algorithm. Using the regionprops function, we extracted valuable and quantifiable features from the image. Moreover, we have shown that preprocessing techniques such as binarization, template matching, morphological operations, and blob detection can be used to achieve our objective.</p><p>From this example, I hope you could appreciate how each technique can be used to manipulate the image to our advantage and how it can automate our procedures in creating a machine learning pipeline!</p><p><em>Want to learn more? Check out my GitHub repository at this </em><a href="https://github.com/jephraim-manansala/iip-machine-learning-2"><em>link</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c13ac25f7f6f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/image-processing-with-python-applications-in-machine-learning-part-2-c13ac25f7f6f">Image Processing with Python: Applications in Machine Learning Part 2</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing with Python: Enhancement for Image Differencing Applications]]></title>
            <link>https://medium.com/swlh/image-processing-with-python-enhancement-for-image-differencing-applications-fd05fe1df472?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/fd05fe1df472</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Sun, 07 Feb 2021 07:11:06 GMT</pubDate>
            <atom:updated>2021-02-14T13:39:19.008Z</atom:updated>
            <content:encoded><![CDATA[<h4>Why image sharpening is not suitable for image differencing?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oBG2xfy6lSZPY5sJ2J02Hw.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>In this post, we will learn how we can properly preprocess images for image differencing applications. Specifically, we will explore how sharpening an image using convolutional filters and correcting colors using histogram manipulation can affect image differencing. This article serves as an extension to my <a href="https://jephraim-manansala.medium.com/image-processing-with-python-image-differencing-for-video-feeds-ee071519e7d2">previous post</a> about image differencing on video feeds.</p><p>Let’s begin.</p><p>As usual, we import libraries such as numpy and matplotlib. We also import specific functions from the skimage and scipy library.</p><pre>import numpy as np<br>import matplotlib.pyplot as plt<br>from skimage.io import imread, imshow<br>from skimage.color import rgb2gray<br>from scipy.signal import convolve2d<br>from skimage import transform<br>from skimage import img_as_ubyte<br>from skimage.exposure import histogram, cumulative_distribution<br>from scipy.stats import norm, cauchy, logistic</pre><p>Now’, let’s go back to how <strong>image differencing</strong> is done. It is done by taking the difference between two aligned and grayscale images. By doing so, we can pinpoint the changes between the two images. All static objects will be deleted, whereas all dynamic objects will have reflected on the differenced image.</p><p>The specks of color in the differenced image depend on the difference between the intensity of the two images’ pixels. <strong>The greater the intensity of the pixel </strong>(whether be it high intensity or low intensity), <strong>the more apparent the object will be on the differenced image</strong>. Note that when an object is moved from its spot, we can assume that the spot it left is the mean pixel intensity (such as the highway, railway). Thus, we wanted to maximize the difference between the high/low-intensity pixels to the mean intensity pixel.</p><p>With this in mind, it is quite intuitive that <em>we want to preprocess the image to accentuate the intensity of the high and low-intensity pixels </em>to maximize its difference from the mean pixel intensity. So what image processing technique does this?</p><p>From our previous post, we have shown that <strong><em>image sharpening using convolutional filters</em></strong> can be applied to bring out detail in an image by enhancing the contrast of pixels on edges. Therefore, it can add contrast around an edge by accentuating bright and dark areas. This is exactly what we want to do, right? Let’s try it!</p><p>Again, let’s use a traffic surveillance feed from a major highway in Metro Manila, Philippines. We will use Frames #10 and #12 from the previous post. <em>(Video courtesy of GMA News TV, Philippines)</em>.</p><pre>fig, ax = plt.subplots(1,2, figsize=(16,8))</pre><pre>traffic = imread(&#39;traffic_frame10.jpg&#39;)<br>traffic2 = imread(&#39;traffic_frame12.jpg&#39;)<br>ax[0].imshow(traffic)<br>ax[0].set_title(&#39;Frame #10&#39;);</pre><pre>ax[1].imshow(traffic2)<br>ax[1].set_title(&#39;Frame #12&#39;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2tJ4tPyvC_dbaGNkIsir_A.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>Now, let’s define and apply the functions needed for the sharpening convolutional filter! We can copy these functions from the ones we used in the <a href="https://medium.com/swlh/image-processing-with-python-convolutional-filters-and-kernels-b9884d91a8fd">previous post</a>.</p><pre>def rgb_convolve2d(image, kernel):<br>    red = convolve2d(image[:,:,0], kernel, &#39;valid&#39;)<br>    green = convolve2d(image[:,:,1], kernel, &#39;valid&#39;)<br>    blue = convolve2d(image[:,:,2], kernel, &#39;valid&#39;)<br>    return np.stack([red, green, blue], axis=2)</pre><pre>fig, ax = plt.subplots(1,2, figsize=(16,8))<br>kernel = np.array([[-1, -1, -1, -1, -1],<br>                   [-1, -1, -1, -1, -1],<br>                   [-1, -1, 25, -1, -1],<br>                   [-1, -1, -1, -1, -1],<br>                   [-1, -1, -1, -1, -1]])</pre><pre>conv_im1 = (rgb_convolve2d(traffic, kernel[::-1, ::-1])<br>                          .clip(0,255)<br>                          .astype(np.uint8))<br>ax[0].imshow(conv_im1, cmap=&#39;gray&#39;)<br>ax[0].set_title(&quot;Sharpened Frame#10&quot;)</pre><pre>conv_im2 = (rgb_convolve2d(traffic2, kernel[::-1, ::-1])<br>                          .clip(0,255)<br>                          .astype(np.uint8))<br>ax[1].imshow(conv_im2, cmap=&#39;gray&#39;)<br>ax[1].set_title(&quot;Sharpened Frame#12&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KmH8yZBYI7hYoSImp-JEtQ.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>Notice how the <em>bright and dark spots were accentuated</em> in the image? This only shows that the convolutional filter was effective in sharpening the image. With more exemplified bright and dark spots, we can indeed have a more effective image differencing, right? Let’s find out!</p><p>Let’s apply the same image processing techniques <em>(such as homography matrix)</em> that we did in the <a href="https://jephraim-manansala.medium.com/image-processing-with-python-image-differencing-for-video-feeds-ee071519e7d2">previous post</a>.</p><pre>area_of_interest = [(890, 575),<br>                    (1085, 575),<br>                    (670, 1030),<br>                    (440, 1030)]</pre><pre>area_of_projection = [(930, 600),<br>                      (1070, 600),<br>                      (1070, 1000),<br>                      (930, 1000)]</pre><pre>def project_planes(image, src=area_of_interest, <br>                   dst=area_of_projection):<br>    new_image = image.copy() <br>    projection = np.zeros_like(new_image)<br>    tform = transform.estimate_transform(&#39;projective&#39;, <br>                                         np.array(src), <br>                                         np.array(dst))<br>    transformed = transform.warp(image, tform.inverse)<br>    return transformed</pre><pre>transformed1 = project_planes(rgb2gray(conv_im1), <br>                             area_of_interest, area_of_projection)<br>transformed2 = project_planes(rgb2gray(conv_im2), <br>                             area_of_interest, area_of_projection)</pre><pre>diff = transformed1 - transformed2<br>imshow(diff);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*tLis1AisppI2-0w0ytIiAw.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>By taking the difference of the two preprocessed frames of the video feed, we can see significant changes in the differenced image. First, looking at the color bar at the side of the image, we can see that the range became -1.00 to 1.00 (from -0.65 to 0.65 in the previous post). This signifies that the sharpening convolutional filter effectively accentuates the bright and dark spots in the image.</p><p>However, notice that the <em>differenced image is dirtier than before</em>? Even though the static objects did not change, there are still slight representations of these static objects on the differenced image. Moreover, even though the pixel intensity is accentuated, the <em>noise still prevails</em>, making it more challenging to apply the object detection algorithm in this image. This is because even though the two raw frames are almost identical, there are slight changes even on the static objects. By applying the sharpening convolutional filter on the image, we have also accentuated these slight changes. These differences might not be apparent in the resulting sharpened image. However, when we apply image differencing, these slight changes became visible as noise.</p><p>With this finding, it is clear that we <em>cannot use</em> the sharpening convolutional filter to preprocess our video feeds since even though it can accentuate the bright and dark spots in the image, it can also accentuate the noise in the image.</p><p>Let’s try other image preprocessing techniques — such as color correction using histogram manipulation. We are going to use the same functions that we used in my <a href="https://medium.com/swlh/image-processing-with-python-histogram-manipulation-on-digital-images-d4fb426d3513">article</a> about histogram manipulation.</p><pre>def individual_channel(image, dist, channel):<br>    im_channel = img_as_ubyte(image[:,:,channel])<br>    freq, bins = cumulative_distribution(im_channel)<br>    new_vals = np.interp(freq, dist.cdf(np.arange(0,256)), <br>                               np.arange(0,256))<br>    return new_vals[im_channel].astype(np.uint8)</pre><pre>def distribution(image, function, mean, std):<br>    dist = function(mean, std)<br>    image_intensity = img_as_ubyte(rgb2gray(image))<br>    <br>    plt.figure(figsize=(8,6))<br>    freq, bins = cumulative_distribution(image_intensity)<br>    plt.step(bins, freq, c=&#39;b&#39;, label=&#39;Actual CDF&#39;)<br>    plt.plot(dist.cdf(np.arange(0,256)), <br>             c=&#39;r&#39;, label=&#39;Target CDF&#39;)<br>    plt.legend()<br>    plt.title(&#39;Actual vs. Target Cumulative Distribution&#39;)</pre><p>We can use different distribution functions to manipulate the histogram of the image pixel intensities<em> by trial and error</em>. Distribution functions like the uniform, Cauchy, and Logistic functions can be used. Moreover, we can adjust the mean and standard deviation of these distribution functions to match the image differencing application’s needs.</p><pre>distribution(traffic, norm, 120, 50);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*2mDUc8PjEn3MdKGB8847LQ.png" /><figcaption>(Image by Author)</figcaption></figure><p>Ultimately, I have chosen a normal distribution with its mean equal to 120 and its standard deviation equal to 50. I have chosen this value because the maximum and minimum value of the CDF remains almost the same <em>(as evident by the actual and target CDF on the top and bottom to be almost coincidental)</em>. Moreover, this accentuates the maximum and minimum pixel intensity values by reducing the pixel intensity near the maximum and minimum pixel intensity values <em>(from 10 to 50, and 150 to 250).</em> Finally, the distribution mean is set to 120 to ensure that the maximum-to-mean and the minimum-to-mean difference is comparable.</p><pre>def show_distribution(image, function, mean, std, ax, title):<br>    dist = function(mean, std)<br>    <br>    image_intensity = img_as_ubyte(rgb2gray(image))<br>    freq, bins = cumulative_distribution(image_intensity)<br>    <br>    red = individual_channel(image, dist, 0)<br>    green = individual_channel(image, dist, 1)<br>    blue = individual_channel(image, dist, 2)<br>    <br>    ax.imshow(np.dstack((red, green, blue)))<br>    ax.set_title(title)<br>    return np.dstack((red, green, blue))</pre><pre>fig, ax = plt.subplots(1,2, figsize=(16,8))<br>new = show_distribution(traffic, norm, 120, 40, <br>                        ax[0], &quot;Corrected Frame#10&quot;);<br>new2 = show_distribution(traffic2, norm, 120, 40, <br>                        ax[1], &quot;Corrected Frame#12&quot;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*37ihI2cHarpkwSWw_FB1aA.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>By applying color correction using histogram manipulation, we have accentuated the bright and dark spots of the image while also effectively improving the image’s overall color quality! Now, let’s apply image differencing to these images!</p><pre>transformed1 = project_planes(rgb2gray(new), <br>                             area_of_interest, area_of_projection)<br>transformed2 = project_planes(rgb2gray(new2), <br>                             area_of_interest, area_of_projection)</pre><pre>diff = transformed1 - transformed2<br>imshow(diff)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*h3J6YbJqQcTIffFIQloDVQ.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>Comparing with the differenced image in the previous post, we can see that the color bar on the side of the image increased its range. The range has become -0.80 to 0.80 (from previously -0.65 to 0.65). This only shows that accentuation of the bright and dark spots is effective with histogram manipulation. Moreover, we achieved this improved intensity range while still avoiding increased noise! Amazing,<em> right?</em></p><p><strong>In summary</strong></p><p>We have explored how to use the appropriate preprocessing techniques to enhance the raw images for image differencing applications. We have shown that image sharpening using convolutional filters might not be suitable because it introduces a lot of noise to the image. Color correction is deemed the best image enhancement technique because it accentuates the bright and dark areas in the image and improves the overall color quality of the image.</p><p>Ultimately, it upon the discretion of the data scientist to know what preprocessing techniques should be used on each application. I hope you were able to pick up on the strengths and weaknesses of these image enhancement techniques!</p><p><em>Want to learn more? Check out my GitHub repository at this </em><a href="https://github.com/jephraim-manansala/image-diff-enhancement"><em>link</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd05fe1df472" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/image-processing-with-python-enhancement-for-image-differencing-applications-fd05fe1df472">Image Processing with Python: Enhancement for Image Differencing Applications</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing With Python: Image Differencing for Video Feeds]]></title>
            <link>https://medium.com/swlh/image-processing-with-python-image-differencing-for-video-feeds-ee071519e7d2?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/ee071519e7d2</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Sun, 31 Jan 2021 09:35:52 GMT</pubDate>
            <atom:updated>2021-02-14T13:38:15.805Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to detect the changes and movement in video frames?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*grY4QQp7iTR7yyFarBZ5rw.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>In this post, we will learn how we can apply the image differencing to detect changes and movement in each frame of a video.</p><p>Let’s begin.</p><p>As usual, we import libraries such as numpy and matplotlib. We also import specific functions from the skimage library. Lastly, we will introduce the cv2 library to convert video files to images.</p><pre>import numpy as np<br>import matplotlib.pyplot as plt<br>from skimage.io import imread, imshow<br>from skimage.color import rgb2gray<br>from skimage.feature import match_template, peak_local_max<br>from skimage import transform<br>import cv2</pre><p>But first, let us define what <strong>image differencing</strong> is. Basically, it is an image processing technique used to determine changes between images by finding the difference between each pixel in each image. The generated image highlights the pixel values’ changes, while all pixels that have no changes will be zero. The critical point is that the two images must be properly aligned for this technique to work.</p><p>Let’s use a traffic surveillance feed from a major highway in Metro Manila, Philippines. <em>(Video courtesy of GMA News TV, Philippines)</em>.</p><p>To implement image processing techniques on this video file, we will convert a short clip of this surveillance feed into images using the video frames. For this problem, the image’s capture interval will be 5 seconds — to emphasize the difference between images.</p><pre>vidcap = cv2.VideoCapture(&#39;traffic.mp4&#39;)<br>def getFrame(sec):<br>    vidcap.set(cv2.CAP_PROP_POS_MSEC,sec*1000)<br>    hasFrames,image = vidcap.read()<br>    if hasFrames:<br>        cv2.imwrite(&quot;traffic_frame&quot;+str(count)+&quot;.jpg&quot;, image) <br>    return hasFrames<br>sec = 0<br>frameRate = 5 #capture an image every 5 seconds<br>count = 1<br>success = getFrame(sec)<br>while success:<br>    count = count + 1<br>    sec = sec + frameRate<br>    sec = round(sec, 2)<br>    success = getFrame(sec)</pre><p>After converting the image, let’s inspect the image frame</p><pre>rgb_frames = [&#39;traffic_frame&#39;+str(num) for num in range(1, 15)]<br>gray_frames = [&#39;traffic_frame&#39;+str(num)+&#39;_gray&#39; <br>               for num in range(1, 15)]<br>images = []<br>for rgb, gray in zip(rgb_frames, gray_frames): <br>    globals()[rgb] = imread(rgb+&#39;.jpg&#39;)<br>    globals()[gray] = rgb2gray(globals()[rgb])<br>    images.append(globals()[rgb])<br>    <br>fig, axis = plt.subplots(4, int(len(images)/4), figsize=(15,12))</pre><pre>i = 1<br>for im, ax in zip(images, axis.flatten()):<br>    ax.imshow(im)<br>    ax.set_title(&quot;Frame #{}&quot;.format(i))<br>    i +=1</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gKnFucYxnsc4az5s9DqyTw.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p><em>Great!</em> For ease of our analysis, we will only be using two frames in this example. Let’s use Frame #10 and #12 because this is where the train is the clearest.</p><pre>fig, axis = plt.subplots(1,2, figsize=(18,6))</pre><pre>axis[0].imshow(traffic_frame10)<br>axis[0].set_title(&#39;Frame #10&#39;)<br>axis[1].imshow(traffic_frame12)<br>axis[1].set_title(&#39;Frame #12&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8vsFlwCCOyTb-3HiVrTrcQ.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>Now, let’s apply image processing techniques to prepare our image for image differencing. Since the image is taken at an angle, we can apply the homography matrix to change the image’s perspective. I will no longer discuss how the homography matrix works. If you are not yet familiar with this technique, I suggest reading it in my <a href="https://jephraim-manansala.medium.com/image-processing-with-python-image-warping-using-homography-matrix-22096734f09a">previous article</a>!</p><pre>area_of_interest = [(890, 575),<br>                    (1085, 575),<br>                    (670, 1030),<br>                    (440, 1030)]</pre><pre>area_of_projection = [(930, 600),<br>                      (1070, 600),<br>                      (1070, 1000),<br>                      (930, 1000)]</pre><pre>def project_planes(image, src=area_of_interest, <br>                   dst=area_of_projection):<br>    new_image = image.copy() <br>    projection = np.zeros_like(new_image)<br>    tform = transform.estimate_transform(&#39;projective&#39;, <br>                                         np.array(src), <br>                                         np.array(dst))<br>    transformed = transform.warp(image, tform.inverse)<br>    return transformed</pre><pre>def plot_project_planes(image, transformed, src, dst):<br>    x_src = [val[0] for val in src] + [src[0][0]]<br>    y_src = [val[1] for val in src] + [src[0][1]]</pre><pre>    x_dst = [val[0] for val in dst] + [dst[0][0]]<br>    y_dst = [val[1] for val in dst] + [dst[0][1]]</pre><pre>    fig, ax = plt.subplots(1,2, figsize=(13,8))<br>    ax[0].imshow(image, cmap=&#39;gray&#39;);<br>    ax[0].plot(x_src, y_src, &#39;r--&#39;)<br>    ax[0].set_title(&#39;Original Image&#39;)<br>    ax[1].imshow(transformed, cmap=&#39;gray&#39;)<br>    ax[1].plot(x_dst, y_dst, &#39;r--&#39;)<br>    ax[1].set_title(&#39;Warped Image&#39;)</pre><pre>transformed = project_planes(traffic_frame10, <br>                             area_of_interest, area_of_projection)<br>plot_project_planes(traffic_frame10, transformed, <br>                             area_of_interest, area_of_projection)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*tYULMXdW_5tW2k77KSTCgA.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><pre>transformed = project_planes(traffic_frame12, <br>                             area_of_interest, area_of_projection)<br>plot_project_planes(traffic_frame12, transformed, <br>                            area_of_interest, area_of_projection)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*gGZGcnN-z4qk1yecC4qISw.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>We can see that by applying the homography matrix, the objects at the backside of the image are now comparatively the same scale as the objects at the image’s front side.</p><p>Now, let’s apply image differencing! This can be computed by taking the difference between two aligned images. Note these images should be in the grayscale format for more straightforward analysis.</p><pre>diff = (project_planes(traffic_frame10_gray) - <br>        project_planes(traffic_frame12_gray))<br>imshow(diff)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*DKLEbAD1sMwMOKq7yTpgeg.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>By taking the difference between the two frames of the video, we can <em>see all the changes within that period</em>. Notice that the image shows the train on the tracks and the vehicles on the highways. These are all of the objects that moved! Moreover, notice how the roads, buildings, and pedestrian overpass are not reflected on the differenced image — this is because no changes happened on these objects!</p><p>Notice that the objects on the right side of the highway have red specks on top of the blue specks. Meanwhile, on the left side of the highway, the red specks are below the blue specks. This is because the initial position of the object is reflected as blue specks in the resulting image. Meanwhile, the final position is reflected as red specks.</p><p>Now, let’s see how far the object went from its initial to its final position. To do this, let’s get the absolute difference of the image so that the object in its initial and final position looks the same.</p><pre>absdiff = abs(diff)<br>imshow(absdiff)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*7wjp3IgmxPMYrpAqDxU5Cg.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>Great! Now, let’s use the template matching algorithm to detect these objects. Again, I will no longer discuss how template matching works. If you are not yet familiar with this technique, I suggest reading it in my <a href="https://jephraim-manansala.medium.com/image-processing-with-python-object-detection-using-template-matching-fa82b8c94fbd">previous article</a>!</p><p>Let’s use the train as the object!</p><pre>def find_template(image, x, y, x_width, y_width):<br>    fig, ax = plt.subplots(1,2, figsize=(12,4))<br>    template = image[y:y+y_width, x:x+x_width]<br>    ax[0].imshow(template, cmap=&#39;gray&#39;)<br>    ax[0].set_title(&#39;Object Template&#39;)<br>    result = match_template(image, template)<br>    ax[1].imshow(result, cmap=&#39;viridis&#39;);<br>    ax[1].set_title(&#39;Cross-correlation image&#39;)<br>    coor_x, coor_y = np.unravel_index(np.argmax(result), <br>                                      result.shape)<br>    return coor_x, coor_y, template, result</pre><pre>x, y, template, result = find_template(absdiff, <br>                                       x=890, y=540, <br>                                       x_width=90, <br>                                       y_width=150)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*P3FdqO-B_cfUvn3j3hMpnA.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><pre>plt.imshow(diff, cmap=&#39;RdBu&#39;)<br>template_width, template_height = template.shape<br>for x, y in peak_local_max(result, threshold_abs=0.6):<br>    rect = plt.Rectangle((y, x), <br>                         template_height, <br>                         template_width, <br>                         linewidth=3,<br>                         color=&#39;r&#39;, <br>                         fc=&#39;none&#39;)<br>    plt.gca().add_patch(rect);<br>    <br>dist = np.diff([val[0] for val in <br>                peak_local_max(result, threshold_abs=0.6)])[0]   <br>plt.text(x+260, y, <br>         &#39;Distance travelled is: {} units&#39;.format(dist), <br>         weight=&#39;bold&#39;, fontsize=10)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*LDhtd2g01HMTk-0ycoXY8A.png" /><figcaption>(Original Image by GMA News TV)</figcaption></figure><p>By using object detection, we were able to pinpoint where the object’s initial and final positions are in the image! We can now compute the distance traveled by the train by getting the difference between the coordinates of the initial and final position!</p><p>Moreover, we know that the time interval between the initial and final position, we can also compute for the train’s speed! Amazing, <em>right?</em></p><p><strong>In summary</strong></p><p>We have explored how to use image differencing to detect changes and movement of objects in video feeds. We have also applied image processing techniques such as homography matrix and template matching to enhance our analysis of this problem further.</p><p>I hope you were able to appreciate the potential of image differencing in solving real-world problems. Stay tuned for more posts!</p><p><em>Want to learn more? Check out my GitHub repository at this </em><a href="https://github.com/jephraim-manansala/image-differencing"><em>link</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ee071519e7d2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/image-processing-with-python-image-differencing-for-video-feeds-ee071519e7d2">Image Processing With Python: Image Differencing for Video Feeds</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing with Python: Object Detection using Template Matching]]></title>
            <link>https://medium.com/swlh/image-processing-with-python-object-detection-using-template-matching-fa82b8c94fbd?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa82b8c94fbd</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[image-processing]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Sun, 31 Jan 2021 00:38:16 GMT</pubDate>
            <atom:updated>2021-02-14T13:37:24.447Z</atom:updated>
            <content:encoded><![CDATA[<h4>How can we detect and recognize objects in an image?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*ZzqqA8IBweMfBz2K8Yur8A.png" /><figcaption>Detecting open windows in the building (Image by Author)</figcaption></figure><p>In this post, we will learn how we can find an object in an input image using template matching. This technique is particularly useful in image detection and recognition, as well as to object tracking.</p><p>Let’s begin.</p><p>As usual, we import libraries such as numpyand matplotlib. Additionally, we import specific functions from the skimage library.</p><pre>import numpy as np<br>import matplotlib.pyplot as plt<br>from skimage.io import imread, imshow<br>from skimage.color import rgb2gray<br>from skimage.feature import match_template, peak_local_max<br>from skimage import transform</pre><p>Let us define what <strong>template matching</strong> is. It is a technique for finding a reference image <em>(or a template image)</em> in the source image. In its most basic sense, the algorithm works by comparing the template for each part of the source image by sliding it one pixel at a time. This process is known as <strong>cross-correlation</strong>. This algorithm’s output is another image whose pixel value corresponds to the similarity of the template image to the source image when it was placed at that pixel location.</p><p>To explore how this works, let’s use an image of a condominium building. Let’s say that our object of interest is the windows in the building.</p><pre>condo = imread(‘condo.jpg’)<br>condo_gray = rgb2gray(condo)<br>imshow(condo);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*gLTZCX6d7Fh9GJlk_p3WwQ.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice the repeating patterns of the windows on this building. Let’s try to use this algorithm to count the number of windows in this building!</p><p>To do this, I have created the find_template function that inputs the template’s coordinates to be used and will output the similarity of the template image and the input image.</p><pre>def find_template(image, x, y, x_width, y_width):<br>    fig, ax = plt.subplots(1,2, figsize=(14,10))<br>    <br>    template = image[y:y+y_width, x:x+x_width]<br>    ax[0].imshow(template, cmap=&#39;gray&#39;)</pre><pre>    result = match_template(image, template)<br>    ax[1].imshow(result, cmap=&#39;viridis&#39;);<br>    <br>    coor_x, coor_y = np.unravel_index(np.argmax(result),<br>                                      result.shape)<br>    return coor_x, coor_y, template, result</pre><pre>x, y, template, result = find_template(condo_gray, <br>                                       x=2900, y=3350, <br>                                       x_width=250, <br>                                       y_width=300)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/1*l2PhHYm044axRcSOdDpBUw.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice how there are yellow spots on the resulting cross-correlation image. This signifies the regions in the input image where the template matches. Let’s try to reflect these regions on the actual image for a more straightforward analysis!</p><pre>imshow(condo)<br>template_width, template_height = template.shape<br>for x, y in peak_local_max(result, threshold_abs=0.5):<br>    rect = plt.Rectangle((y, x), <br>                         template_height, <br>                         template_width, <br>                         color=&#39;r&#39;, <br>                         fc=&#39;none&#39;)<br>    plt.gca().add_patch(rect);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*eY4-s8rM_hHKvYXFa6_T3Q.png" /><figcaption>(Image by Author)</figcaption></figure><p>We can see that even though the windows are repeating on the building, the template matching algorithm failed to detect these objects. This can be attributed to the underlying limitations of the algorithm. These are:</p><p>1. It is <strong>NOT scale-invariant</strong>, which means that the size of the objects should be the same<br>2. It is <strong>NOT rotational-invariant</strong>, which means that the orientation of the objects should be the same<br>3. It is <strong>NOT intensity-invariant</strong>, which means the color shades <em>(in grayscale)</em> of the objects should be the same</p><p>So is this problem a lost case? <em>Not necessarily! </em>We can apply image processing techniques to solve this problem.</p><p>From my previous post, We can use the homography matrix to transform the image to be more manageable before running the template matching algorithm. I will no longer discuss how the homography matrix works. <em>If you haven’t read </em><a href="https://jephraim-manansala.medium.com/image-processing-with-python-image-warping-using-homography-matrix-22096734f09a"><em>that post</em></a><em> yet, I suggest you read it first to understand the image processing steps that will be done here.</em></p><p>By defining the area_of_interest to be the edges of the building, we can warp the image to adjust the image’s perspective.</p><pre>area_of_interest = [(1359, 460),<br>                    (1931, 670),<br>                    (3830, 2800),<br>                    (1790, 2700)]</pre><pre>area_of_projection = [(1500, 1000),<br>                      (2500, 1000),<br>                      (2500, 4500),<br>                      (1500, 4500)]</pre><pre>def project_planes(image, src, dst):<br>    x_src = [val[0] for val in src] + [src[0][0]]<br>    y_src = [val[1] for val in src] + [src[0][1]]<br>    x_dst = [val[0] for val in dst] + [dst[0][0]]<br>    y_dst = [val[1] for val in dst] + [dst[0][1]]<br>    <br>    fig, ax = plt.subplots(1,2, figsize=(13,8))<br>    new_image = image.copy() <br>    projection = np.zeros_like(new_image)</pre><pre>    tform = transform.estimate_transform(&#39;projective&#39;, <br>                                         np.array(src), <br>                                         np.array(dst))<br>    transformed = transform.warp(image, tform.inverse)<br>    ax[0].imshow(new_image);<br>    ax[0].plot(x_src, y_src, &#39;r--&#39;)<br>    ax[0].set_title(&#39;Area of Interest&#39;)<br>    ax[1].imshow(transformed)<br>    ax[1].plot(x_dst, y_dst, &#39;r--&#39;)<br>    return transformed</pre><pre>transformed = project_planes(condo, area_of_interest, <br>                             area_of_projection)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*iI5vmE397AENCcdcUbR-rw.png" /><figcaption>(Image by Author)</figcaption></figure><p>Amazing, <em>right!?</em> Now that the building’s face is planar, let’s inspect if the limitations of the template matching algorithm are addressed.</p><p>First, the window size is now approximately the same. Second, the window orientation is all upright. Lastly, the color shades of windows of the same class are also the same. With this, we can now apply the template matching algorithm!</p><p>Let’s use the double window on the emergency staircase of the building as our object template.</p><pre>transformed_gray = rgb2gray(transformed)<br>x, y, template, result = find_template(transformed_gray, <br>                                       x=1970, y=4450, <br>                                       x_width=160, <br>                                       y_width=110)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*uM5IyzWr8R_OxCUUh-1jEw.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice that the yellow spots on the resulting cross-correlation image are much greater than when we did not use the homography matrix. This signifies the homography matrix is effective in addressing the limitations of the template matching algorithm! Let’s reflect these matches on the actual image.</p><pre>def show_matches(image, template, result, x, y, threshold=1):       <br>    imshow(image)<br>    template_width, template_height = template.shape<br>    for x, y in peak_local_max(result, threshold_abs=threshold):<br>        rect = plt.Rectangle((y, x), <br>                             template_height, <br>                             template_width, <br>                             color=&#39;r&#39;, <br>                             fc=&#39;none&#39;)<br>        plt.gca().add_patch(rect);</pre><pre>show_matches(transformed, template, result, x, y, threshold=1)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*p5qXtSnN8jVnoRl6KNArfA.png" /><figcaption>(Image by Author)</figcaption></figure><p>When the threshold is set to 1, only one object is detected — <em>and this is our template</em>. This signifies that the windows are not matching. However, based on the yellow spots on the cross-correlation image, more than one object is detected. From this, we can deduce that the objects are not exact matches of each other. To address this, we can reduce the threshold to detect more windows of the same structure.</p><pre>show_matches(transformed, template, result, x, y, threshold=0.8)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*-8zDtqRIGAPLmecM477h6Q.png" /><figcaption>(Image by Author)</figcaption></figure><p>By setting the threshold to 0.80, we have successfully detected most of the windows of a similar structure as the template image. We can see the turned on lights at the windows that were not detected. Remember that one of the limitations of the template matching algorithm is it is not intensity invariant? This is the reason why the algorithm does not detect these objects. Moreover, we should also take note that the <em>threshold should not be too low</em> because it can cause the detection of ghost objects.</p><p>Now, let’s try other objects in the image. We can use the opened windows as our template in this next example.</p><pre>x, y, template, result = find_template(transformed_gray, <br>                                       x=1600, y=2640, <br>                                       x_width=100, <br>                                       y_width=110)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*LegmkAQJGhQ0DuGVtLguWA.png" /><figcaption>(Image by Author)</figcaption></figure><p>Again, notice how there are yellow spots on the cross-correlation matrix? This signifies that it has detected matches between the object template and the resulting image. Let’s take a look!</p><pre>show_matches(transformed, template, result, x, y, 0.7)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*2Kct6fFspoI3Oql57oOxfg.png" /><figcaption>(Image by Author)</figcaption></figure><p>From this image, it is clear that the algorithm could detect all of the opened windows from the condominium building. Amazing, <em>right?</em></p><p><strong>In summary</strong></p><p>We have explored how to use the template matching algorithm to detect objects from our image. We have also explored how the homography matrix can help address the template matching algorithm’s limitations by warping the image to achieve similar scales and orientations for the objects.</p><p>I hope you were able to appreciate the potential of the template matching algorithm. However, note that template matching is one of the most basic algorithms for object detection. Other algorithms have already addressed the limitations of the template matching algorithm, but I’m going to leave it for you to research!</p><p><em>Want to learn more? Check out my GitHub repository at this </em><a href="https://github.com/jephraim-manansala/object-detection"><em>link</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa82b8c94fbd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/image-processing-with-python-object-detection-using-template-matching-fa82b8c94fbd">Image Processing with Python: Object Detection using Template Matching</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing with Python: Image Warping using Homography Matrix]]></title>
            <link>https://medium.com/swlh/image-processing-with-python-image-warping-using-homography-matrix-22096734f09a?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/22096734f09a</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Sat, 30 Jan 2021 13:06:42 GMT</pubDate>
            <atom:updated>2021-02-14T13:36:34.122Z</atom:updated>
            <content:encoded><![CDATA[<h4>How can we get a different perspective on images?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*k1Lipx9teNEjpnHpfDUDAg.png" /><figcaption>(Image by Author)</figcaption></figure><p>In this post, we will learn how we can apply the homography matrix to adjust the camera perspective in images.</p><p>Let’s begin.</p><p>As usual, we import libraries such as numpyand matplotlib. Additionally, we import specific functions from the skimage library.</p><pre>import numpy as np<br>import matplotlib.pyplot as plt<br>from skimage.io import imread, imshow<br>from skimage import transform</pre><p>Let’s first define what a homography is. In computer vision, <strong>homography </strong>is a transformation matrix in a homogenous coordinates space that is mapped between two planar projections of an image. These transformations can be a combination of rotation, translation, scaling, or skew operations.</p><p>Still difficult to understand? <em>Worry no more!</em> We will try to explore this <em>as visually as possible</em> in this blog post.</p><p>Let’s use this image I took at Puerto Princesa City Baywalk Park at Palawan Island.</p><pre>palawan = imread(&#39;palawan.jpg&#39;)<br>imshow(palawan);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*Rw76-UsHas17kr7P3ATz0g.png" /><figcaption>(Image by Author)</figcaption></figure><p>Because of the length of the “I ❤ Puerto Princesa” letters, I could not capture it from the front at one shot — thus, I just took the photo from the side. Let’s see if the homography matrix can adjust the perspective of this image from the front.</p><p>Let’s define the source plane and the destination plane from the original image. Note that we are doing a trial and error method to determine the coordinates of the plane’s vertices. Thus, this could take some time.</p><p><em>PRO-TIP: Use the MS Paint application on your personal computer to determine the pixel coordinates of your desired regions in your image.</em></p><pre>area_of_interest = [(500, 600),<br>                    (3220, 1950),<br>                    (3220, 2435),<br>                    (500, 3100)]</pre><pre>area_of_projection = [(100, 1000),<br>                      (3400, 1000),<br>                      (3400, 2600),<br>                      (100, 2600)]</pre><p>I have created a function that plots the vertices of the area_of_interest on the image. This function also displays how the shape of the area_of_projection plane.</p><pre>def project_planes(image, src, dst):<br>    x_src = [val[0] for val in src] + [src[0][0]]<br>    y_src = [val[1] for val in src] + [src[0][1]]</pre><pre>    x_dst = [val[0] for val in dst] + [dst[0][0]]<br>    y_dst = [val[1] for val in dst] + [dst[0][1]]<br>    <br>    fig, ax = plt.subplots(1,2, figsize=(13,6))<br>    <br>    new_image = image.copy() <br>    projection = np.zeros_like(new_image)</pre><pre>    ax[0].imshow(new_image);<br>    ax[0].plot(x_src, y_src, &#39;r--&#39;)<br>    ax[0].set_title(&#39;Area of Interest&#39;)<br>    ax[1].imshow(projection)<br>    ax[1].plot(x_dst, y_dst, &#39;r--&#39;)<br>    ax[1].set_title(&#39;Area of Projection&#39;)</pre><pre>project_planes(palawan, area_of_interest, area_of_projection)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*C57-TMoSbIpp0AJMxRIWzg.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice how we mapped the plane on the original image to be wide at the front and narrow towards the back? This is simply because objects will look smaller as their distance is farther away from us. What we want to do is to stretch the area of interest plane to be a rectangular plane.</p><p>Now, let’s try to warp the area of interest to fit the area of projection! To do this, I have also created a function that does this.</p><pre>def project_transform(image, src, dst):<br>    x_dst = [val[0] for val in dst] + [dst[0][0]]<br>    y_dst = [val[1] for val in dst] + [dst[0][1]]<br>    <br>    tform = transform.estimate_transform(&#39;projective&#39;, <br>                                         np.array(src), <br>                                         np.array(dst))<br>    transformed = transform.warp(image, tform.inverse)<br>    <br>    plt.figure(figsize=(6,6))<br>    plt.imshow(transformed)<br>    plt.plot(x_dst, y_dst, &#39;r--&#39;)</pre><pre>project_transform(palawan, area_of_interest, area_of_projection)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*QMsvXoWFRzYzI-QfeuAbHg.png" /><figcaption>(Image by Author)</figcaption></figure><p>Awesome, <em>right?</em> Using the homography matrix, we can pivot the perspective in the image! However, notice how the thickness of the letters increases from left to right. Moreover, notice how even though we are looking at the front-view perspective, we cannot see the spaces in between the letters? This is because the homography matrix merely warps the original image into the projected image. Therefore, it just relies on the information available on the original perspective of the image. This lack of information in the original image explains why the thickness of the letters increases from left to right. It cannot create new information, such as the objects behind the letters to fill in the letters’ gaps. That is why it just stretches the thickness of the letters.</p><p>Now, let’s try to look at a different perspective! Say I was walking down alongside these letters. How would it look like?</p><pre>area_of_interest = [(500, 600),<br>                    (2500, 1000),<br>                    (2500, 3500),<br>                    (500, 3100)]</pre><pre>area_of_projection = [(1000, 1000),<br>                      (2200, 1000),<br>                      (2200, 4000),<br>                      (1000, 4000)]</pre><pre>project_planes(palawan, area_of_interest, area_of_projection)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*Ayu41IAqDkIX9wyOKoieng.png" /><figcaption>(Image by Author)</figcaption></figure><p>In creating the new plane for this perspective, I used the leftmost edge of the plane in our previous perspective while changing the other three edges. To visualize this, let us imagine that there is a rectangular prism in the image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*BzVgC6DHga7Qtf0AaTtktw.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice that we just changed the face of the rectangular prism that we are using as the area of interest plane. Instead of the rectangular prism’s backside face, we are now using the front face of the rectangular prism as the area of interest. Now that you visualized it, let’s project this plane on the destination plane or the area of projection plane!</p><pre>project_transform(palawan, area_of_interest, area_of_projection)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*ndY1qlCli_qFQ7HwaVc-aQ.png" /><figcaption>(Image by Author)</figcaption></figure><p>From this perspective, can you imagine that you are walking alongside these letters? Well,<em> I could!</em> However, notice that there are lots of black spaces on the edges of the projected image. When we projected the points from the area of interest to the area of projection, the image was stretched. Moreover, remember that this algorithm can only get information from the original image. Since those areas are not in the original image, the algorithm cannot predict what will be in those areas — thus, the blank spaces!</p><p>Now, let’s try to take the top-view perspective!</p><pre>area_of_interest = [(3220, 2450),<br>                    (3300, 2475),<br>                    (850, 3200),<br>                    (500, 3100)]</pre><pre>area_of_projection = [(2000, 1000),<br>                      (2150, 1000),<br>                      (2150, 4000),<br>                      (2000, 4000)]</pre><pre>project_planes(palawan, area_of_interest, area_of_projection)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*DA8VbrRX-sJd7wlE13BhEA.png" /><figcaption>(Image by Author)</figcaption></figure><p>Again, this plane is just the bottom side face of the rectangular prism that we used earlier. Let’s try to stretch it!</p><pre>project_transform(palawan, area_of_interest, area_of_projection)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*lLEz46c5l1GiHK8eFGEbOQ.png" /><figcaption>(Image by Author)</figcaption></figure><p>Even though we were able to project the image onto the new points properly, the projected top view of the image looks <em>unnatural </em>since we are still not seeing the top side of the letters and the objects behind the letters. However, we should note that that information is not included in the original image so the algorithm just stretches whatever information that it has. This just shows that the homography matrix is the most effective when we have a large area of interest and is almost front-facing the camera perspective in the original image.</p><p><em>And that’s it!</em> We have seen the power of warping images using the homography matrix! With this tool, we can indeed have a different perspective from an image. I hope you were able to appreciate the usefulness of the homography matrix and I hope you can now visualize how to use it in your projects!</p><p><strong>In summary</strong></p><p>We have explored how to use the homography matrix to get a different perspective on images. We have shown how powerful the homography matrix in warping images. However, we have also observed its limitations when there is a lack of information in the original image — especially in perspectives that are orthogonal from the original perspective.</p><p><em>Want to learn more? Check out my GitHub repository at this </em><a href="https://github.com/jephraim-manansala/image-warping"><em>link</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22096734f09a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/image-processing-with-python-image-warping-using-homography-matrix-22096734f09a">Image Processing with Python: Image Warping using Homography Matrix</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing with Python: Applications in Machine Learning]]></title>
            <link>https://medium.com/analytics-vidhya/image-processing-with-python-applications-in-machine-learning-17d7aac6bc97?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/17d7aac6bc97</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Sat, 30 Jan 2021 02:03:06 GMT</pubDate>
            <atom:updated>2021-02-14T11:14:32.372Z</atom:updated>
            <content:encoded><![CDATA[<h4>How can image processing techniques be used to prepare data for a machine learning algorithm?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*k5_5dp-l2N7vQQbb" /><figcaption>Photo by <a href="https://unsplash.com/@ashishjha?utm_source=medium&amp;utm_medium=referral">Avinash Kumar</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In this post, we will learn the step-by-step procedures on how to preprocess and prepare image datasets to extract quantifiable features that can be used for a machine learning algorithm.</p><p>Let’s begin.</p><p>As usual, we import libraries such as numpy, pandas, and matplotlib. Additionally, we import specific functions from the skimage and sklearn library.</p><pre>import matplotlib.pyplot as plt<br>import numpy as np<br>import pandas as pd<br>from glob import glob<br>from skimage.io import imread, imshow<br>from skimage.color import rgb2gray<br>from skimage.measure import label, regionprops, regionprops_table<br>from skimage.filters import threshold_otsu<br>from skimage.morphology import area_closing, area_opening<br>from sklearn.model_selection import train_test_split<br>from sklearn.ensemble import RandomForestClassifier</pre><p>Our objective is to extract information from each sample that can be used for our machine learning algorithm. Let’s tackle this step-by-step!</p><h3><strong>STEP 1: Perform Exploratory Data Analysis (EDA)</strong></h3><p>EDA is always an essential part when building a machine learning algorithm. First and foremost, you should be able to familiarize yourself with the data — its structure, its formatting, and its nuances. This is crucial to ensure that the project methodology that you will conceptualize will be appropriate for the dataset at hand.</p><p>For this project, the dataset that we will be using is a collection of dried plant leaves specimens in white background <em>(Image Use Permission Granted by Gino Borja, AIM)</em>. There are three classes of plant leaves in this dataset — such as plantA, plantB, and plantC.</p><pre>filepaths = glob(&#39;dataset/*.jpg&#39;)<br>fig, axis = plt.subplots(1,len(filepaths), figsize=(16,8))<br>for x, ax in zip(filepaths, axis.flatten()):<br>    ax.imshow(imread(x))<br>    ax.set_title(x.split(&#39;\\&#39;)[1])<br>print(&quot;The shape of the image is:&quot;, imread(filepaths[0]).shape)</pre><pre>&gt;&gt; The shape of the image is: (876, 637, 3)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*rH-mpyCyRG4B5yLkCTJLdQ.png" /><figcaption>(Original Image by <em>Gino Borja, AIM</em>)</figcaption></figure><p>Notice that each of the class of leaves have a distinct characteristic on it. PlantA can be characterized by the cirration on its leaf body, PlantB can be characterized by its leaves’ rugged edges, and PlantC can be characterized by its leaves’ rounded shape.</p><p>Moreover, notice that even though the data is in grayscale, the images still contain three channels. However, further inspecting, we can notice that the three channels are mere duplicates of each other. Because of this, we can drop the other channels by slicing the array or by using the rgb2gray function.</p><h3><strong>STEP 2: Apply Image Binarization</strong></h3><p>From the EDA findings, we can now utilize the rgb2gray function to convert the image’s three channels into a single channel. After which, we can perform image thresholding to extract the necessary information in the images — in this case, the leaves. Since the image dataset is a classic example of a background-foreground image, we can utilize Otsu’s Method to threshold the image.</p><pre>leaf = imread(filepaths[0])<br>gray_leaf = rgb2gray(leaf)</pre><pre>thresh = threshold_otsu(gray_leaf)<br>binarized = gray_leaf &lt; thresh<br>imshow(binarized)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/1*hBUgWO2A65jBT6Lg9__U9g.png" /><figcaption>(Original Image by <em>Gino Borja, AIM</em>)</figcaption></figure><h3><strong>STEP 3: Apply Morphological Operations</strong></h3><p>Now, you might ask, why do we need to apply morphological operations if the image appears to be clean? It is a good practice to still perform morphological operations to remove the noise in the image — especially in images with a large number of pixels. Remember that a single pixel noise might not be visible to the naked eye, but it can later affect your image analysis!</p><p>To minimize the distortion of the objects’ shapes, we will use the area_closing and area_opening morphological operations. We apply the area_closing function to fill in any holes inside the region of the object of interest. In this dataset, area_closing can be particularly useful if the leaf regions have damages, cirration, or fibers that have a different pixel intensity than the leaf body. After which, we have applied the area_opening function to remove noise in the image background.</p><pre>closed = area_closing(binarized)<br>opened = area_opening(closed)<br>imshow(opened)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/1*1b-ZJcAfZ4TYpVTb5E1fkg.png" /><figcaption>(Original Image by <em>Gino Borja, AIM</em>)</figcaption></figure><h3><strong>STEP 4: Label Each Region using Connected Components</strong></h3><p>After cleaning the dataset, we will utilize the connected components’ label function to identify all regions of the image. Then, we can use the regionprops function to extract properties from each region in the image.</p><pre>label_im = label(opened)<br>regions = regionprops(label_im)<br>imshow(label_im)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/1*a4rIVR2o9ClwdzDYLsQrrg.png" /><figcaption>(Original Image by <em>Gino Borja, AIM</em>)</figcaption></figure><p>Notice how each region has a varying color. Each color represents a region in the image. The numbering of these regions’ grouping is from left to right, then from top to bottom. Moreover, we should remember that the zeroth region in the resulting array of the label function is always the image’s background. Finally, in filtering the regions, we will use an if-else statement. The two conditions in the if-else statement are: the background region and the regions with the negligible area are not included.</p><p>Now, let’s try to look at these regions individually!</p><pre>masks = []<br>bbox = []<br>for num, x in enumerate(regions):<br>    area = x.area<br>    convex_area = x.convex_area<br>    if num!=0 and x.area &gt;= 100: <br>        masks.append(regions[num].convex_image)<br>        bbox.append(regions[num].bbox)    <br>count = len(masks)</pre><pre>fig, axis = plt.subplots(4, int(count/4), figsize=(15,6))<br>for ax, box, mask in zip(axis.flatten(), bbox, masks):<br>    image  =  gray_leaf[box[0]:box[2], box[1]:box[3]] * mask<br>    ax.imshow(image)<br>    ax.axis(&#39;off&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4KfUkjjWa3_qaAS0gfS-Cg.png" /><figcaption>(Original Image by <em>Gino Borja, AIM</em>)</figcaption></figure><h3><strong>STEP 5: Get the Properties of Each Region</strong></h3><p>Now, visualizing the regions can be very useful to counter-check if the image processing steps are effective. However, our objective is to extract key features and properties from each of these regions! We can easily do this using the regionprops_table function, wherein we can specify the properties extracted from the regions. In this dataset, we will be using the following quantifiable properties:<br>1. area — Number of pixels of the region.<br>2. convex_area — Number of pixels of convex hull image, which is the smallest convex polygon that encloses the region.<br>3. bbox_area — Number of pixels of bounding box.<br>4. major_axis_length — The length of the ellipse’s major axis has the same normalized second central moments as the region.<br>5. minor_axis_length — The length of the ellipse’s minor axis has the same normalized second central moments as the region.<br>6. perimeter — Perimeter of object which approximates the contour as a line through the centers of border pixels using a 4-connectivity.<br>7. equivalent_diameter — The diameter of a circle with the same area as the region.<br>8. mean_intensity -Value with the mean intensity in the region.<br>9. solidity — Ratio of pixels in the region to pixels of the convex hull image.<br>10. eccentricity — Eccentricity of the ellipse that has the same second-moments as the region</p><pre>properties = [&#39;area&#39;,&#39;convex_area&#39;,&#39;bbox_area&#39;,<br>              &#39;major_axis_length&#39;, &#39;minor_axis_length&#39;, <br>              &#39;perimeter&#39;, &#39;equivalent_diameter&#39;,<br>              &#39;mean_intensity&#39;, &#39;solidity&#39;, &#39;eccentricity&#39;]<br>pd.DataFrame(regionprops_table(label_im, gray_leaf, <br>                               properties=properties)).head(10)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xUHBNPL-Vo75GCU34sZQHA.png" /><figcaption>(Image by Author)</figcaption></figure><h3><strong>STEP 6: Combine All the Steps into a Single Function</strong></h3><p>Now that we have established the steps and checked that the data extracted is good, we can now <em>automate </em>all previous steps into a single function. This is particularly useful to extract the features needed in the machine learning algorithm quickly.</p><pre>def get_properties(name):<br>    filepaths = glob(&#39;dataset/{}*.jpg&#39;.format(name))<br>    properties = [&#39;area&#39;,&#39;convex_area&#39;,<br>                 &#39;bbox_area&#39;,&#39;major_axis_length&#39;, <br>                 &#39;minor_axis_length&#39;, &#39;perimeter&#39;,  <br>                 &#39;equivalent_diameter&#39;, &#39;mean_intensity&#39;,  <br>                 &#39;solidity&#39;, &#39;eccentricity&#39;]<br>    dataframe = pd.DataFrame(columns=properties)<br>    for file in filepaths:<br>        grayscale = rgb2gray(imread(file))<br>        threshold = threshold_otsu(grayscale)<br>        binarized = grayscale &lt; threshold         <br>        closed = area_closing(binarized, 1000)<br>        opened = area_opening(closed, 1000)<br>        labeled = label(opened)<br>        regions = regionprops(labeled)<br>        data = pd.DataFrame(regionprops_table(labeled, grayscale,<br>                            properties=properties))<br>        data = data[(data.index!=0) &amp; (data.area&gt;100)]<br>        dataframe = pd.concat([dataframe, data])<br>    return dataframe</pre><h3><strong>STEP 7: Collect Features Dataframe</strong></h3><p>Using the get_properties function created, we can now collect each region’s features and properties in the image! Once the properties are collected, don’t forget to input the label classification of these properties — this will serve as the ground truth for the machine learning algorithm.</p><p>Let’s extract the dataframe for plantA.</p><pre>plantA = get_properties(&#39;plantA&#39;)<br>plantA[&#39;type&#39;] = &#39;plantA&#39;<br>print(&quot;The shape of the dataframe is: &quot;, plantA.shape)<br>plantA.head()</pre><pre>&gt;&gt; The shape of the dataframe is:  (50, 11)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vomb1xirBCulCc0msA3isg.png" /><figcaption>(Image by Author)</figcaption></figure><p>Now, for plantB.</p><pre>plantB = get_properties(&#39;plantB&#39;)<br>plantB[&#39;type&#39;] = &#39;plantB&#39;<br>print(&quot;The shape of the dataframe is: &quot;, plantB.shape)<br>plantB.head()</pre><pre>&gt;&gt; The shape of the dataframe is:  (55, 11)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1-4BVu-DPRq4CzNu8AgSRg.png" /><figcaption>(Image by Author)</figcaption></figure><p>Lastly, for plantC.</p><pre>plantC = get_properties(&#39;plantC&#39;)<br>plantC[&#39;type&#39;] = &#39;plantC&#39;<br>print(&quot;The shape of the dataframe is: &quot;, plantC.shape)<br>plantC.head()</pre><pre>&gt;&gt; The shape of the dataframe is:  (53, 11)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xg2IrBFjwWQb3ptPZNr52w.png" /><figcaption>(Image by Author)</figcaption></figure><p>Now that we have collected the properties of each object in the dataset, let us combine these information in a single dataframe!</p><pre>df = pd.concat([plantA, plantB, plantC])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vjnakPx1xx6d0_nd0bDMXg.png" /><figcaption>(Image by Author)</figcaption></figure><h3><strong>STEP 8: Apply Feature Engineering</strong></h3><p>From the extracted features, we can further expand the features by deriving new features. This can easily be done by getting the ratios of the extracted features together. Let’s try this!</p><pre>df[&#39;ratio_length&#39;] = (df[&#39;major_axis_length&#39;] / <br>                      df[&#39;minor_axis_length&#39;])<br>df[&#39;perimeter_ratio_major&#39;] = (df[&#39;perimeter&#39;] /  <br>                               df[&#39;major_axis_length&#39;])<br>df[&#39;perimeter_ratio_minor&#39;] = (df[&#39;perimeter&#39;] /<br>                               df[&#39;minor_axis_length&#39;])<br>df[&#39;area_ratio_convex&#39;] = df[&#39;area&#39;] / df[&#39;convex_area&#39;]<br>df[&#39;area_ratio_bbox&#39;] = df[&#39;area&#39;] / df[&#39;bbox_area&#39;]<br>df[&#39;peri_over_dia&#39;] = df[&#39;perimeter&#39;] / df[&#39;equivalent_diameter&#39;]</pre><pre>final_df = df[df.drop(&#39;type&#39;, axis=1).columns].astype(float)<br>final_df = final_df.replace(np.inf, 0)<br>final_df[&#39;type&#39;] = df[&#39;type&#39;]</pre><p>By doing so, we were able to expand our dataframe to 17 features!</p><h3><strong>STEP 9: Split the Training and Test Set</strong></h3><p>The next step is to split the data from the dataframe of the extracted and derived features on the image data. This is critical to ensure that the machine learning model will have a set of “unseen” data to ensure that the training is not overfitting or underfitting. We do this using the train_test_split function in the sklearn.model_selection library. Moreover, we will also not do any data imbalance handling techniques because the number of instances per class is approximately equal to each other.</p><pre>X = final_df.drop(&#39;type&#39;, axis=1)<br>y = final_df[&#39;type&#39;]</pre><pre>X_train, X_test, y_train, y_test = train_test_split(X, y, <br>                                         test_size=0.25, <br>                                         stratify = y, <br>                                         random_state=1337)</pre><h3><strong>STEP 10: Train a Random Forest Classifier Model</strong></h3><p>Using the training dataset, we can now train a machine learning classifier model. We will be using a Random Forest Classifier because it is known to yield a high level of accuracy despite the simplicity of its algorithm. Note that since the Random Forest model is a tree-based model, we do not need to scale the dataset. However, if you will be using other machine learning models, you must scale the features of the dataset.</p><pre>RF = RandomForestClassifier(max_depth=6, n_estimators=100)<br>RF.fit(X_train, y_train)<br>y_pred_RF= RF.predict(X_test)</pre><pre>df = pd.DataFrame()</pre><pre>cols = [&#39;Machine Learning Classification Method&#39;,<br>        &#39;Train Accuracy&#39;, <br>        &#39;Test Accuracy&#39;, <br>        &quot;Top 1 Feature Predictor&quot;,<br>        &quot;Top 2 Feature Predictor&quot;,<br>        &quot;Top 3 Feature Predictor&quot;,<br>        &quot;Top 4 Feature Predictor&quot;,<br>        &quot;Top 5 Feature Predictor&quot;]</pre><pre>df.loc[&#39;Machine Learning Classification Method&#39;, <br>       &#39;Details&#39;] = &#39;Random Forest Classifier&#39;<br>df.loc[&#39;Train Accuracy&#39;, &#39;Details&#39;] = RF.score(X_train, y_train) <br>df.loc[&#39;Test Accuracy&#39;, &#39;Details&#39;] = RF.score(X_test, y_test)</pre><pre>importances = (X.columns[np.argsort(RF.feature_importances_)][-5:])<br>df.loc[&#39;Top 1 Feature Predictor&#39;, &#39;Details&#39;] = importances[4]<br>df.loc[&#39;Top 2 Feature Predictor&#39;, &#39;Details&#39;] = importances[3]<br>df.loc[&#39;Top 3 Feature Predictor&#39;, &#39;Details&#39;] = importances[2]<br>df.loc[&#39;Top 4 Feature Predictor&#39;, &#39;Details&#39;] = importances[1]<br>df.loc[&#39;Top 5 Feature Predictor&#39;, &#39;Details&#39;] = importances[0]</pre><pre>display(df)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*i_LZ6941BK88n2P-KErvJQ.png" /><figcaption>(Image by Author)</figcaption></figure><p>The Random Forest Classifier yielded a <strong>test accuracy of 90.00%</strong>. The top features in classifying the different leaves are their mean_intensity, area_ratio_convex, solidity, perimeter_ratio_major, and peri_over_dia.</p><p>Going back to our initial findings in the EDA, we can relate why the mean_intensity is the top 1 predictor is because we have previously identified that plantA can be identified by the cirration in its leaves. Cirration can easily be reflected in the pixel intensity of the objects.</p><p>Moreover, the ratio between the area and convex area is the top 2 predictor is because we have identified that plantB to have rugged edges while plantC to have rounded edges. Therefore, plantB will naturally have a lower number, while plantC will have an area_ratio_convexthat is approximately equal to 1.</p><p>Awesome, right? A model that is <strong>both accurate and interpretable!</strong> With this, I hope you were able to appreciate the importance of image processing techniques to achieve a more interpretable machine learning algorithm!</p><h3><strong>In summary</strong></h3><p>We have explored how to use image processing techniques to prepare and preprocess image datasets to implement a machine learning algorithm. Using the regionprops_table function, we were able to extract valuable and quantifiable features from the image. In the end, we have developed a classifier model that achieved a high accuracy while also being interpretable.</p><p><em>Want to learn more? Check out my GitHub repository at this </em><a href="https://github.com/jephraim-manansala/iip-machine-learning"><em>link</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17d7aac6bc97" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/image-processing-with-python-applications-in-machine-learning-17d7aac6bc97">Image Processing with Python: Applications in Machine Learning</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing with Python: Image Segmentation using RG Chromaticity]]></title>
            <link>https://medium.com/swlh/image-processing-with-python-image-segmentation-using-rg-chromaticity-9568c3276db6?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/9568c3276db6</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Fri, 29 Jan 2021 11:27:41 GMT</pubDate>
            <atom:updated>2021-02-14T13:35:17.301Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to pinpoint and segment objects based on their color chromaticity?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lNKFencL2tn2vLzBRwYGDA.png" /><figcaption>(Image by Author)</figcaption></figure><p>In this post, we will explore how to conduct image segmentation using the RG Chromaticity Space. This post serves as an extension of the image segmentation methods from my <a href="https://jephraim-manansala.medium.com/image-processing-with-python-image-segmentation-using-thresholding-methods-423ecdaf8ab4">previous post</a>.</p><p>Let’s begin.</p><p>As usual, we import libraries such as numpy and matplotlib. Additionally, we import specific functions from the skimage library.</p><pre>import numpy as np<br>import matplotlib.pyplot as plt<br>from skimage.io import imread, imshow<br>from matplotlib.patches import Rectangle<br>from skimage.morphology import (erosion, dilation, closing, opening,<br>                                area_closing, area_opening)</pre><p>But first, what is <strong>chromaticity</strong>? Well, it is the objective specification of the quality of a color regardless of its illumination or the intensity of the color. This can be represented using the <strong>RG Chromaticity Space</strong>, a two-dimensional representation of color chromaticity — wherein the x-axis represents the Red channel, and the y-axis represents the Green channel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*1Jy5QltjSAp7L4eirXgCCQ.png" /><figcaption>(Image from Wikipedia)</figcaption></figure><p>Notice how we are only plotting the Red and Green channels in this two-dimensional representation. This is because we will represent the colors with the shade of blue using the absence of the colors Red and Green.</p><p>You might ask about the benefits of using the chromaticity space instead of the RGB or HSV color spaces of an image. Certain image processing applications can benefit well from this method — an example of this is skin color. Let’s see how to use this method.</p><p>Again, to compare how effective this method is compared to the thresholding method, we will be using this image of a Chico tree with some of its fruits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/291/1*FRnERUd6BHpDUQgRegERzg.png" /><figcaption>(Image by Author)</figcaption></figure><p>Now, let’s see where the colors in this image are in the RG chromaticity space.</p><pre>img = plt.imread(&quot;rg_chromaticity_noaxis.png&quot;)<br>chico_R = chico[:,:,0]*1.0/chico.sum(axis=2)<br>chico_G = chico[:,:,1]*1.0/chico.sum(axis=2)<br>plt.imshow(img, extent=[0, 1, 0, 1])<br>plt.scatter(chico_R.flatten(), chico_G.flatten(), <br>            color=&#39;black&#39;, alpha=0.1)<br>plt.xlim(0,1)<br>plt.ylim(0,1)<br>plt.xlabel(&#39;Red channel&#39;)<br>plt.ylabel(&#39;Green channel&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/331/1*1dVkNfnIuJi3S-4yNOKzwg.png" /><figcaption>(Image by Author)</figcaption></figure><p>Since the colors in the original image range from blue to green to yellow, we can see that the scatter plot points occupied these colors in the RG chromaticity space. From here, we can see which colors are used by the image, and we could also estimate the neighboring colors</p><p>Now, to help ease our analysis, I have created the functions that we will be using. These are the following: <br>1. rg_chromaticity, which will return the RG chromaticity values of the specified patch; <br>2. mean_and_std, which will return the mean and standard deviation value of the specified patch; <br>3. gaussian, which will compute the range of colors most similar to the specified patch using a Gaussian distribution, <br>4. rg_chromaticity_mask, which will return the resulting mask based on the specified patch.</p><pre>def rg_chromaticity(patch):<br>    return (patch[:,:,0]*1.0 / patch.sum(axis=2), <br>            patch[:,:,1]*1.0 / patch.sum(axis=2))</pre><pre>def mean_and_std(patch):<br>    return  np.mean(patch.flatten()), np.std(patch.flatten())</pre><pre>def gaussian(p, mean, std):<br>    return (np.exp(-(p-mean)**2/(2*std**2))*(1/(std*           <br>                  ((2*np.pi)**0.5))))</pre><pre>def rg_chromaticity_mask(image, from_row, from_column, <br>                         row_width, column_width, <br>                         mean_factor=1, std_factor=1):<br>    #convert to RG space<br>    image_R = image[:,:,0]*1.0/image.sum(axis=2)<br>    image_G = image[:,:,1]*1.0/image.sum(axis=2)<br>    <br>    #set the patch value<br>    image_patch = image[from_column:from_column+column_width, <br>                        from_row:from_row+row_width]<br>    fig, ax = plt.subplots(1,2, figsize=(12,6))<br>    ax[0].imshow(image)<br>    ax[0].add_patch(Rectangle((from_row, from_column), <br>                              row_width, column_width, <br>                              linewidth=3,<br>                              edgecolor=&#39;r&#39;, facecolor=&#39;none&#39;));<br>    ax[0].set_title(&#39;Original image&#39;)<br>    <br>    #convert patch to RG space<br>    patch_R, patch_G = rg_chromaticity(image_patch)</pre><pre>    #get the mean and std of patch<br>    mean_patch_R, std_patch_R = mean_and_std(patch_R)<br>    mean_patch_G, std_patch_G = mean_and_std(patch_G)<br>    <br>    #get the range of colors of the patch<br>    prob_R = gaussian(image_R, <br>                      mean_factor*mean_patch_R, <br>                      std_factor*std_patch_R)<br>    prob_G = gaussian(image_G, <br>                      mean_factor*mean_patch_G, <br>                      std_factor*std_patch_G)</pre><pre>    #get the mask<br>    prob = prob_R * prob_G<br>    <br>    #clean the mask using area_opening<br>    mask = area_opening(prob&gt;0.1)<br>    ax[1].imshow(mask);<br>    ax[1].set_title(&#39;Mask&#39;)<br>    return mask</pre><p>Let’s use these functions on the Chico fruit!</p><pre>rg_chromaticity_mask(chico, 1300, 2700, 110, 80)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*hA-4USGU4OehwX3fNc1G7A.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice how effective the RG chromaticity is in segmenting the Chico fruit on the image? The algorithm no longer captures the leaves of the tree. This is a significant improvement compared to the mask created in my previous blog post.</p><p>However, since the tree branches are also colored brown, these are also captured by the RG chromaticity algorithm. To address this, we can inspect a bit more and see that the branches’ shade of brownness is different from that of the actual fruit. Therefore, we can vary the standard deviation of the patch values to lessen the neighboring colors that it will capture. Let’s try that!</p><pre>rg_chromaticity_mask(chico, 1300, 2700, 110, 80, <br>                     mean_factor=1, std_factor=0.6)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*1nPK3X-sRoTIlBZhV6eDdA.png" /><figcaption>(Image by Author)</figcaption></figure><p>The masking image is indeed clean! Now, all we need to do is apply this masking image to the original image to get our object of interest. We could also use regionprops to determine the properties of this region if our applications require them.</p><p>This only shows that the RG chromaticity space can be effective when you have already determined the color shade you will target from the image. However, this example is relatively easy because the contrast between the green leaves and the brown fruit is quite large. So let’s try this method on a much harder image!</p><p>Here is an image of a sponge gourd vines and its fruit. Notice how the color of the fruit is almost the same as the colors of the leaves. Let’s try if the RG chromaticity can create the right masking image for this application.</p><pre>patola = imread(‘patola.jpg’)<br>imshow(patola);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/291/1*ePeQzHkYgfoVm5k0vWYaPw.png" /><figcaption>(Image by Author)</figcaption></figure><p>By applying RG chromaticity on the patch of fruit, we have:</p><pre>mask_patola = rg_chromaticity_mask(patola, 1600, 2450, 120, 350)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*OdZ7PXMH5UnDK-HBh0dh3Q.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice how even though the image is still uncleaned, the masking image was still able to pinpoint and detect the fruits! This only shows that it can detect objects even though the colors are approximately the same as the other objects in the image.</p><p>The only thing left to do is to clean this image using morphological operations!</p><pre>element = np.array([[0,1,1,0],<br>                    [1,1,1,1],<br>                    [1,1,1,1],<br>                    [0,1,1,0]])</pre><pre>def multi_dil(im, num, element=element):<br>    for i in range(num):<br>        im = dilation(im, element)<br>    return im</pre><pre>def multi_ero(im, num, element=element):<br>    for i in range(num):<br>        im = erosion(im, element)<br>    return im</pre><pre>area_opened = area_opening(mask, 50000)<br>multi_eroded = multi_ero(area_opened, 3)<br>opened = area_opening(multi_eroded, 35000)<br>multi_dilated = multi_dil(opened, 3)<br>closed = area_closing(multi_dilated, 35000)<br>imshow(closed)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/291/1*wRcc15HvvyibbVmuCGgXFA.png" /><figcaption>(Image by Author)</figcaption></figure><p>We have successfully created a masking image that focuses on the five fruits in the image by applying morphological operations. However, we have also captured a leaf in the masking image because its colors are close to the fruit. But no worries! Since the image is already clean, this unnecessary object can easily be removed using regionprops. I guess we can use the major_axis_length and minor_axis_length properties to filter the leaf in the masking image — but I’m going to leave that for you to do!</p><p>Overall, the examples presented here only prove that indeed, the RG chromaticity space can be used to create a good masking image even though the difference in color shades is really tight!</p><p><strong>In summary</strong></p><p>We have explored how to segment images using the RG Chromaticity method properly. Given a color patch of an object in the image, we can detect all regions in the original image that is approximately the same as the color patch. Moreover, we have explored how we can further adjust the mean value and standard deviation value to filter out colors of similar shade further.</p><p>Segmenting images using the RG Chromaticity method is indeed a powerful tool in image processing. I believe that this one of the methods you need to master to help you develop your image processing projects!</p><p><em>Want to learn more? Check out my GitHub repository at this </em><a href="https://github.com/jephraim-manansala/chromaticity"><em>link</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9568c3276db6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/image-processing-with-python-image-segmentation-using-rg-chromaticity-9568c3276db6">Image Processing with Python: Image Segmentation using RG Chromaticity</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing with Python: Image Segmentation using Thresholding Methods]]></title>
            <link>https://medium.com/swlh/image-processing-with-python-image-segmentation-using-thresholding-methods-423ecdaf8ab4?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/423ecdaf8ab4</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[image-processing]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Fri, 29 Jan 2021 06:18:22 GMT</pubDate>
            <atom:updated>2021-02-14T13:34:04.050Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to pinpoint and segment objects based on their color channels?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*MRGqceM58xeIm9FaXfxLLg.png" /><figcaption>(Image by Author)</figcaption></figure><p>In this post, we will explore how to conduct image segmentation using trial and error thresholding and Otsu’s method. Moreover, we will explore how the RGB and HSV color spaces can be useful in segmenting images.</p><p>Let’s begin.</p><p>As usual, we import libraries such as numpy and matplotlib. Additionally, we import specific functions from the skimage library.</p><pre>import numpy as np<br>import matplotlib.pyplot as plt<br>from skimage.io import imread, imshow<br>from skimage.color import rgb2gray, rgb2hsv<br>from skimage.morphology import area_opening<br>from skimage.exposure import histogram<br>from skimage.filters import threshold_otsu</pre><p>Let’s use this image of a <em>Chico</em> tree with some of its fruits.</p><pre>chico = imread(‘chico.jpg’)<br>imshow(chico);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/291/1*FRnERUd6BHpDUQgRegERzg.png" /><figcaption>(Image by Author)</figcaption></figure><p>We have previously discussed how binarizing an image can be useful since it simplifies the analysis when the pixel values are constrained only to 0s and 1s. So far, we have been using trial and error to find the arbitrary threshold value that best captures our desired shape of the objects.</p><pre>th_values = np.linspace(0, 1, 11)<br>fig, axis = plt.subplots(2, 5, figsize=(15,8))</pre><pre>chico_gray = rgb2gray(chico)<br>for th, ax in zip(th_values, axis.flatten()):<br>    <br>    chico_binarized = chico_gray &lt; th<br>    ax.imshow(chico_binarized)<br>    ax.set_title(&#39;$Threshold = %.2f$&#39; % th)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1nqbLowzFQk1wbcKjSHcYg.png" /><figcaption>(Image by Author)</figcaption></figure><p>From this image, we might choose a thresholding value of 0.40 to 0.60 since it captures most of the leaves in the tree. However, this method is <strong>subjective</strong>. Let’s see if we can automate determining the thresholding value by looking at the intensity values histogram.</p><pre>freq, bins = histogram(chico_gray)<br>plt.step(bins, freq*1.0/freq.sum())<br>plt.xlabel(&#39;Intensity value&#39;)<br>plt.ylabel(&#39;Fraction of pixels&#39;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/1*Vy4JjLFvfPScb9-tc9I-mw.png" /><figcaption>(Image by Author)</figcaption></figure><p>From the histogram, we can see that there are two distributions in the image. This also explains why the optimal value in the trial and error method is in the range of 0.40 to 0.60 thresholding value. However, what if I told you that there is an automated way to determine this threshold value? This solution is called the Otsu’s method.</p><p><strong>Otsu’s method</strong> assumes that the image is composed of a background and a foreground. This method works by minimizing the intra-class variance or maximizing the inter-class variance. Let’s see how this works.</p><pre>def masked_image(image, mask):<br>    r = image[:,:,0] * mask<br>    g = image[:,:,1] * mask<br>    b = image[:,:,2] * mask<br>    return np.dstack([r,g,b])</pre><pre>fig, ax = plt.subplots(1, 2, figsize=(12,6))</pre><pre>thresh = threshold_otsu(chico_gray)<br>chico_otsu  = chico_gray &lt; thresh<br>ax[0].imshow(chico_otsu)</pre><pre>filtered = masked_image(chico, chico_otsu)<br>ax[1].imshow(filtered)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*w8vxQaZhXgZ-ZwRl1zMuwQ.png" /><figcaption>(Image by Author)</figcaption></figure><p>Amazing, right? Now, we could bother less on determining the best threshold value in binarizing the image! However, we should ALWAYS remember this method’s underlying assumption: <strong>the image is composed of a background and a foreground.</strong> This means that if we take a look at the histogram, there should only be two separable distributions.</p><p>Now, dividing the background and the foreground can be really useful. However, most of our image segmentation problems is not a background-foreground problem. For example, we cannot simply use Otsu’s method in segmenting the Chico fruits from the leaves — this is because both are in the foreground of the image.</p><p>To address this problem, we should look at the color spaces of the images. As I have mentioned in my previous posts, any color is simply a combination of varying pixel intensity values in the image’s channels. Therefore, to segment a specific color in an image, we must use the thresholding method on the particular ratios of pixel intensity values in the three channels. Let’s try this in the RGB color space.</p><pre>fig, ax = plt.subplots(1, 3, figsize=(15,6))</pre><pre>ax[0].imshow(chico[:,:,0], cmap=&#39;Reds&#39;)<br>ax[0].set_title(&#39;Red&#39;)</pre><pre>ax[1].imshow(chico[:,:,1], cmap=&#39;Greens&#39;)<br>ax[1].set_title(&#39;Green&#39;)</pre><pre>ax[2].imshow(chico[:,:,2], cmap=&#39;Blues&#39;)<br>ax[2].set_title(&#39;Blue&#39;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m3jvKvOuc9aQaggosGYsrg.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice how the background has a high-intensity value while the objects themselves have a low-intensity value in all of the channels. This is because the background is the sky — which technically is the source of light. Thus, it naturally has a higher intensity value. This makes segmenting the image to background-foreground easier. However, this also makes segmenting images in the foreground much more challenging because of the much tighter pixel intensity range.</p><pre>figure, ax = plt.subplots(1,2, figsize=(12,6))</pre><pre>chico_red = chico[:,:,0]<br>chico_green = chico[:,:,1]<br>chico_blue = chico[:,:,2]</pre><pre>binarized = ((chico_red &lt; 200) &amp; (chico_red &gt; 75) &amp; <br>             (chico_green &lt; 120) &amp; (chico_green &gt; 50) &amp; <br>             (chico_blue &gt; 20))</pre><pre>opened = area_opening(binarized, 5000)<br>ax[0].imshow(binarized)<br>ax[1].imshow(masked_image(chico, opened))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*ueUYHWf2_UFeInwyxSXiow.png" /><figcaption>(Image by Author)</figcaption></figure><p>Difficult, right? Even if we used area_opening to clean the image, it still does not show good results. Now, let’s try if we can make this easier by using the HSV color space.</p><pre>chico_hsv = rgb2hsv(chico)<br>fig, ax = plt.subplots(1, 3, figsize=(15,6))</pre><pre>ax[0].imshow(chico_hsv[:,:,0], cmap=’hsv’)<br>ax[0].set_title(‘Hue’)</pre><pre>ax[1].imshow(chico_hsv[:,:,1], cmap=’hsv’)<br>ax[1].set_title(‘Saturation’)</pre><pre>ax[2].imshow(chico_hsv[:,:,2], cmap=’hsv’)<br>ax[2].set_title(‘Value’);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qsSEww6hBqjM4KKnN7MMxQ.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice how the HSV color space can visualize the image’s objects much better than the RGB color space. Let’s try to use thresholding methods using this color space.</p><pre>figure, ax = plt.subplots(1,2, figsize=(12,6))</pre><pre>chico_hue = chico_hsv[:,:,0] <br>chico_sat = chico_hsv[:,:,1]<br>chico_val = chico_hsv[:,:,2]</pre><pre>binarized_hsv = ((chico_hue &lt; 0.18) &amp; (chico_hue &gt; 0.05) &amp; <br>                 (chico_sat &gt; 0.55)  &amp; (chico_sat &lt; 0.80))<br>opened = area_opening(binarized_hsv, 5000)<br>ax[0].imshow(binarized_hsv)<br>ax[1].imshow(masked_image(chico, opened))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*z0RF7gXGhd6YyB_XJ_-bmQ.png" /><figcaption>(Image by Author)</figcaption></figure><p>See? Now, that’s easier! This can be attributed to the Hue channel of the HSV color space that clearly identifies the objects on the images based on their hue.</p><p>Ultimately, the choice of what color space should be used to segment the objects in the images is upon the discretion of the data scientist. I hope that by reading this post, you were able to get the intuition needed to implement your own image processing projects!</p><p><strong>In summary</strong></p><p>We have explored how to properly segment images to the background and foreground using the trial and error thresholding method and the Otsu’s Method. Moreover, we have also explored how the RGB and HSV color spaces can help pinpoint and segment objects in images.</p><p><em>Want to learn more? Check out my GitHub repository at this </em><a href="https://github.com/jephraim-manansala/thresholding"><em>link</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=423ecdaf8ab4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/image-processing-with-python-image-segmentation-using-thresholding-methods-423ecdaf8ab4">Image Processing with Python: Image Segmentation using Thresholding Methods</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Image Processing with Python: Connected Components and Region Labeling]]></title>
            <link>https://medium.com/swlh/image-processing-with-python-connected-components-and-region-labeling-3eef1864b951?source=rss-8de95334d9ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/3eef1864b951</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[JManansala]]></dc:creator>
            <pubDate>Fri, 29 Jan 2021 02:33:41 GMT</pubDate>
            <atom:updated>2021-02-14T13:33:09.620Z</atom:updated>
            <content:encoded><![CDATA[<h4>How can we extract and quantify region properties in an image?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*1El7CsWwP_RoJL-ioLx0ug.png" /><figcaption>(Image by Author)</figcaption></figure><p>In this post, we will explore how to automatically detect, label, and measure objects in images using connected components. This method addresses the shortcomings of blob detection methods by grouping pixels based on their connectivity.</p><p>Let’s begin.</p><p>As usual, we import libraries such as numpy and matplotlib. Additionally, we import specific functions from the skimage library.</p><pre>import numpy as np<br>import pandas as pd<br>import matplotlib.pyplot as plt<br>from skimage.io import imread, imshow<br>from skimage.color import rgb2gray<br>from skimage.morphology import (erosion, dilation, closing, opening,<br>                                area_closing, area_opening)<br>from skimage.measure import label, regionprops, regionprops_table</pre><p>What are <strong>connected components</strong>? Basically, it allows us to detect objects with irregular shapes and sizes based on the pixels’ connectivity to their neighbors. However, the use of connected components operations heavily relies on adequately cleaning the image using morphological operation.</p><p>Let’s try this on an image I took when I visited the <em>Pinto Art Museum</em> in Rizal Province, Philippines. Our goal will be to extract quantifiable features on each of the paintings on the image.</p><pre>painting = imread(‘painting.jpg’)<br>imshow(painting)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/291/1*0c88v1q0f4tDi2X-bTxveA.png" /><figcaption>(Image by Author)</figcaption></figure><p>If we apply the connected components algorithm on the raw image, this would almost immediately fail because of the various colors in the image. Thus, the first step would be to binarize the image using an arbitrary threshold value.</p><pre>gray_painting = rgb2gray(painting)<br>binarized = gray_painting&lt;0.55<br>imshow(binarized)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*TcYDPvTGNOM8ekg_UP0Jjg.png" /><figcaption>(Image by Author)</figcaption></figure><p>We have set the arbitrary threshold value to maximize the shape of the frame of each painting. However, notice that the resulting image has several holes in the regions due to the light color used in the paintings. Let’s try to fix these using morphological operations.</p><pre>square = np.array([[1,1,1],<br>                   [1,1,1],<br>                   [1,1,1]])</pre><pre>def multi_dil(im, num, element=square):<br>    for i in range(num):<br>        im = dilation(im, element)<br>    return im</pre><pre>def multi_ero(im, num, element=square):<br>    for i in range(num):<br>        im = erosion(im, element)<br>    return im</pre><pre>multi_dilated = multi_dil(binarized, 7)<br>area_closed = area_closing(multi_dilated, 50000)<br>multi_eroded = multi_ero(area_closed, 7)<br>opened = opening(multi_eroded)<br>imshow(opened)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/339/1*_8GM3aRpggKue9Z-MSkPSA.png" /><figcaption>(Image by Author)</figcaption></figure><p>We have applied morphological operations such as successive dilation to close the pixels of the painting frame, area_closing to fill the holes inside the painting frame, successive erosion to restore the original shape of the objects, and finally, opening to remove the noise on the image. After applying these operations, the image is now ready for connected components operations!</p><p>The first function is label which labels the regions of the input image depending on the connectivity of the pixels to each other. As long as neighboring pixels share the same value, they will be labeled as a single region. This function will return a labeled array where all connected regions are assigned the same integer value.</p><p>After this, we can measure the properties of the regions in the labeled image by inputting the labeled array to the regionprops function. Several properties can be measured by this function, some of which are the <em>area</em>, <em>perimeter</em>, <em>major axis lengths</em>, <em>minor axis length</em>, and many more! Let’s try to use these functions.</p><pre>label_im = label(opened)<br>regions = regionprops(label_im)<br>imshow(label_im)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*mJ-mfE-bGH3DIaKP1u3Jpw.png" /><figcaption>(Image by Author)</figcaption></figure><p>The label function will label the regions from left to right, and from top to bottom. For this reason, region# 1 will be on the top-rightmost region in the image until all regions are assigned with an integer number. Each region’s color represents its integer region number and can be checked using the color bar on the side. Notice how there are 20+ regions on the image while the visible regions in the image are only about 10. This is because there is still noise present in the image and the noise is also considered as a region. Let’s try to filter these using the properties of each region.</p><p>Using the regionprops_table function, we can extract features and properties for each of the regions identified. In this example, we will be exploring the property features such as <em>area</em>, <em>convex_area</em>, <em>bbox_area</em>, <em>extent</em>, <em>mean_intensity</em>, <em>solidity</em>, <em>eccentricity</em>, and <em>orientation</em>. Each of these properties quantifies and explains the characteristics of each region compared to other regions. For example, the <em>extent </em>parameter measures the object’s fill area to its bounding box, while the <em>mean_intensity </em>measures the mean intensity value of the pixels in the region.</p><pre>properties = [‘area’,’convex_area’,’bbox_area’, ‘extent’,  <br>              ‘mean_intensity’, ‘solidity’, ‘eccentricity’, <br>              ‘orientation’]<br>pd.DataFrame(regionprops_table(label_im, gray_painting, <br>             properties=properties))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*H6HlgYzJTkbsJJBFP4SO6Q.png" /><figcaption>(Image by Author)</figcaption></figure><p>Notice how some regions’ area is almost negligible — these areas can be attributed to noise and can be filtered out using these properties.</p><p>We can also notice that region# 0 has one of the region’s most significant areas. However, region# 0 is always assigned by the label function for the background or the pixel intensities with 0. Thus, we should always filter out this region in our analysis.</p><p>Finally, notice how some regions have almost the same area covered while also almost being consecutive in the region numbering. This is because paintings are arranged in a lattice, and as mentioned earlier, the label function labels the regions from left to the right and then from top to bottom.</p><p>With these observations, we can filter our unnecessary regions in the image.</p><pre>masks = []<br>bbox = []<br>list_of_index = []<br>for num, x in enumerate(regions):<br>    area = x.area<br>    convex_area = x.convex_area<br>    if (num!=0 and (area&gt;100) and (convex_area/area &lt;1.05)<br>    and (convex_area/area &gt;0.95)):<br>        masks.append(regions[num].convex_image)<br>        bbox.append(regions[num].bbox)   <br>        list_of_index.append(num)<br>count = len(masks)</pre><p>In filtering, we define an if-else statement to determine which regions will be used based on observed conditions — such as, regions that are NOT region# 0, regions with an area greater than 100, and regions with a convex_area to area ratio that is approximately equal to 1 <em>(to ensure that the object is indeed rectangular)</em>. From this, we can individually extract the desired regions from the image.</p><pre>fig, ax = plt.subplots(2, int(count/2), figsize=(15,8))<br>for axis, box, mask in zip(ax.flatten(), bbox, masks):<br>    red  =  painting[:,:,0][box[0]:box[2], box[1]:box[3]] * mask<br>    green = painting[:,:,1][box[0]:box[2], box[1]:box[3]] * mask<br>    blue  = painting[:,:,2][box[0]:box[2], box[1]:box[3]] * mask<br>    image = np.dstack([red,green,blue])<br>    axis.imshow(image)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*jgvBKWKfiB_ZRGL8p1vj_A.png" /><figcaption>(Image by Author)</figcaption></figure><p>To extract the image within each region, we can use the bounding box attribute of region_props to determine each region’s exact coordinates in the image. From there, we would only need to slice the original image to get the image within each region.</p><p>We can also highlight the desired regions in the original image by creating a mask that hides unnecessary regions of the image. This can be done by combining the labeled array of the desired regions and using the combined mask on the original image.</p><pre>rgb_mask = np.zeros_like(label_im)<br>for x in list_of_index:<br>    rgb_mask += (label_im==x+1).astype(int)<br>red  =  painting[:,:,0] * rgb_mask<br>green = painting[:,:,1] * rgb_mask<br>blue  = painting[:,:,2] * rgb_mask<br>image = np.dstack([red, green, blue])<br>imshow(image)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/291/1*WGshvb6XWzQ-KhEDlrbxnQ.png" /><figcaption>(Image by Author)</figcaption></figure><p>And that’s it! I hope you were able to realize the potential of label, regionprops, and regionprops_table function in the skimage.measure library. This can be a very powerful tool when conducting image processing projects in Python. Use it well!</p><p><strong>In summary</strong></p><p>We have explored how the connected components can extract features and information on each region of the image. This skillset can be particularly useful when getting creating machine learning algorithms from image datasets. Instead of employing a <em>blackbox</em> <em>approach</em>, we can opt for a more interpretable method by extracting quantifiable information from the image regions.</p><p><em>Want to learn more? Check out my GitHub repository at this </em><a href="https://github.com/jephraim-manansala/connected-components"><em>link</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3eef1864b951" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/image-processing-with-python-connected-components-and-region-labeling-3eef1864b951">Image Processing with Python: Connected Components and Region Labeling</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>