<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Oleksii Sheremet on Medium]]></title>
        <description><![CDATA[Stories by Oleksii Sheremet on Medium]]></description>
        <link>https://medium.com/@dynamic_phlox_coyote_991?source=rss-c3f778e21722------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*4bRAfxFxxgHvbs9rN2mkyA.jpeg</url>
            <title>Stories by Oleksii Sheremet on Medium</title>
            <link>https://medium.com/@dynamic_phlox_coyote_991?source=rss-c3f778e21722------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:25:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dynamic_phlox_coyote_991/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Image Classification Pipeline with Intel Neural Compute Stick 2 (NCS2)]]></title>
            <link>https://medium.com/data-science/image-classification-pipeline-with-intel-neural-compute-stick-2-ncs2-2a69aab8b570?source=rss-c3f778e21722------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a69aab8b570</guid>
            <category><![CDATA[image-classification]]></category>
            <category><![CDATA[neural-compute-stick]]></category>
            <category><![CDATA[pipeline]]></category>
            <category><![CDATA[openvino]]></category>
            <category><![CDATA[mobilenet]]></category>
            <dc:creator><![CDATA[Oleksii Sheremet]]></dc:creator>
            <pubDate>Wed, 09 Sep 2020 02:15:12 GMT</pubDate>
            <atom:updated>2020-09-09T03:16:48.380Z</atom:updated>
            <content:encoded><![CDATA[<h4>Go from your dataset to image classification with NCS2 with this full-on guide</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z6okCVQ9A2Tdl-Q0Pdeq-Q.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://www.adobe.com/ru/products/photoshop.html">Adobe Photoshop</a></figcaption></figure><p><strong>Introduction</strong></p><p>What we’ll cover in this story:</p><ul><li>Installing OpenVINO toolkit for Ubuntu.</li><li>Data preprocessing and model training with Google Colab.</li><li>Saving a TensorFlow model to protocol buffer (pb) format.</li><li>Converting a TensorFlow model to intermediate representation.</li><li>Running a trained model and prediction obtaining with NCS2 device.</li></ul><p>In the past few years, there has been an active introduction of machine learning for industrial tasks solvings. Machine learning allows you to solve problems that humans cannot cope with. This can be work in hard-to-reach places, in hazardous chemical production, in conditions of increased radiation, etc. Machine learning is applicable in areas where human intelligence can also be applied, but is ineffective: predicting critical failures, preventing sudden equipment failures, condition-based maintenance, predicting the remaining life of equipment.</p><p>Often, in industrial enterprises there is no opportunity to use workstations for data analysis and processing. Therefore, special devices that can be easily connected to existing equipment (most often an industrial microcomputer) are in demand. The computational module <a href="https://ark.intel.com/content/www/us/en/ark/products/140109/intel-neural-compute-stick-2.html">NCS2</a> is one of such devices.</p><p>Our company <a href="http://ai-labs.org/">AI Labs</a> periodically receives requests for solving some industrial problems related to machine learning. Not so long ago, we had the opportunity to take advantage of the opportunities provided by NCS2. As a result, I developed a pipeline for image classification using NCS2. I would like to share the results of this work with the readers.</p><p><strong>Installing OpenVINO toolkit for Ubuntu</strong></p><p>First of all, you need to download and install the current version of <a href="https://docs.openvinotoolkit.org/">OpenVINO</a>. You can get OpenVINO from <a href="https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit/choose-download/linux.html">here</a>. You can find a detailed installation guide <a href="https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html">here</a>. Quick installation guide:</p><ol><li>Go to the folder (for example, <em>Downloads</em>) with the downloaded archive:</li></ol><pre>cd ~/Downloads/</pre><p>2. Unpack the .tgz file (where <em>&lt;version&gt;</em> is the version of the downloaded archive, you will see it in the file name):</p><pre>tar -xvzf l_openvino_toolkit_p_&lt;version&gt;.tgz</pre><p>3. Go to the <em>l_openvino_toolkit_p_&lt;version&gt;</em> directory:</p><pre>cd l_openvino_toolkit_p_&lt;version&gt;</pre><p>4. Run the installation script as root:</p><pre>sudo ./install.sh</pre><p>5. Follow the instructions on your screen.</p><p>6. Set environment variables by editing the <em>.bashrc</em> file. Go to the user directory and open <em>.bashrc</em> file for editing (for example, with <em>nano</em>):</p><pre>cd ~<br>nano .bashrc</pre><p>Add the following line to the end of the .bashrc file, save it and reload:</p><pre>source /opt/intel/openvino/bin/setupvars.sh<br>source .bashrc</pre><p>That’s all. All the necessary functionality for local work with NCS2 has been prepared. You can perform data preprocessing and model training.</p><p><strong>Data preprocessing and model training with Google Colab</strong></p><p><a href="https://www.kaggle.com/alessiocorrado99/animals10">Animals-10</a> dataset was used to solve the classification problem. To simplify the problem, 5 classes (species of animals) were selected from Animals-10: cat, dog, chicken, horse, sheep. Images related to each of the classes are grouped into corresponding folders (‘cat’, ‘dog’, ‘chicken’, ‘horse’, ‘sheep’). The model was trained in Google Colab. Let’s take a closer look.</p><p>First of all, you need to install a version of TensorFlow suitable for working with NCS2:</p><pre>!pip install tensorflow==1.15.2</pre><p>Import libraries:</p><pre>import tensorflow.compat.v1 as tf<br>import tensorflow_hub as hub<br>import matplotlib.pylab as plt<br>import numpy as np</pre><p>Mount the Google Drive (with dataset) to Google Colab:</p><pre>from google.colab import drive<br>drive.mount(‘/content/drive’)</pre><p>Go to the directory where the data was uploaded (‘animals’):</p><pre>%cd ‘drive/My Drive/animals’</pre><p>The folder structure should be as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/92/1*FFBD1LHt6u0j7nIlVREFrA.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://en.wikipedia.org/wiki/Tree_(command)">tree</a> command</figcaption></figure><p>Create a list of classes (labels):</p><pre>image_dir = &#39;animals&#39;<br>import os</pre><pre>class_labels = []<br>for x in tf.gfile.Walk(image_dir):<br>    try:<br>        y = os.path.basename(x[0])<br>        if y != &#39;animals&#39;:<br>            class_labels.append(y)<br>    except:<br>        pass<br>print(class_labels)</pre><p>We get the following list:</p><pre>[‘cat’, ‘chicken’, ‘dog’, ‘horse’, ‘sheep’]</pre><p>Based on this list, index ‘0’ corresponds to the image of a cat, ‘1’ — a chicken, ‘2’ — a dog, ‘3’ — a horse, ‘4’ — a sheep.</p><p>Set the size of the images (224 x 224). We use <a href="https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator">ImageDataGenerator</a> to create a flow of images from folders when training the model.</p><pre>IMAGE_SIZE = (224,224)</pre><pre>image_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1/255, validation_split=0.2)</pre><pre>training_set = image_generator.flow_from_directory(str(image_dir),                                              target_size=IMAGE_SIZE, subset=&#39;training&#39;)</pre><pre>validation_set = image_generator.flow_from_directory(str(image_dir),                                              target_size=IMAGE_SIZE, subset=&#39;validation&#39;)</pre><p>Retrieving pre-trained MobileNet network:</p><pre>feature_extractor = tf.keras.applications.MobileNet(weights=&#39;imagenet&#39;, <br>                                include_top=False,<br>                                input_shape=(IMAGE_SIZE+(3,))) <br>feature_extractor.trainable=False</pre><p>Build model:</p><pre>try:<br>    del model<br>except:<br>    pass</pre><pre>x=feature_extractor.output<br>x=tf.keras.layers.GlobalAveragePooling2D()(x)</pre><pre>classifier=tf.keras.layers.Dense(label_batch.shape[1],activation=&#39;softmax&#39;)(x)</pre><pre>model=tf.keras.Model(inputs=feature_extractor.input,outputs=classifier)</pre><pre>model.build((None,)+IMAGE_SIZE+(3,))<br>model.summary()</pre><p>Compile the model:</p><pre>model.compile(<br>    optimizer=tf.keras.optimizers.Adam(),<br>    loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),<br>    metrics=[&#39;acc&#39;])</pre><p>Callback method for collecting logs:</p><pre>class CollectBatchStats(tf.keras.callbacks.Callback):<br>  def __init__(self):<br>    self.batch_losses = []<br>    self.batch_acc = []<br>    self.validation_losses = []<br>    self.validation_acc = []</pre><pre>def on_train_batch_end(self, batch, logs=None):<br>    self.batch_losses.append(logs[&#39;loss&#39;])<br>    self.batch_acc.append(logs[&#39;acc&#39;])<br>    try:<br>        self.validation_losses.append(logs[&#39;val_loss&#39;])<br>        self.validation_acc.append(logs[&#39;val_acc&#39;])<br>    except:<br>        self.validation_losses.append(None)<br>        self.validation_acc.append(None)<br>    self.model.reset_metrics()</pre><p>Train the model:</p><pre>steps_per_epoch = np.ceil(training_set.samples/training_set.batch_size)</pre><pre>batch_stats_callback = CollectBatchStats()</pre><pre>history = model.fit_generator(training_set, epochs=5,<br>                              steps_per_epoch=steps_per_epoch,<br>                              validation_data=validation_set,<br>                              callbacks = [batch_stats_callback])</pre><p>Check predictions:</p><pre>for image_batch, label_batch in validation_set:<br>    print(&quot;Image batch shape: &quot;, image_batch.shape)<br>    print(&quot;Label batch shape: &quot;, label_batch.shape)<br>    break</pre><p>We have the following shapes:</p><pre>Image batch shape:  (32, 224, 224, 3)<br>Label batch shape:  (32, 5)</pre><p>We can get predictions and their classes:</p><pre>predictions = model.predict(image_batch)<br>predicted_class = np.argmax(predictions, axis=-1)</pre><p>Visualize the result:</p><pre>plt.figure(figsize=(12,10))<br>plt.subplots_adjust(hspace=0.5)<br>for n in range(30):<br>    plt.subplot(6,5,n+1)<br>    plt.imshow(image_batch[n])<br>    plt.title(f&#39;pred: {class_labels[predicted_class[n]]}\norig: {class_labels[np.array(label_batch[n]).argmax()]}&#39;)<br>    plt.axis(&#39;off&#39;)<br>_ = plt.suptitle(&quot;Animals&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*s_iM9ytLTIuqJJ90Jn1FLQ.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://matplotlib.org/">matplotlib</a> module</figcaption></figure><p><strong>Saving a TensorFlow model to protocol buffer (pb) format</strong></p><p>Freeze and save model:</p><pre>from tensorflow.compat.v1.keras import backend as K</pre><pre>session = K.get_session()<br>graph = session.graph<br>with graph.as_default():<br>    freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference([]))<br>    output_names = [out.op.name for out in model.outputs]<br>    output_names += [v.op.name for v in tf.global_variables()]<br>    input_graph_def = graph.as_graph_def()<br>    for node in input_graph_def.node:<br>        node.device = &quot;&quot;<br>    frozen_graph = tf.graph_util.convert_variables_to_constants(session, input_graph_def, output_names, freeze_var_names)<br>tf.train.write_graph(frozen_graph, &quot;animals&quot;, &quot;animals.pb&quot;, as_text=False)</pre><p>As a result, the trained model is saved to the <em>animals.pb</em> file.</p><p><strong>Converting a TensorFlow model to intermediate representation</strong></p><p>There is a detailed <a href="https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html">tutorial</a> on converting TensorFlow models. To convert the trained model, you need to perform the operations described below.</p><p>Go to the <em>model_optimizer</em> folder (you need <em>mo_tf.py</em>):</p><pre>cd ~/intel/openvino/deployment_tools/model_optimizer</pre><p>Run <em>mo_tf.py </em>with following command (please note that you must use a scaling factor of 255, the same as in the image preprocessing stage):</p><pre>python mo_tf.py --input_model &lt;path_to_model&gt;/animals.pb --batch 1 --scale 255</pre><p>If the conversion is successful, you will see the following messages in the console:</p><pre>[ SUCCESS ] Generated IR version 10 model.<br>[ SUCCESS ] XML file: /home/user_name/intel/openvino_2020.4.287/deployment_tools/model_optimizer/./animals.xml<br>[ SUCCESS ] BIN file: /home/user_name/intel/openvino_2020.4.287/deployment_tools/model_optimizer/./animals.bin<br>[ SUCCESS ] Total execution time: 31.81 seconds.<br>[ SUCCESS ] Memory consumed: 370 MB.</pre><p>Thus, as a result of the conversion, we get 3 files: <em>animals.xml</em> and <em>animals.bin, animals.mapping</em>. It is advisable to copy these files to a separate folder (for example, <em>test_folder</em>).</p><p><strong>Running a trained model and prediction obtaining with NCS2 device</strong></p><p>To check the performance of the model on NCS2, you can use the <em>classification_sample.py</em> script, which is included in OpenVINO (/home/user_name/intel/openvino/deployment_tools/inference_engine/samples/python/classification_sample/classification_sample.py). Copy this script to the folder with 3 trained model files (<em>animals.xml, animals.bin, animals.mapping</em>). Also, to check the functionality, you can copy several images (for example, <em>001.jpeg</em> and <em>002.jpeg</em>) to the same folder.</p><p>Connect NCS2 to USB, go to the <em>test_folder</em> and run (<em>MYRIAD</em> key to run the model on NCS2):</p><pre>python classification_sample.py -m animals.xml -i 001.jpeg -d MYRIAD</pre><p>Test image and output from NCS2 in console:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*_2o2EnOdBZhAlXGML_rxJA.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://www.adobe.com/ru/products/photoshop.html">Adobe Photoshop</a></figcaption></figure><pre>python classification_sample.py -m animals.xml -i 002.jpeg -d MYRIAD</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*D2sYMURJO7_X7bZoUJPLuA.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://www.adobe.com/ru/products/photoshop.html">Adobe Photoshop</a></figcaption></figure><p>As you can see from the examples presented, the classification is correct. Image <em>001.jpeg</em> shows a horse (classid=3), and image <em>002.jpeg</em> shows a chicken (classid=1).</p><p><strong>Conclusion</strong></p><p>The presented pipeline allows you to quickly go through all the stages from data preprocessing and model training to its testing based on the NCS2 module. As practice has shown, NCS2 copes well with the task of image classification and can be recommended for solving simple industrial classification tasks.</p><p><strong>References</strong></p><p><a href="https://ark.intel.com/content/www/us/en/ark/products/140109/intel-neural-compute-stick-2.html">Intel® Neural Compute Stick 2</a></p><p><a href="https://docs.openvinotoolkit.org/">OpenVINO toolkit</a></p><p><a href="https://arxiv.org/pdf/1704.04861.pdf">MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications</a></p><p><a href="https://github.com/fchollet/deep-learning-models/releases/tag/v0.6">MobileNet</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a69aab8b570" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/image-classification-pipeline-with-intel-neural-compute-stick-2-ncs2-2a69aab8b570">Image Classification Pipeline with Intel Neural Compute Stick 2 (NCS2)</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Extract annotations from CVAT XML file into mask files in Python]]></title>
            <link>https://medium.com/data-science/extract-annotations-from-cvat-xml-file-into-mask-files-in-python-bb69749c4dc9?source=rss-c3f778e21722------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb69749c4dc9</guid>
            <category><![CDATA[annotations]]></category>
            <category><![CDATA[labelling]]></category>
            <category><![CDATA[data-labeling]]></category>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[xml]]></category>
            <dc:creator><![CDATA[Oleksii Sheremet]]></dc:creator>
            <pubDate>Sun, 09 Aug 2020 23:40:33 GMT</pubDate>
            <atom:updated>2020-08-10T02:06:21.527Z</atom:updated>
            <content:encoded><![CDATA[<h4>An approach to get mask files from the data contained in the CVAT XML</h4><p>The <a href="https://github.com/opencv/cvat">Computer Vision Annotation Tool (CVAT)</a> is a well-known image annotation tool. The results of the data labelers work can be imported in XML file. This XML file contains all the necessary information about the markup. However, for image segmentation task it is necessary to have masks in the form of image files (JPEG, GIF, PNG, etc.). In other words, having the markup coordinates in the CVAT XML file, you need to draw the corresponding masks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HFMWTJhWrpUXL_XIghMD3g.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://www.adobe.com/ru/products/photoshop.html">Adobe Photoshop</a></figcaption></figure><p>If the data labelers worked with images in a higher resolution than it is supposed to be used for training, then the task will become more complicated. It is necessary to take into account the influence of the image compression factor on the numerical values of the cue points presented in the XML file.</p><p>All code for extracting annotations is implemented as a script in Python. The lxml library is used for parsing XML. It is a fast and flexible solution for handling XML and HTML markup The <a href="https://lxml.de/">lxml</a> package has XPath and XSLT support, including an API for SAX and an API for compatibility with C modules.</p><p>The <a href="https://tqdm.github.io/">tqdm</a> package is used as a progress bar to illustrate the processing of a large number of files.</p><p>Let’s take a closer look. Import libraries:</p><pre>import os<br>import cv2<br>import argparse<br>import shutil<br>import numpy as np<br>from lxml import etree<br>from tqdm import tqdm</pre><p>A useful function for creating a new directory and recursively deleting the contents of an existing one:</p><pre>def dir_create(path):<br>    if (os.path.exists(path)) and (os.listdir(path) != []):<br>        shutil.rmtree(path)<br>        os.makedirs(path)</pre><pre>    if not os.path.exists(path):<br>        os.makedirs(path)</pre><p>The arguments for the script in question are the following data: directory with input images, input file with CVAT annotation in XML format, directory for output masks and scale factor for images. A function for parsing arguments from the command line:</p><pre>def parse_args():<br>    parser = argparse.ArgumentParser(<br>        fromfile_prefix_chars=&#39;@&#39;,<br>        description=&#39;Convert CVAT XML annotations to contours&#39;<br>    )</pre><pre>parser.add_argument(<br>        &#39;--image-dir&#39;, metavar=&#39;DIRECTORY&#39;, required=True,<br>        help=&#39;directory with input images&#39;<br>    )</pre><pre>parser.add_argument(<br>        &#39;--cvat-xml&#39;, metavar=&#39;FILE&#39;, required=True,<br>        help=&#39;input file with CVAT annotation in xml format&#39;<br>    )</pre><pre>parser.add_argument(<br>        &#39;--output-dir&#39;, metavar=&#39;DIRECTORY&#39;, required=True,<br>        help=&#39;directory for output masks&#39;<br>    )</pre><pre>parser.add_argument(<br>        &#39;--scale-factor&#39;, type=float, default=1.0,<br>        help=&#39;choose scale factor for images&#39;<br>    )</pre><pre>return parser.parse_args()</pre><p>For understanding how the extracting function works, let’s take a closer look at the section of the CVAT XML file:</p><pre>&lt;image id=&quot;7&quot; name=&quot;7.jpg&quot; width=&quot;4800&quot; height=&quot;2831&quot;&gt;<br>    &lt;polygon label=&quot;roofs&quot; occluded=&quot;0&quot; points=&quot;2388.11,2069.80;2313.80,2089.10;2297.46,2080.21;2285.57,2043.80;2339.07,2031.17;2336.10,2018.54;2428.23,2060.89&quot;&gt;<br>    &lt;/polygon&gt;<br>    &lt;polygon label=&quot;roofs&quot; occluded=&quot;0&quot; points=&quot;1431.35,1161.11;1353.11,1179.63;1366.25,1229.79;1398.80,1219.94;1396.11,1210.08;1437.91,1194.26&quot;&gt;<br>    &lt;/polygon&gt;<br>    &lt;polygon label=&quot;roofs&quot; occluded=&quot;0&quot; points=&quot;1344.81,1673.28;1270.10,1619.40;1213.00,1697.00&quot;&gt;<br>    &lt;/polygon&gt;<br>    &lt;polygon label=&quot;roofs&quot; occluded=&quot;0&quot; points=&quot;1498.35,939.31;1573.30,923.19;1586.74,985.00;1509.10,1002.32&quot;&gt;<br>    &lt;/polygon&gt;<br>...</pre><p>At first, it is necessary to find in the XML file the area corresponding to the currently processed image. The easiest way to do this is by the file name (‘<em>7.jpg</em>’ in the example). Next, you need to find the tags ‘<em>polygon</em>’ or ‘<em>box</em>’ and extract the necessary data from them (in this example, roofs are marked on the basis of polygons). You can use the following function to obtain markup results from CVAT XML:</p><pre>def parse_anno_file(cvat_xml, image_name):<br>    root = etree.parse(cvat_xml).getroot()<br>    anno = []</pre><pre>image_name_attr = &quot;.//image[<a href="http://twitter.com/name">@name</a>=&#39;{}&#39;]&quot;.format(image_name)</pre><pre>for image_tag in root.iterfind(image_name_attr):<br>        image = {}<br>        for key, value in image_tag.items():<br>            image[key] = value<br>        image[&#39;shapes&#39;] = []<br>        for poly_tag in image_tag.iter(&#39;polygon&#39;):<br>            polygon = {&#39;type&#39;: &#39;polygon&#39;}<br>            for key, value in poly_tag.items():<br>                polygon[key] = value<br>            image[&#39;shapes&#39;].append(polygon)<br>        for box_tag in image_tag.iter(&#39;box&#39;):<br>            box = {&#39;type&#39;: &#39;box&#39;}<br>            for key, value in box_tag.items():<br>                box[key] = value<br>            box[&#39;points&#39;] = &quot;{0},{1};{2},{1};{2},{3};{0},{3}&quot;.format(<br>                box[&#39;xtl&#39;], box[&#39;ytl&#39;], box[&#39;xbr&#39;], box[&#39;ybr&#39;])<br>            image[&#39;shapes&#39;].append(box)<br>        image[&#39;shapes&#39;].sort(key=lambda x: int(x.get(&#39;z_order&#39;, 0)))<br>        anno.append(image)<br>    return anno</pre><p>Next, we need to create mask files. Draw the sides of the mask polygons in white, and the inner content in red (as shown in the picture above). The following function allows you to do this:</p><pre>def create_mask_file(width, height, bitness, background, shapes, scale_factor):<br>    mask = np.full((height, width, bitness // 8), background, dtype=np.uint8)<br>    for shape in shapes:<br>        points = [tuple(map(float, p.split(&#39;,&#39;))) for p in shape[&#39;points&#39;].split(&#39;;&#39;)]<br>        points = np.array([(int(p[0]), int(p[1])) for p in points])<br>        points = points*scale_factor<br>        points = points.astype(int)<br>        mask = cv2.drawContours(mask, [points], -1, color=(255, 255, 255), thickness=5)<br>        mask = cv2.fillPoly(mask, [points], color=(0, 0, 255))<br>    return mask</pre><p>And in the end, the main function:</p><pre>def main():<br>    args = parse_args()<br>    dir_create(args.output_dir)<br>    img_list = [f for f in os.listdir(args.image_dir) if os.path.isfile(os.path.join(args.image_dir, f))]<br>    mask_bitness = 24<br>    for img in tqdm(img_list, desc=&#39;Writing contours:&#39;):<br>        img_path = os.path.join(args.image_dir, img)<br>        anno = parse_anno_file(args.cvat_xml, img)<br>        background = []<br>        is_first_image = True<br>        for image in anno:<br>            if is_first_image:<br>                current_image = cv2.imread(img_path)<br>                height, width, _ = current_image.shape<br>                background = np.zeros((height, width, 3), np.uint8)<br>                is_first_image = False<br>            output_path = os.path.join(args.output_dir, img.split(&#39;.&#39;)[0] + &#39;.png&#39;)<br>            background = create_mask_file(width,<br>                                          height,<br>                                          mask_bitness,<br>                                          background,<br>                                          image[&#39;shapes&#39;],<br>                                          args.scale_factor)<br>        cv2.imwrite(output_path, background)</pre><p>When we execute file as command to the python interpreter, we must add the following construct:</p><pre>if __name__ == &quot;__main__&quot;:<br>    main()</pre><p>That’s all. To run the script, you should run the following command (scale factor is 1 by default when after markup you don’t resize images):</p><pre>python script_name.py --image-dir original_images_dir --cvat-xml cvat.xml --output-dir masks_dir --scale-factor 0.4</pre><p>An original image example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4AkFUThpV6zRTuKuNN4P-g.jpeg" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://www.google.com.ua/intl/en/earth/">Google Earth</a></figcaption></figure><p>The mask obtained as a result of the script:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QG-yYW0LEn8Cc_rN-6czgw.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://opencv.org/">OpenCV</a> library</figcaption></figure><p><strong>Conclusion</strong></p><p>The considered approach allows obtaining more complex mask files from the data contained in the CVAT XML. You can extract individual polygons or highlight polygons with different colors depending on the number of vertices. In addition, after a little revision, the considered script will allow cutting polygonal sections from the original images in accordance with the marking contour.</p><p><strong>References</strong></p><p><a href="https://github.com/opencv/cvat">Computer Vision Annotation Tool (CVAT)</a></p><p><a href="https://lxml.de/">lxml — XML and HTML with Python</a></p><p><a href="https://opencv.org/">OpenCV</a></p><p><a href="https://realpython.com/run-python-scripts/">How to Run Your Python Scripts</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb69749c4dc9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/extract-annotations-from-cvat-xml-file-into-mask-files-in-python-bb69749c4dc9">Extract annotations from CVAT XML file into mask files in Python</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Images and masks splitting into multiple pieces in Python with Google Colab]]></title>
            <link>https://medium.com/data-science/images-and-masks-splitting-into-multiple-pieces-in-python-with-google-colab-2f6b2ddcb322?source=rss-c3f778e21722------2</link>
            <guid isPermaLink="false">https://medium.com/p/2f6b2ddcb322</guid>
            <category><![CDATA[splitting]]></category>
            <category><![CDATA[masks]]></category>
            <category><![CDATA[annotation-tools]]></category>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[Oleksii Sheremet]]></dc:creator>
            <pubDate>Thu, 06 Aug 2020 01:08:10 GMT</pubDate>
            <atom:updated>2020-08-06T02:00:58.658Z</atom:updated>
            <content:encoded><![CDATA[<h4>A practical example of images and masks splitting into smaller parts</h4><p>Data labelers use special annotation tools for objects annotation. For example, the <a href="https://github.com/opencv/cvat">Computer Vision Annotation Tool (CVAT)</a> is widely known in computer vision. Naturally, it is more convenient for labelers to work with high-resolution images. This is especially true when you need to mark a large number of objects.</p><p>In one of the roof segmentation tasks that I participated in, it was necessary to highlight triangular segments, quadrangular segments, other segments and edges of the roof. An example of such markup is shown in the following figure (white color for edges, red color for triangles, green color for quadrangles, blue color for other polygons):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kam5rcIQhFLoOsXwQVe9GA.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://matplotlib.org/">matplotlib</a> module</figcaption></figure><p>The original images were obtained from Google Earth at 2048x1208 pixels. The masks were annotated by data labelers using CVAT at the same resolution. To train the model, images and masks should be in a lower resolution (from 128x128 to 512x512 pixels). It is well known that image splitting is a technique most often used to slice a large image into smaller parts. Thus, the logical solution was to split the images and their corresponding masks into the parts with the same resolution.</p><p>All code for splitting was implemented in Google Colab. Let’s take a closer look. Import libraries:</p><pre>import os<br>import sys<br>import shutil<br>import glob<br>import matplotlib.pyplot as plt<br>import matplotlib.image as mpimg<br>from PIL import Image</pre><p>Mount the Google Drive (with images and masks) to Google Colab:</p><pre>from google.colab import drive<br>drive.mount(&#39;/content/gdrive&#39;)<br>%cd &quot;gdrive/My Drive/File Folder&quot;</pre><p>A useful function for creating a new directory and recursively deleting the contents of an existing one:</p><pre>def dir_create(path):<br>    if (os.path.exists(path)) and (os.listdir(path) != []):<br>        shutil.rmtree(path)<br>        os.makedirs(path)<br>    if not os.path.exists(path):<br>        os.makedirs(path)</pre><p>The crop function that goes over the original image are adjusted to the original image limit and contain the original pixels:</p><pre>def crop(input_file, height, width):<br>    img = Image.open(input_file)<br>    img_width, img_height = img.size<br>    for i in range(img_height//height):<br>        for j in range(img_width//width):<br>            box = (j*width, i*height, (j+1)*width, (i+1)*height)<br>            yield img.crop(box)</pre><p>The function for splitting images and masks into smaller parts (the height and width of the cropping window, and the starting number are taken as input parameters):</p><pre>def split(inp_img_dir, inp_msk_dir, out_dir, height, width, <br>          start_num):<br>    image_dir = os.path.join(out_dir, &#39;images&#39;)<br>    mask_dir = os.path.join(out_dir, &#39;masks&#39;)<br>    dir_create(out_dir)<br>    dir_create(image_dir)<br>    dir_create(mask_dir)<br>    img_list = [f for f in<br>                os.listdir(inp_img_dir)<br>                if os.path.isfile(os.path.join(inp_img_dir, f))]<br>    file_num = 0<br>    for infile in img_list:<br>        infile_path = os.path.join(inp_img_dir, infile)<br>        for k, piece in enumerate(crop(infile_path,<br>                                       height, width), start_num):<br>            img = Image.new(&#39;RGB&#39;, (height, width), 255)<br>            img.paste(piece)<br>            img_path = os.path.join(image_dir, <br>                                    infile.split(&#39;.&#39;)[0]+ &#39;_&#39;<br>                                    + str(k).zfill(5) + &#39;.png&#39;)<br>            img.save(img_path)<br>        infile_path = os.path.join(inp_msk_dir,<br>                                   infile.split(&#39;.&#39;)[0] + &#39;.png&#39;)<br>        for k, piece in enumerate(crop(infile_path,<br>                                       height, width), start_num):<br>            msk = Image.new(&#39;RGB&#39;, (height, width), 255)<br>            msk.paste(piece)<br>            msk_path = os.path.join(mask_dir,<br>                                    infile.split(&#39;.&#39;)[0] + &#39;_&#39;<br>                                    + str(k).zfill(5) + &#39;.png&#39;)<br>            msk.save(msk_path)<br>        file_num += 1<br>        sys.stdout.write(&quot;\rFile %s was processed.&quot; % file_num)<br>        sys.stdout.flush()</pre><p>Let’s set the necessary variables:</p><pre>inp_img_dir = ‘./input_dir/images’<br>inp_msk_dir = ‘./input_dir/masks’<br>out_dir = ‘./output_dir’<br>height = 512<br>width = 512<br>start_num = 1</pre><p>Let’s form a list of files with original images and masks and split them:</p><pre>input_images_list = glob.glob(inp_img_dir + ‘/*.jpg’)<br>input_masks_list = glob.glob(inp_msk_dir + ‘/*.png’)<br>split(inp_img_dir, inp_msk_dir, out_dir, height, width, start_num)</pre><p>As an example, two original images and masks are shown using the following code:</p><pre>for i, (image_path, mask_path) in enumerate(zip(input_images_list,<br>                                                input_masks_list)):<br>    fig, [ax1, ax2] = plt.subplots(1, 2, figsize=(18, 9))<br>    image = mpimg.imread(image_path)<br>    mask = mpimg.imread(mask_path)<br>    ax1.set_title(‘Image ‘ + str(i+1))<br>    ax1.imshow(image)<br>    ax2.imshow(mask)<br>    ax2.set_title(‘Mask ‘ + str(i+1))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*Tg4Y2CWvl9jmb1xfE2nyEA.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://matplotlib.org/">matplotlib</a> module</figcaption></figure><p>Using the following function, you can show all parts of the splitted image (divided into 8 parts):</p><pre>def image_part_plotter(images_list, offset):<br>    fig = plt.figure(figsize=(12, 6))<br>    columns = 4<br>    rows = 2<br>    # ax enables access to manipulate each of subplots<br>    ax = []<br>    for i in range(columns*rows):<br>        # create subplot and append to ax<br>        img = mpimg.imread(images_list[i+offset])<br>        ax.append(fig.add_subplot(rows, columns, i+1))<br>        ax[-1].set_title(“image part number “ + str(i+1))<br>        plt.imshow(img)<br>    plt.show() # Render the plot</pre><p>Let’s see what we got as result of images and masks splitting. For the first image:</p><pre>image_part_plotter(output_images_list, 0)<br>image_part_plotter(output_masks_list, 0)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*VXFhi-Wte1VJoI4l40eIcg.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://matplotlib.org/">matplotlib</a> module</figcaption></figure><pre>image_part_plotter(output_images_list, 8)<br>image_part_plotter(output_masks_list, 8)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*uqc5Vz1sRE_D1UuWaInndA.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://matplotlib.org/">matplotlib</a> module</figcaption></figure><p><strong>Conclusion</strong></p><p>The images and masks splitted using the proposed approach are saved with the same file names in different directories, that is, if the file <em>‘./output_dir/images/1_00001.png’</em> is in the folder with images, then the file <em>‘./output_dir/masks/1_00001.png’</em> will correspond to it in the directory with masks. After splitting the image and mask, you can apply augmentation to each part of it (for example, change brightness, contrast, rotate or flip). To do this, just add the augmentation function.</p><p><strong>References</strong></p><p><a href="https://pillow.readthedocs.io/en/stable/">Pillow (PIL Fork)</a></p><p><a href="https://github.com/opencv/cvat">Computer Vision Annotation Tool (CVAT)</a></p><p><a href="https://colab.research.google.com/notebooks/io.ipynb">This notebook provides recipes for loading and saving data from external sources</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2f6b2ddcb322" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/images-and-masks-splitting-into-multiple-pieces-in-python-with-google-colab-2f6b2ddcb322">Images and masks splitting into multiple pieces in Python with Google Colab</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Intersection over union (IoU) calculation for evaluating an image segmentation model]]></title>
            <link>https://medium.com/data-science/intersection-over-union-iou-calculation-for-evaluating-an-image-segmentation-model-8b22e2e84686?source=rss-c3f778e21722------2</link>
            <guid isPermaLink="false">https://medium.com/p/8b22e2e84686</guid>
            <category><![CDATA[intersection-over-union]]></category>
            <category><![CDATA[ipython]]></category>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[opencv]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Oleksii Sheremet]]></dc:creator>
            <pubDate>Fri, 24 Jul 2020 22:37:42 GMT</pubDate>
            <atom:updated>2020-09-07T19:21:39.715Z</atom:updated>
            <content:encoded><![CDATA[<h4>A practical example of calculating the IoU metric that allows us to evaluate how similar a predicted bounding box is to the ground truth box</h4><p>In the simplest case, segmentation is the process of dividing a digital image into several segments. The result of instance segmentation using <a href="https://github.com/matterport/Mask_RCNN">Mask R-CNN</a> is a mask applied to the desired object and a bounding box around this object.</p><p>In a practical task that I was solving, it was necessary to determine the buildings in the Google Earth photos. This task was successfully accomplished using Mask R-CNN for instance segmentation. After training the model, it was necessary to evaluate it quality for prediction the bounding boxes around buildings. I would like to acquaint readers with the approach (with code) I have used to evaluate the quality of predicting the coordinates of bounding boxes.</p><p><a href="https://www.pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/">Intersection over union</a> (IoU) is known to be a good metric for measuring overlap between two bounding boxes or masks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/423/1*2w493Z_V6-sE_3aYa48a9w.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://www.microsoft.com/uk-ua/microsoft-365/visio/flowchart-software">Microsoft Visio</a></figcaption></figure><p>If the prediction is completely correct, IoU = 1. The lower the IoU, the worse the prediction result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/976/1*kK0G-BmCqigHrc1rXs7tYQ.jpeg" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://www.microsoft.com/uk-ua/microsoft-365/visio/flowchart-software">Microsoft Visio</a></figcaption></figure><p>All code was implemented in Google Colab. Let’s take a closer look. Import libraries and load source image:</p><pre>import cv2</pre><pre>import numpy</pre><pre>import matplotlib.pyplot as plt</pre><pre>img = cv2.imread(“./some_image.jpg”)</pre><pre>plt.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/253/1*jGjWrdgrjcNzQ6WSatpSEw.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://matplotlib.org/">matplotlib</a> module</figcaption></figure><p>Import libraries and load source image:</p><pre># An example of first bounding box</pre><pre>first_bb_points = [[250, 210], [440, 210], [440, 390], [250, 390]]</pre><pre>stencil = numpy.zeros(img.shape).astype(img.dtype)</pre><pre>contours = [numpy.array(first_bb_points)]</pre><pre>color = [255, 255, 255]</pre><pre>cv2.fillPoly(stencil, contours, color)</pre><pre>result1 = cv2.bitwise_and(img, stencil)</pre><pre>result1 = cv2.cvtColor(result1, cv2.COLOR_BGR2RGB)</pre><pre>plt.imshow(result1)</pre><p>Let’s set the coordinates of the original bounding box and fill the entire content of the photo outside the bounding box with black:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/255/1*laK_yCKgQVkBgVLVhKhTYQ.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://matplotlib.org/">matplotlib</a> module</figcaption></figure><p>The coordinates of the second bounding box are obtained as a result of prediction:</p><pre># An example of second bounding box</pre><pre>second_bb_points = [[280, 190], [438, 190], [438, 390], [280, 390]]</pre><pre>stencil = numpy.zeros(img.shape).astype(img.dtype)</pre><pre>contours = [numpy.array(second_bb_points)]</pre><pre>color = [255, 255, 255]</pre><pre>cv2.fillPoly(stencil, contours, color)</pre><pre>result2 = cv2.bitwise_and(img, stencil)</pre><pre>result2 = cv2.cvtColor(result2, cv2.COLOR_BGR2RGB)</pre><pre>plt.imshow(result2)</pre><p>Like in the previous case, fill all the content outside the bounding box with black:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/1*aBOfdndycTuE_LetWo6Ysg.png" /><figcaption>Image is created by Oleksii Sheremet with <a href="https://matplotlib.org/">matplotlib</a> module</figcaption></figure><p>By filling images with black outside of the bounding boxes, pixels outside the area of interest do not affect the IoU calculation:</p><pre># IoU calculation</pre><pre>intersection = numpy.logical_and(result1, result2)</pre><pre>union = numpy.logical_or(result1, result2)</pre><pre>iou_score = numpy.sum(intersection) / numpy.sum(union)</pre><pre>print(‘IoU is %s’ % iou_score)</pre><p>Thus, we get the following result:</p><pre>IoU is 0.7625239952938262</pre><p><strong>Conclusion</strong></p><p>The proposed approach has worked well in practice. It is possible to define overlap between image areas, not necessarily rectangular. You can calculate the coordinates of the contour around the masked object, for example, using <a href="https://opencv.org/">OpenCV</a>. Then using these points and the approach described above, you can calculate IoU between the masks.</p><p><strong>References</strong></p><ul><li><a href="https://github.com/matterport/Mask_RCNN">Mask R-CNN for Object Detection and Segmentation</a></li><li><a href="https://opencv.org/">Open Computer Vision Library</a></li><li><a href="https://www.pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/">Intersection over Union (IoU) for object detection</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8b22e2e84686" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/intersection-over-union-iou-calculation-for-evaluating-an-image-segmentation-model-8b22e2e84686">Intersection over union (IoU) calculation for evaluating an image segmentation model</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>