<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vraj Mistry on Medium]]></title>
        <description><![CDATA[Stories by Vraj Mistry on Medium]]></description>
        <link>https://medium.com/@mistryvraj3198?source=rss-1673e797be38------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*1pt65JS4IrpGmqaCzLKPWA@2x.jpeg</url>
            <title>Stories by Vraj Mistry on Medium</title>
            <link>https://medium.com/@mistryvraj3198?source=rss-1673e797be38------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 02:59:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mistryvraj3198/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Smart City Community Watch — Camera-Based Community Watch for Traffic and Illegal Dumping]]></title>
            <link>https://medium.com/@mistryvraj3198/smart-city-community-watch-camera-based-community-watch-for-traffic-and-illegal-dumping-d2642d7b04fa?source=rss-1673e797be38------2</link>
            <guid isPermaLink="false">https://medium.com/p/d2642d7b04fa</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[illegal-dumping]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[machine-learning-engineer]]></category>
            <dc:creator><![CDATA[Vraj Mistry]]></dc:creator>
            <pubDate>Mon, 09 Jan 2023 00:17:01 GMT</pubDate>
            <atom:updated>2023-01-09T00:28:05.638Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Smart City Community Watch — Camera-Based Community Watch for Traffic and Illegal Dumping</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/402/1*BTtlDQH2uIl650PDLYHFGg.png" /><figcaption>No Dumping</figcaption></figure><blockquote>I focused my Master’s project on data engineering, machine learning pipelines, and computer vision. My goal was to make sure I understood all these concepts by detecting illegal dumping. Every year, government and municipality departments spend a lot of money cleaning up illegal dumping sites. As a result, my aim is to spread awareness and punish people who commit illegal activities in the community in order to reduce waste.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*G44BI1inq_P3qhL_w15mvQ.gif" /><figcaption>My own created test video which is not used for training.</figcaption></figure><h4>Data Collection</h4><blockquote>Existing datasets like ALPR, COCO, and RoboFlow data were collected from the web.Additionally, I collected data manually using my iPhone camera for different tasks (license plate, object, and human detection).Self-collected data was collected for both day and night vision and from different angles.All the collected data was then stored in RoboFlow and Amazon Web Service(AWS) S3 bucket.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/909/1*E1xUDb8feZwvzoEzpeM7WQ.png" /><figcaption>Data preparation process</figcaption></figure><blockquote>Cropped and resized the images (640 <em>x</em> 640) to normalize the data.Image Sharpening technique to enhance the image characteristics.Rotate, Translate, Flip, color space, and horizontal shift are some of the techniques used for data augmentation to increase the number of images for training.Labeling the images using RoboFlow and LabelImg.</blockquote><h3>Machine learning Models</h3><h4>Detecting Human</h4><blockquote>Using pre-train weights from human-crowded GitHub. There is the<a href="https://github.com/deepakcrk/yolov5-crowdhuman"> link.</a> Please upvote this guy, for a great repository and clear explanation.</blockquote><h4><em>Detecting License Plate</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/169/1*BBJ-E7T2BMGCO8LDlD9jxA.png" /><figcaption><strong><em>Yolov5 + OCR</em></strong></figcaption></figure><blockquote>license plate bounding box is detected using Yolo v5 model.Tesseract OCR is used to extract text from the detection.Images are sharpened before extracting text using 2D filters with kernel.</blockquote><h4><em>Detecting Illegal Dumping Action</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*-R_Nq2vqUvYIs-dlFs8I-w.png" /><figcaption><strong><em>Yolov5 + Decision Module</em></strong></figcaption></figure><blockquote>human drops the trash and the distance between human bounding box and trash bounding box increases(in our case 75+ euclidean distance), then it is classified as illegal dumping action.</blockquote><h3>Model Evaluation</h3><blockquote>models were evaluation based on the Mean-Average Precision, precision and Recall. Models were trained on 300 epochs and I acheived 0.976 mAP on detecting trash objects with 0.882 mAP on license-plate detection and 0.967 on human detection. Over the semester I collected more data and training longers with lesser classes gave me about 23% more accurate then basline models I had earlier.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*YFiNLjKAHZ-xiIKnP4N4mg.png" /><figcaption>All-in-one graphs for the model</figcaption></figure><h3>Model Results</h3><blockquote>The GIFs below are not used in the model training data.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*-1dWDrDeq_EAJIN8ZXaTyw.gif" /><figcaption>Night video. Advantage good lighting</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*jro2tJYJLLiLC5md-SXcow.gif" /><figcaption>No-Illegal dumping Test video.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*FvgFEnDzLWuPzZEL1tfSKA.png" /><figcaption>Night test snapshots</figcaption></figure><h3>System Infrastructure</h3><blockquote>Training data is stored in RoboFlow and AWS S3 bucket. Video from surveillance could be stored on cloud depending on client’s requirement.Data processing happens on Colab but can also be done locally.Model output is a .mp4 file and text object containing timestamp, license plate information.The client can opt for any relational database like MySQL, Oracle SQL, SQL Server etc to store model output.The computer that hosts the system and the computer that hosts the database must be able to communicate with each other and with the servers in the cloud.All machines/databases involved must be connected to the internetServer-to-server communication requires the use of APIs.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*LdQpel9CZaml9Jklika-eQ.png" /><figcaption>GUI flow</figcaption></figure><blockquote>The information displayed from model output would include -</blockquote><blockquote>Resultant video with detections.ID of the perpetrator.Timestamp value of when the action occurred.Video ID if a specific video is fed in.OCR-created record of the license plate of the perpetrator (if available).</blockquote><h3>Future Work</h3><blockquote><strong>Data</strong> : Working on adding more images and videos for illegal dumping to improve the model accuracy. Moreover, if the data is collected on the dumping side where people throw things, that can help the model accuracy and more trash objects can be detected.</blockquote><blockquote><strong>Enhancement: </strong>Better quality data from infrared night camera. Connecting the UI to SQL database to store information.</blockquote><blockquote><strong>Categorization: </strong>Adding more categories for trash like couch, mattress, plastic bags etc.</blockquote><blockquote>Thanks for taking the time to read about my project on my blog. Here is a link to the <a href="https://github.com/vraj1231/Illegal-Dumping-Action-Detection">GitHub repository.</a> Interested in connecting with like-minded people with a desire to improve society.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d2642d7b04fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Driver Drowsiness Classification- Deep Learning]]></title>
            <link>https://medium.com/@mistryvraj3198/driver-drowsiness-detection-computer-vision-b6a20be3931a?source=rss-1673e797be38------2</link>
            <guid isPermaLink="false">https://medium.com/p/b6a20be3931a</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[opencv]]></category>
            <category><![CDATA[modeling]]></category>
            <dc:creator><![CDATA[Vraj Mistry]]></dc:creator>
            <pubDate>Thu, 28 Jul 2022 06:45:24 GMT</pubDate>
            <atom:updated>2022-07-28T07:06:19.668Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/511/1*7z9ziDBTZNXlv0mlL6ox2Q.png" /></figure><blockquote><em>Drowsiness detection is a car safety technology that helps prevent accidents caused by drivers who fell asleep while driving. According to NHTSA (National Highway Traffic Safety Administration), 91,000 police-reported crashes involved drowsy drivers. These crashes led to an estimated 50,000 people injured and nearly 800 deaths in 2017. Currently, methods are mainly focused on eye blink detection using deep learning or machine learning technologies, however, </em>what if drivers wear sunglasses<em>?</em></blockquote><h3>My Approach!!!</h3><blockquote><em>What if we took into consideration about driver’s head tilt, yawning, and other factors as well? Yes, that is exactly what I did. Before I move into feature extraction part, I took the data from The “ULg Multimodality Drowsiness Database” also called </em><a href="http://www.drozy.ulg.ac.be/"><em>DROZY</em></a><em>, a database containing various types of drowsiness-related data (signals, images, etc.). The dataset contains about 45 video clips which are label as per their Karolinska Sleepiness Scale (KSS). KSS scale ranges from 1 to 9, with 1 being very alert and 9 be very sleepy.</em></blockquote><blockquote><em>Since the data and labels were lacking in this dataset, I converted labels from 1–9 to 1–3, which stated no drowsiness, moderate drowsiness, and high drowsiness. I would have used video- classification process instead, but since the data is not enough, I would rather extract features first and use them as my model input. This way the model will be more accurate with less data.</em></blockquote><h3>Features Extraction</h3><blockquote><em>For this particular task, I am going to use TensorFlow-GPU 2.6 and python 3.6 with libraries open-cv, dlib, scipy preinstalled using pip. I will link my GitHub at the very end of the page so you can get full access to it. All the libraries neeed for the feature extraction</em></blockquote><pre><strong>from</strong> scipy.spatial <strong>import</strong> distance <strong>as</strong> dist<br><strong>from</strong> imutils.video <strong>import</strong> FileVideoStream<br><strong>from</strong> imutils.video <strong>import</strong> VideoStream<br><strong>from</strong> imutils <strong>import</strong> face_utils<br><strong>import</strong> numpy <strong>as</strong> np<br><strong>import</strong> argparse<br><strong>import</strong> imutils<br><strong>import</strong> time<br><strong>import</strong> dlib<br><strong>import</strong> cv2 <br><strong>import</strong> datetime<br><strong>import</strong> csv<br><strong>import</strong> os<br><strong>import</strong> math</pre><blockquote><em>Average Blink Duration: the duration for which the eye aspect ratio is below 0.3 and then goes above 0.3 is detected as a blink. The time for which a blink happens is called blink duration. Blink duration per minute is averaged out to calculate average blink duration.</em></blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3518530ec0cbd48fa5fc819ed9d58386/href">https://medium.com/media/3518530ec0cbd48fa5fc819ed9d58386/href</a></iframe><blockquote><em>Blink Frequency: number of blinks per minute is called blink frequency.</em></blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4ccc16463a5080ee524a5e08980dc3d7/href">https://medium.com/media/4ccc16463a5080ee524a5e08980dc3d7/href</a></iframe><blockquote><em>Mouth Aspect Ratio: MAR is calculated to detect whether a person is yawning or not.</em></blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a615108c936656b0879709e46bc01658/href">https://medium.com/media/a615108c936656b0879709e46bc01658/href</a></iframe><blockquote><em>Head Pose: different angles are calculated at each frame to get the head pose</em>.</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ab6f0426055ecb0dcb9ca4c5831c9bdd/href">https://medium.com/media/ab6f0426055ecb0dcb9ca4c5831c9bdd/href</a></iframe><blockquote><em>This is what my feature extraction looks like during the process.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LeTrPdWCafNnGCLh" /></figure><h3>Time to make the model!</h3><blockquote><em>Since we have already done the feature selection part, we don&#39;t have to build the complex model. I am going to use Artificial Neural Network</em></blockquote><pre><strong>import</strong> numpy <strong>as</strong> np<br><strong>import</strong> sklearn<br><strong>from</strong> sklearn <strong>import</strong> preprocessing<br><em>#from sklearn.datasets.samples_generator import make_blobs</em><br><em>#from sklearn.preprocessing import LabelEncoder, StandardScaler</em><br><strong>import</strong> csv<br><strong>import</strong> os<br><strong>from</strong> tensorflow <strong>import</strong> keras<br><strong>import</strong> random<br><strong>from</strong> keras.models <strong>import</strong> Sequential<br><strong>from</strong> keras.layers <strong>import</strong> Dense, Dropout, Activation, BatchNormalization <br><strong>from</strong> keras <strong>import</strong> regularizers<br><strong>import</strong> matplotlib.pyplot <strong>as</strong> plt<br><em>#from keras.utils import plot_model</em><br><strong>import</strong> sklearn<br><strong>from</strong> sklearn.metrics <strong>import</strong> confusion_matrix<br><strong>from</strong> sklearn.metrics <strong>import</strong> accuracy_score <br><strong>from</strong> sklearn.metrics <strong>import</strong> classification_report <br><strong>import</strong> pickle<br><strong>from</strong> keras.utils <strong>import</strong> np_utils<br><strong>from</strong> keras <strong>import</strong> optimizers<br><strong>from</strong> keras.models <strong>import</strong> load_model</pre><blockquote>Here is the design to the simplest model in deep learning and yet effective because of feature extraction. Make sure since,we are calculating drowniess per minute, you concat your input and then pass into the model.</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb45b9f109ce785ff20909e18b01cc78/href">https://medium.com/media/fb45b9f109ce785ff20909e18b01cc78/href</a></iframe><blockquote><em>Model performance using accuracy vs validation accuracy and training loss vs validation loss.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*jw1vMRqNMs_Ywqhz516Svg.png" /></figure><blockquote><em>Hyperparameter tuning the model! I change the learning rate(0.01 -&gt; 0.001), different optimizer(RMSprop), number of epochs(20 -&gt; 50).</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/997/1*vIILeaNB57F6CqgphntkwA.png" /></figure><blockquote><em>With Sklearn’s confusion matrix, I evaluated the model on the test set and got 73% accuracy and 89% training accuracy. Using about 4 more hidden layers, I got about 74% accuracy on the test set and 93% accuracy on the training dataset.</em></blockquote><blockquote><em>I hope you liked my blog post! thank you for taking the time to read and here is my Github link about this project(</em><a href="https://github.com/vraj1231/Driver-Drowsiness-Detection--Computer-Vision"><em>link</em></a><em>).</em></blockquote><blockquote><em>I am still new when it comes to posting blogs, but I would appreciate any feedback from your side in the comments below. Thank you.</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b6a20be3931a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Semantic Segmentation- Satellite Images]]></title>
            <link>https://medium.com/@mistryvraj3198/semantic-segmentation-satellite-images-95ce75144809?source=rss-1673e797be38------2</link>
            <guid isPermaLink="false">https://medium.com/p/95ce75144809</guid>
            <category><![CDATA[semantic-segmentation]]></category>
            <category><![CDATA[aerial]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[rasterio]]></category>
            <dc:creator><![CDATA[Vraj Mistry]]></dc:creator>
            <pubDate>Tue, 26 Jul 2022 04:30:53 GMT</pubDate>
            <atom:updated>2022-10-17T04:18:58.284Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Hello Everyone, this is my first blog post about the recent work I did using Rasterio to read GeoTiff files and use them for semantic Segmentation tasks. I spend 3–4 hours to figure out how to import rasterio without breaking other module and I would like to share the experience also with some insight of my project.</blockquote><blockquote>Rasterio is an open-source library for python, but installing it is a bit tricky, (not like pip install Rasterio)and requires correct configuration files and path. I am using Tensorflow-GPU 2.6 with python 3.6 and for this particular config, I had to download GDAL “<strong><em>gdal-305–1928-x64-core.msi</em></strong>” and “<strong><em>GDAL-3.5.0.win-amd64-py3.6.msi</em></strong>”. You can download this from <a href="https://www.gisinternals.com/release.php">here</a>. Also once you finish installing GDAL, go to your enivornmental path and add GDAL file path into it.</blockquote><blockquote>After you have install these independencies, go in anaconda prompt, activate your virtual environment, (I recommended doing that), and install files as listed in this <a href="https://rasterio.readthedocs.io/en/latest/installation.html#installing-from-binaries">documentation</a>. In my case, since I have python 3.6 I had to download “<a href="https://www.lfd.uci.edu/~gohlke/pythonlibs/#rasterio"><strong>rasterio-1.1.8-cp36-cp36m-win_amd64.whl</strong></a>” and “<a href="https://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal"><strong>GDAL-3.1.4-cp36-cp36m-win_amd64.whl</strong></a>”. Run these files using pip in your enviornment and you are all set to use rasterio for satellite images.</blockquote><h4>Using aerial images, we gonna classify crop classification using pixel-wise classification.</h4><pre># Required libraries to work with semantic segmentation<br>import pandas as pd <br>import numpy as np<br>import matplotlib.pyplot as plt<br>import tensorflow as tf<br>from PIL import Image<br>import os<br>import rasterio as rs<br>from rasterio.plot import show<br>from osgeo import gdal<br>from matplotlib import gridspec<br>import time<br>from patchify import patchify,unpatchify<br>from tensorflow.keras.utils import to_categorical<br>from sklearn.preprocessing import MinMaxScaler, StandardScaler<br>scaler = MinMaxScaler()<br>from sklearn.model_selection import train_test_split<br>import random<br>import segmentation_models as sm<br>from tensorflow.keras.metrics import MeanIoU</pre><blockquote>Visualizing the image using Rasterio software!</blockquote><pre>#looking at the data for first time..<br>file_loc =  os.getcwd() + &quot;\XYZ.tif&quot;<br>geo_img = rs.open(file_loc)<br>show(img)<br>print(&quot;Count: &quot;,geo_img.count)<br>print(&quot;Height and widht: &quot;, geo_img.height ,geo_img.width)<br>print(&quot;CRS&quot;, geo_img.crs)<br>print(&quot;bounds: &quot;,geo_img.bounds)</pre><figure><img alt="I have kept the image size small since I do not want cover whole post with the image" src="https://cdn-images-1.medium.com/max/830/1*DdlQmVwv9DalvQDs1wkZRw.png" /><figcaption>it is a huge image to process</figcaption></figure><pre>#visualizing all the spectral bands individually<br>display_name = [&quot;Blue&quot;, &quot;green&quot;, &quot;red&quot;, &quot;red edge&quot;, &quot;near infrared&quot;]<br>fig = plt.figure()<br># to change size of subplot&#39;s<br>fig.set_figheight(20)<br>fig.set_figwidth(20)</pre><pre>spec = gridspec.GridSpec(ncols=5, nrows=1,wspace=0.5,<br>                         hspace=0.5)<br>n_cols = 5<br>for index in range(n_cols):<br>    ax = fig.add_subplot(spec[index])<br>    ax.title.set_text(display_name[index])<br>    ax.imshow(geo_img.read(index+1))<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HmH13lDZ9onXAGRzgDa5fw.png" /><figcaption>Different visualization of the spectral bands</figcaption></figure><pre>#visualizing the distribution of bands in image<br>plt.figure(figsize=(10,10))<br>rs.plot.show_hist(geo_img)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*xfcuvEFQy4C7JSbLiAjeuw.png" /></figure><blockquote>To read the GeoTIFF image into an array, we can just use Rasterio. read and save it into the NumPy array.</blockquote><pre>### Features for the data<br>with rs.open(file_loc, &#39;r&#39;) as file:<br>    arr_st = file.read()<br># Data<br># arr_st = np.stack(features)<br>features = np.moveaxis(arr_st, 0, -1)</pre><pre>print(features.shape) #(11652, 9210, 5)</pre><blockquote>Oh no! The image shape is too large to process at once. However, we can divide the image into smaller parts, and then reconstruct it again. To do this process, I am going to use patchify an open-source library (pip install patchify) to divide the aerial images into small images and use them as input for our model. Do the same thing for your label image.</blockquote><pre># making an empty list<br>image_dataset = []</pre><pre>#I am going to divide my image into 256*256<br>#make sure your image shape is divisible by the patch size or reshape your image accordingly</pre><pre>patch_size = 256<br>patches_img = patchify(features, (patch_size, patch_size, 3), step=patch_size)  #Step=256 for 256 patches means no overlap<br>        <br>for i in range(patches_img.shape[0]):<br>    for j in range(patches_img.shape[1]):</pre><pre>single_patch_img = patches_img[i,j,:,:]</pre><pre>#Use minmaxscaler instead of just dividing by 255. <br>        single_patch_img = scaler.fit_transform(single_patch_img.reshape(-1, single_patch_img.shape[-1])).reshape(single_patch_img.shape)</pre><pre>#single_patch_img = (single_patch_img.astype(&#39;float32&#39;)) / 255. <br>        single_patch_img = single_patch_img[0] #Drop the extra unecessary dimension that patchify adds.                               <br>        image_dataset.append(single_patch_img)</pre><blockquote>To make sure it was correctly done, I am going to visualize it randomly and see how it looks.</blockquote><pre>#visualizing the image to see if they are patch with correct label (no neccessary but to be sure)<br>image_number = random.randint(0, len(image_dataset))<br>plt.figure(figsize=(12, 6))<br>plt.subplot(121)<br>plt.imshow(np.reshape(image_dataset[image_number], (patch_size, patch_size, 3)))<br>plt.subplot(122)<br>plt.imshow(np.reshape(mask_dataset[image_number], (patch_size, patch_size, 3)))<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/951/1*ELsNf86vkR_r9dr4f4R4EA.png" /><figcaption>random sample and its corresponding label</figcaption></figure><blockquote>Hexadecimal values are typically used for labels when dealing with pixel-wise classification problems. To fit the label into the model, you must first convert this hex decimal to integer and then one-hot encode it. Python can be used for this, for example:</blockquote><pre>#hexa -&gt; decimal<br>rice = &#39;#ffd300&#39;.lstrip(&#39;#&#39;)<br>rice= np.array(tuple(int(rice[i:i+2], 16) for i in (0, 2, 4)))</pre><pre>Wheat = &#39;#267000&#39;.lstrip(&#39;#&#39;)<br>wheat = np.array(tuple(int(wheat[i:i+2], 16) for i in (0, 2, 4)))</pre><blockquote>Using this we can use np. where() function from NumPy and replace all the pixels with the decimal labels.</blockquote><pre>#use this function to all mask_dataset values <br>def rgb_to_2D_label(label):</pre><pre>label_seg = np.zeros(label.shape,dtype=np.uint8)<br>    label_seg [np.all(label == wheat,axis=-1)] = 1<br>    label_seg [np.all(label==rice,axis=-1)] = 2<br>    <br>    label_seg = label_seg[:,:,0]  #Just take the first channel, no need for all 3 channels<br>    <br>    return label_seg</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/1*QFRuxECO_pEI_NCbYhMmVQ.png" /><figcaption>Using decimal values as labels instead of pixels</figcaption></figure><blockquote>Finally one-hot encoding on these pixels</blockquote><pre>n_classes = len(np.unique(labels))<br>print(n_classes)<br>labels_cat = to_categorical(labels, num_classes=n_classes)</pre><pre>#spliting training test using sklearn library<br>X_train, X_test, y_train, y_test = train_test_split(image_dataset, labels_cat, test_size = 0.20, random_state = 42)</pre><pre>print(X_train.shape)<br>print(y_train.shape)</pre><pre>print(X_test.shape)<br>print(y_test.shape)</pre><blockquote>For modeling, I am going to use ResNet34 as the backbone, with the UNet model and ImageNet weights. Using Jaccard coefficient as metrics with accuracy.</blockquote><pre><br>sm.set_framework(&#39;tf.keras&#39;)</pre><pre>sm.framework()</pre><pre>BACKBONE = &#39;resnet34&#39;<br>preprocess_input = sm.get_preprocessing(BACKBONE)</pre><pre># preprocess input<br>X_train_prepr = preprocess_input(X_train)<br>X_test_prepr = preprocess_input(X_test)</pre><pre># define model<br>model_resnet_backbone = sm.Unet(BACKBONE, encoder_weights=&#39;imagenet&#39;, classes=n_classes, activation=&#39;softmax&#39;)</pre><pre>from keras import backend as K<br>def jacard_coef(y_true, y_pred):<br>    y_true_f = K.flatten(y_true)<br>    y_pred_f = K.flatten(y_pred)<br>    intersection = K.sum(y_true_f * y_pred_f)<br>    return (intersection + 1.0) / (K.sum(y_true_f) + K.sum(y_pred_f) - intersection + 1.0)</pre><pre>metrics=[&#39;accuracy&#39;, jacard_coef]</pre><pre>optimizer = tf.keras.optimizers.Adam(learning_rate= 0.0001)</pre><pre>model_resnet_backbone.compile(optimizer=optimizer, loss=&#39;categorical_crossentropy&#39;, metrics= metrics)</pre><pre>#print(model_resnet_backbone.summary())</pre><blockquote>Fitting the model. Got about 90 % accuracy and 0.28 loss.</blockquote><pre>history =model_resnet_backbone.fit(X_train_prepr, <br>          y_train,<br>          batch_size=4, <br>          epochs=10,<br>          verbose=1,<br>          validation_data=(X_test_prepr, y_test))</pre><pre>Epoch 1/10<br>315/315 [==============================] - 38s 84ms/step - loss: 0.7561 - accuracy: 0.6814 - jacard_coef: 0.3897 - val_loss: 1.4274 - val_accuracy: 0.3684 - val_jacard_coef: 0.2081<br>Epoch 2/10<br>315/315 [==============================] - 25s 79ms/step - loss: 0.5269 - accuracy: 0.7969 - jacard_coef: 0.5302 - val_loss: 1.1957 - val_accuracy: 0.3343 - val_jacard_coef: 0.1997<br>Epoch 3/10<br>315/315 [==============================] - 25s 80ms/step - loss: 0.4562 - accuracy: 0.8299 - jacard_coef: 0.5842 - val_loss: 1.2462 - val_accuracy: 0.3022 - val_jacard_coef: 0.2064<br>Epoch 4/10<br>315/315 [==============================] - 25s 80ms/step - loss: 0.4033 - accuracy: 0.8503 - jacard_coef: 0.6228 - val_loss: 0.7891 - val_accuracy: 0.7008 - val_jacard_coef: 0.4709<br>Epoch 5/10<br>315/315 [==============================] - 25s 80ms/step - loss: 0.3544 - accuracy: 0.8689 - jacard_coef: 0.6619 - val_loss: 0.3950 - val_accuracy: 0.8537 - val_jacard_coef: 0.6244<br>Epoch 6/10<br>315/315 [==============================] - 25s 81ms/step - loss: 0.3150 - accuracy: 0.8840 - jacard_coef: 0.6952 - val_loss: 0.3725 - val_accuracy: 0.8578 - val_jacard_coef: 0.6704<br>Epoch 7/10<br>315/315 [==============================] - 25s 81ms/step - loss: 0.2941 - accuracy: 0.8905 - jacard_coef: 0.7159 - val_loss: 0.4112 - val_accuracy: 0.8472 - val_jacard_coef: 0.6698<br>Epoch 8/10<br>315/315 [==============================] - 25s 81ms/step - loss: 0.2797 - accuracy: 0.8963 - jacard_coef: 0.7282 - val_loss: 0.3748 - val_accuracy: 0.8582 - val_jacard_coef: 0.6806<br>Epoch 9/10<br>315/315 [==============================] - 25s 81ms/step - loss: 0.2784 - accuracy: 0.8958 - jacard_coef: 0.7318 - val_loss: 0.4816 - val_accuracy: 0.7981 - val_jacard_coef: 0.5954<br>Epoch 10/10<br>315/315 [==============================] - 25s 81ms/step - loss: 0.2849 - accuracy: 0.8944 - jacard_coef: 0.7293 - val_loss: 0.4007 - val_accuracy: 0.8561 - val_jacard_coef: 0.6879</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*78YjHcE8DydtupzRAT0A2g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*T-zlWaVChsJwyxPikQRKnQ.png" /></figure><blockquote>Based on the Dice coefficient, Jaccard coefficient, and Intersection over Union, the model was evaluated as 0.81, 0.68, and 0.73 respectively.</blockquote><blockquote>Quite good. My first model was vanilla UNet, then custom UNET, and then Resnet34 — UNET without hyperparameters. Following is an example of a predicted image based on my model. There was some accuracy, but not 100%, so it was a good enough tool to use. It is likely that if I had more computing power, I would have experimented with more epoch, more layers, larger size, and soon.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/918/1*B6kfx5dHKzKh-7LFOwSHHw.png" /></figure><blockquote>Thanks for your time, I hope you like it, I am a newbie, so if you have any feedback, let me know in the comment section below. Also here is my github link for this task: <a href="https://github.com/vraj1231/Semantic-Segmentation">https://github.com/vraj1231/Semantic-Segmentation</a>.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=95ce75144809" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>