<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Siddhant Baldota on Medium]]></title>
        <description><![CDATA[Stories by Siddhant Baldota on Medium]]></description>
        <link>https://medium.com/@sidatwork99?source=rss-6a0cec244490------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*aMkx3NSlRr4_KH70hAaa3A.jpeg</url>
            <title>Stories by Siddhant Baldota on Medium</title>
            <link>https://medium.com/@sidatwork99?source=rss-6a0cec244490------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 11:51:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sidatwork99/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How YOLOv5 solved an ambiguity encountered by YOLOv3]]></title>
            <link>https://medium.com/data-science/indian-car-license-plate-detection-using-yolo-v5-ae2574578175?source=rss-6a0cec244490------2</link>
            <guid isPermaLink="false">https://medium.com/p/ae2574578175</guid>
            <category><![CDATA[ds-in-the-real-world]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[yolov5]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Siddhant Baldota]]></dc:creator>
            <pubDate>Sun, 21 Jun 2020 00:36:21 GMT</pubDate>
            <atom:updated>2022-03-29T06:30:44.217Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/data-science-in-the-real-world/home?gi=ea0e55878a54">Data science in the real world</a></h4><h4>Robust Indian license plate detection using YOLOv5</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*-1D-CabftTbQpm01cp3Dpg.gif" /><figcaption>my results on general traffic data</figcaption></figure><p>To the ones who not might be knowing, a new version of YOLO (You Only Look Once) is here, namely YOLO v5. Many thanks to <a href="https://www.ultralytics.com/">Ultralytics</a> for putting this repository together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*4BQ2ZXPPHuGPrmExq5FGsg.png" /><figcaption>source: ‘<a href="https://github.com/ultralytics/yolov5">https://github.com/ultralytics/yolov5</a>’</figcaption></figure><h4>The problem with YOLOv3</h4><p>You only look once (YOLO) is a state-of-the-art, real-time object detection system. On a Pascal Titan X it processes images at 30 FPS and has a mAP of 57.9% on COCO test-dev.</p><p>YOLOv3 is extremely fast and accurate. In mAP measured at .5 IOU YOLOv3 is on par with Focal Loss but about 4x faster. Moreover, you can easily tradeoff between speed and accuracy simply by changing the size of the model, no retraining required!</p><p>YOLOv3 has a problem though, despite being so accurate, it is biased. The bias is towards the size of the object in the image. If encountered with bigger objects while training, it cannot detect the same object of a smaller scale perfectly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/953/1*p0xn8H0tYRtSG6sKVUcpyw.png" /><figcaption>since the model was trained for larger license plates, it couldn’t detect the small one properly in this very small image</figcaption></figure><p>YOLOv3 detects features in images and learn how to recognize objects with this information. Layers near the start detecting really simple features like edges and layers that are deeper can detect more complex features like eyes, noses, or an entire face. It then uses all of these features which it has learned, to make a final prediction. Herein lies the flaws of this system — there is no spatial that is used anywhere in a CNN and the pooling function that is used to connect layers is rendered inefficient.</p><p><strong>YOLOv5</strong></p><p>YOLOv5 consists of three parts :</p><ol><li>Model Backbone</li><li>Model head</li><li>Model neck</li></ol><p>The model head helps in the process of feature extraction. The model backbone consists of CSP Nets which help extract essential features from tensor passed through the model head. CSPNets enable quicker inference as well. The tensor from the model backbone is passed through the model neck which has feature pyramids, which try to eliminate bias with respect to size of objects.</p><p>To know more about feature pyramids, check this out</p><p><a href="https://arxiv.org/abs/1612.03144">Feature Pyramid Networks for Object Detection</a></p><p>To know more about YOLOv5, check this out.</p><p><a href="https://blog.roboflow.ai/yolov5-is-here/">YOLOv5 is Here</a></p><p>Check out the YOLO v5 repository at</p><p><a href="https://github.com/ultralytics/yolov5">GitHub - ultralytics/yolov5: YOLOv5 🚀 in PyTorch &gt; ONNX &gt; CoreML &gt; TFLite</a></p><p>For our use case, I have created a similar repository at</p><p><a href="https://github.com/sid0312/anpr_yolov5">sid0312/anpr_yolov5</a></p><h4>Clone the repository</h4><pre>git clone <a href="https://github.com/sid0312/anpr_yolov5">https://github.com/sid0312/anpr_yolov5</a><br>cd anpr_yolov5</pre><h4>Detect License Plates</h4><p>To detect a license plate in the car image, save it in the sample_cars folder of the directory. Name it as example.jpg</p><pre>python detect.py --source sample_cars/example.jpg  --weights weights/best.pt --conf 0.4</pre><h4>Results</h4><p>Get the final image in the inference/output directory . To get an intuition of the detection process, check out the following notebook</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e3208f7fe5b2e72d28a73bef75f26b4e/href">https://medium.com/media/e3208f7fe5b2e72d28a73bef75f26b4e/href</a></iframe><p><strong>Here are some results</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*DZBtVdOu0NBXQBMAb6wnMw.jpeg" /><figcaption>a batch prediction on validation images</figcaption></figure><h4>We notice that the license plate is detected correctly with YOLOv5 which wasn’t done at all by YOLOv3</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/193/1*grFzjUOUQzrsubneD3N2mQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/957/1*FB7z1tHjjiEVaA3OGDGdgQ.jpeg" /><figcaption>predictions on test sample image</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/1*mwK36lxXwkqF-ETLza0rmw.jpeg" /><figcaption>prediction on test sample image</figcaption></figure><h3>How did I achieve this?</h3><p>This repository included the following changes to the original utlralytics repository</p><p><strong>Created data/license_plate.yaml</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f23224bbd46b834fbfe3360f10fb8b25/href">https://medium.com/media/f23224bbd46b834fbfe3360f10fb8b25/href</a></iframe><p><strong>Data Preparation</strong></p><ul><li>Step 1- Download the starter dataset in JSON format from</li></ul><p><a href="https://www.kaggle.com/dataturks/vehicle-number-plate-detection">Vehicle Number Plate Detection</a></p><p>This serves as our starting dataset. Upload the JSON file to your Google Drive</p><ul><li>Step 2- Conversion of the JSON starter dataset to YOLO format</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9a4f80a9c1a5ee63b3e638f1da6bdb1a/href">https://medium.com/media/9a4f80a9c1a5ee63b3e638f1da6bdb1a/href</a></iframe><p>The following repository is an end to end detection and recognition of Indian license plates using YOLO v3 Darknet and Pytesseract. It serves as a precursor for our project. This is a fun individual project on its own!</p><p><a href="https://github.com/sid0312/ANPR">sid0312/ANPR</a></p><ul><li>Creating folder structure</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/1*GcONnXpu6BNYwcnFqszE9w.png" /><figcaption>the data folder of the repository</figcaption></figure><ul><li>In the data/images folder ,create two folder namely, train and valid</li><li>The train folder consists of 201 images obtained from Step 2. These are to be used for training</li><li>The valid folder consists of 36 images obtained from Step 2, which are to be used for validation</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7CKeHRjODPHHWSlk_nQhfg.png" /><figcaption>the contents of my data/images/train folder</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZP6R0WIfGEe8kufWX88a-A.png" /><figcaption>the contents of my data/images/train/valid folder</figcaption></figure><p>Only images won’t do for detection as we need labels for bounding boxes and classes for each image.</p><p>The YOLO format for a single class bounding box label is as follows:</p><pre>class_number x y width height</pre><p>The labels are already obtained from Step 2.</p><p>The labels folder consists of a train and valid folder consisting of labels for training and validation images respectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/884/1*PoKX1CsiBGZE2BB3y4ticA.png" /><figcaption>the labels for the training images</figcaption></figure><p>A sample label</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*CNUnf-Lu1qWeiQFpqD0ITQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*XLU2d4MZ32ap8Wlq892fYQ.png" /><figcaption>the labels for validation images</figcaption></figure><p>Next, we create the data/train.txt file</p><p>To get the contents of this file, simply write the following in command prompt or terminal. Make sure you are in the anpr_yolov5 directory</p><pre>D:&gt;git clone <a href="https://github.com/sid0312/anpr_yolov5">https://github.com/sid0312/anpr_yolov5</a><br>cd anpr_yolov5<br>python<br>&gt;&gt;import os<br>&gt;&gt;for image in os.list(&#39;data/images/train&#39;):<br>     path = &#39;D:/anpr_yolov5/data/images/train&#39;<br>     print(path+&#39;/&#39;+image)</pre><p>You will get the following output</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/475/1*qzTNaIjn5imzOycVjr067g.png" /></figure><p>Copy the output to a file data/train.txt</p><p>Do the same for valid.txt</p><p>If you intend to train Yolo v5 on Google Colab, commit the repository in its present state without copying the contents above. Commit and push the current state of your repository to GitHub.</p><pre>git add . <br>git commit -m&quot;some msg&quot;<br>git push</pre><p>Now follow the following notebook and copy the contents of the print logs to the files data/train.txt and data/val.txt respectively</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4867453f3baed974bf7e002c66f2999f/href">https://medium.com/media/4867453f3baed974bf7e002c66f2999f/href</a></iframe><ul><li><strong>Change the nc parameter in yolov5s.yaml to 1</strong></li><li><strong>Repeat the git add,commit,push process again</strong></li><li><strong>Check out the following notebook for the training process</strong></li></ul><p><a href="https://github.com/sid0312/anpr_yolov5/blob/master/yolov5_license_plate_train.ipynb">sid0312/anpr_yolov5</a></p><p><strong>Note: You can use yolo5s.pt weights too. I have used randomly initialized weights and trained the model for 100 epochs.</strong></p><h4>Some of my training logs are as follows</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b37FLoaM44BBlamTBhZ95Q.png" /></figure><h4>We achieve awesome performance with mAP@0.5 as 0.978 or 97.8% which is pretty awesome!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0s3xShFcBbd0eDlS8YDmRw.png" /></figure><p><strong>After running for 100 epochs, we get</strong></p><p><strong>Precision as 0.659</strong></p><p><strong>Recall as 0.972</strong></p><p><strong>map@0.5 as 0.978</strong></p><p><strong>map@0.5 with 95% confidence as 0.471</strong></p><h3><strong>Resources Used</strong></h3><ul><li><a href="https://blog.roboflow.ai/">https://blog.roboflow.ai</a></li><li><a href="https://github.com/ultralyitcs/yolov5">https://github.com/ultralyitcs<strong>/</strong>yolov5</a></li></ul><h3>Related Repositories</h3><p><strong>In case, you have skipped the training part and jumped right to the conclusion, I have something more for you. After finding the region of interest of the license plate, we can use it for character recognition using pytesseract</strong></p><p><strong>I have done it for a previous project. You can check it out at:</strong></p><p><a href="https://github.com/sid0312/ANPR">sid0312/ANPR</a></p><h4>Author’s Recommended Blogs</h4><p>If you’re into reading more about Object Detection Algorithms and Libraries, I highly recommend this blog by <strong>neptune.ai</strong></p><p><a href="https://neptune.ai/blog/object-detection-algorithms-and-libraries">Object Detection Algorithms and Libraries - neptune.ai</a></p><h4>Author’s Note</h4><p>If you’re into my articles, check these out!</p><ul><li><a href="https://towardsdatascience.com/face-detection-on-the-browser-with-tensorflow-js-27846a5fe954">Optimizing Face Detection on your browser with Tensorflow.js</a></li><li><a href="https://towardsdatascience.com/segmentation-of-optical-coherence-tomography-images-with-diabetic-macular-edema-the-gluon-82093cfd8e24">OCT Image Segmentation using U-Nets</a></li><li><a href="https://towardsdatascience.com/counting-people-on-your-webcam-using-gluoncv-and-mxnet-d1a9f05c427d">Counting people on your webcam using GluonCV and MxNet</a></li></ul><p><strong>If you wish to connect with me on Linkedin</strong>, <strong>here’s my profile</strong></p><p><a href="https://www.linkedin.com/in/siddhant-baldota-051059180/">Siddhant Baldota - Research Intern - San Diego Supercomputer Center | LinkedIn</a></p><p>Happy Computer Vision and Happy Deep Learning ❤.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae2574578175" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/indian-car-license-plate-detection-using-yolo-v5-ae2574578175">How YOLOv5 solved an ambiguity encountered by YOLOv3</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimizing Face Detection on your browser with Tensorflow.js]]></title>
            <link>https://medium.com/data-science/face-detection-on-the-browser-with-tensorflow-js-27846a5fe954?source=rss-6a0cec244490------2</link>
            <guid isPermaLink="false">https://medium.com/p/27846a5fe954</guid>
            <category><![CDATA[ds-in-the-real-world]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[facedetection]]></category>
            <category><![CDATA[tensorflowjs]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Siddhant Baldota]]></dc:creator>
            <pubDate>Wed, 17 Jun 2020 23:08:57 GMT</pubDate>
            <atom:updated>2020-06-21T20:49:27.681Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/data-science-in-the-real-world/home?gi=f64157fe3425">Data Science in the Real World</a></h4><h4>A browser based face detector is just what you need</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/1*9qm6F3_Dlraf9Woe0OOkPw.gif" /><figcaption>My robust browser based face detection results</figcaption></figure><blockquote><em>Let’s jump on to the most important question.</em></blockquote><h4>Why this detector?</h4><p>You might have come across a lot of face detection tutorials and projects. This one is different though. It is deployed on the browser and can be used anywhere from proctoring online exams to many different arcade games to using it for detecting face masks, blurring or improving the resolution of the face in real-time (since the region of interest is obtained) while also performing at quite high speeds. Machine learning on the web is the requirement of this age and shouldn’t be restricted to local systems. An alternative approach is face-api.js which uses the Multi-Task CNN but our detector has higher accuracy. Many other projects would be based on face detection models deployed on Flask Apps which are quite slow in comparison.</p><p>Tensorflow serving makes use of gRPC and Protobuf while Flask apps use REST and JSON. JSON relies on HTTP 1.1 while gRPC uses HTTP/2. HTTP 1.1 suffers from latency issues. A TCP handshake is needed for each individual request, and a large number of requests takes a significant toll on the time needed to load a page. HTTP 1.1 also suffers from Head of Line Blocking which restricts the number of connections to the same domain. With HTTP pipelining, you can send a request while waiting for the response to a previous request, effectively creating a queue. But that introduces other problems. If your request gets stuck behind a slow request then your response time will suffer. HTTP /2 maintains the basic premises and paradigms of HTTP but does away with optional parts of HTTP 1.1. REST supports only the request-response model available in HTTP 1.x. But gRPC takes full advantage of the capabilities of HTTP/2 and lets you stream information constantly. Protobuf is a binary format used to serialize data and is more efficient than JSON. Tensorflow serving can batch requests to the same model, which uses hardware (GPUs) more appropriately. Tensorflow serving has performance equal to code written in C/C++.</p><p>Moreover, Flask apps are written in Python whereas Tensorflow.js has Node.js or Chrome V8 engine as its server side node. In this project, we use Chrome’s V8 Javascript engine which is an open source high-performance JavaScript engine.</p><p>To get more intuition on the Chrome V8 engine, read this awesome blog at</p><p><a href="https://www.freecodecamp.org/news/understanding-the-core-of-nodejs-the-powerful-chrome-v8-engine-79e7eb8af964/">Understanding How the Chrome V8 Engine Translates JavaScript into Machine Code</a></p><p>Tensorflow.js provides the right kind of hardware acceleration needed using the Web GL layer, which might be considered as a browser-side GPU. WebGL is a JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser without the use of plug-ins. WebGL is fully integrated with other web standards, allowing GPU-accelerated usage of physics and image processing and effects as part of the web page canvas.</p><p>Thus this client side serving face-detector proves to be quicker than the erstwhile flask face detection apps.</p><h4><strong>Tensorflow.js</strong></h4><p>Tensorflow.js is a machine learning library in Javascript. It is an open-source hardware-accelerated JavaScript library for training and deploying machine learning models.</p><p>It can be used to develop ML in the browser by using flexible and intuitive APIs to build models from scratch using the low-level JavaScript linear algebra library or the high-level layers API. It can also be used to develop ML in Node.js by running native TensorFlow with the same TensorFlow.js API under the Node.js runtime. Pretrained Tensorflow or Keras models can be used in the browser by the TensorFlow.js model converters. Tensorflow.js can also retrain pre-existing model using sensor data-connected to the browser.</p><p>To know more about Tensorflow.js , you can check out the official documenation.</p><p><a href="https://www.tensorflow.org/js">TensorFlow.js | Machine Learning for JavaScript Developers</a></p><p>To get pretrained models, you can clone the following official tfjs GitHub repository</p><p><a href="https://github.com/tensorflow/tfjs-models">tensorflow/tfjs-models</a></p><h3>Prerequisites</h3><ul><li><strong>A text editor (optional) (Sublime, Brackets, Visual Studio Code, etc.)</strong></li></ul><p>You can download the<strong> Brackets </strong>editor from</p><p><a href="http://brackets.io/">A modern, open source code editor that understands web design</a></p><ul><li><strong>Google Chrome</strong></li></ul><p><a href="https://www.google.com/chrome/">Google Chrome - The Fast &amp; Secure Web Browser Built to be Yours</a></p><h3>Implementation</h3><p>To use the browser based face detector, check out my GitHub repository</p><p><a href="https://github.com/sid0312/tfjs-face_detection">sid0312/tfjs-face_detection</a></p><h3>Steps for replication</h3><blockquote>If you want to get hands on and do things on your own, here’s a guide</blockquote><ul><li><strong>Create a starter HTML file</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/84b9c4b4d7d1672b32939f9ddaa12b62/href">https://medium.com/media/84b9c4b4d7d1672b32939f9ddaa12b62/href</a></iframe><ul><li><strong>Add the following lines to the html file tensorflow.js headers to import the tfjs model in the head tag</strong></li></ul><pre>&lt;script src=”https://cdn.jsdelivr.net/npm/@tensorflow/tfjs&quot;&gt;&lt;/script&gt;</pre><pre>&lt;script src=”https://cdn.jsdelivr.net/npm/@tensorflow-models/blazeface&quot;&gt;<br>&lt;/script&gt;</pre><ul><li><strong>Styling (optional): Create the div(s) for formatting and add the style-sheet link. Add the video element and the canvas</strong></li><li><strong>Add the JavaScript file link in which the model is loaded</strong></li><li><strong>Rename your file to index.html</strong></li></ul><p>Your index.html file should look like this</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e96ea469bf034b2c1d9393e8a690b532/href">https://medium.com/media/e96ea469bf034b2c1d9393e8a690b532/href</a></iframe><ul><li><strong>Create a main.css file for some optional styling. Add the following code to it. Change as per convenience</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/282bdd9f3032bec335b45b8ac733fc54/href">https://medium.com/media/282bdd9f3032bec335b45b8ac733fc54/href</a></iframe><p>You can get some lovely CSS at</p><p><a href="https://codepen.io/">CodePen</a></p><p><em>Note: The main.css load a GIF which is for intial loading and is included in the repository. Feel free to use your own GIFs.</em></p><ul><li><strong>Create a blaze_pred.js file. Add the following code to it.</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0fccd7c66119f154febbec958b939fd0/href">https://medium.com/media/0fccd7c66119f154febbec958b939fd0/href</a></iframe><ul><li><strong>Explanation</strong></li></ul><p>We create a self calling function in which we obtain the canvas and video elements by their id. To get the context for the canvas, we use the getContext() method.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a0466ad3f52cd86d4405832441fdc5c4/href">https://medium.com/media/a0466ad3f52cd86d4405832441fdc5c4/href</a></iframe><p>To recieve the video feed using the navigator.getUserMedia method and adding the stream to the video source object.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f46e582f4a02ffd75cb927b3fd7e29a5/href">https://medium.com/media/f46e582f4a02ffd75cb927b3fd7e29a5/href</a></iframe><p>video.play() is used to play the video but since we have set the visibility to false in the video element of the index.html file, the video feed cannot be physically viewed. The feed which is displayed is the canvas. The face predictions are drawn using the canvas context.</p><p>To synchronize the video feed and the canvas element, we add an event listener and call a function draw which draws the predictions to it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f277de6c4f060eb322360a87ea39eeb0/href">https://medium.com/media/f277de6c4f060eb322360a87ea39eeb0/href</a></iframe><ul><li><strong>Create an asynchronous draw function with parameters as the video feed, context, the width and height of the canvas.</strong></li></ul><p>Draw the current frame of the video feed on it. Load the model using await blazeface.load(). Get the predictions on the model await model.estimateFaces() while setting the <strong>returnTensors</strong> parameter to false. Draw the bounding box on the canvas context using the predictions tensor. Obtain the confidence score using the predictions tensor and add the text to the canvas context relative to the bounding box. To call the draw function repeatedly set the timeout as 250 ms.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3ad8b46aa95e081f60ea9a2ff8816357/href">https://medium.com/media/3ad8b46aa95e081f60ea9a2ff8816357/href</a></iframe><ul><li><strong>Add the working files (index.html, main.css, blaze_pred.js) to a folder. Name the folder tfjs-face_detection. You can give any other name too.</strong></li></ul><h3><strong>Clone the repository</strong></h3><p>If you haven’t done the replication steps, type the following in your shell/terminal/command prompt</p><pre>git clone https://github.com/sid0312/tfjs-face_detection</pre><h3>Using the detector</h3><p><strong>For Linux and Mac Users,</strong></p><pre>user@username:~$ cd tfjs-face_detection<br>user@username:~/tfjs-face_detection$ google-chrome index.html</pre><p><strong>For Windows Users,</strong></p><pre>C:\Users\username&gt; cd tfjs-face_detection<br>C:\Users\username\tfjs-face_detection&gt; index.html</pre><p><strong>Allow access to the webcam</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*SPHdOJymDFlipzfABxByLw.png" /></figure><p><strong>Wait for a few seconds and you’re all set!</strong></p><h3><strong>Conclusion</strong></h3><p>We have successfully detected our face on the browser using Tensorflow.js. You can create your own deep learning face model and convert it into a tfjs model using the tfjs model converter to increase its robustness.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HJ76CaHOw70NzCcx" /><figcaption>Photo by <a href="https://unsplash.com/@ianstauffer?utm_source=medium&amp;utm_medium=referral">Ian Stauffer</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Author</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/286/1*xvib3vN3pSb1kkvShMZxnQ.gif" /><figcaption>Siddhant Baldota</figcaption></figure><p>Yeah! The one in the video feed is me! Check out my other articles:</p><ul><li><a href="https://towardsdatascience.com/segmentation-of-optical-coherence-tomography-images-with-diabetic-macular-edema-the-gluon-82093cfd8e24">OCT Image Segmentation using U-Nets</a></li><li><a href="https://towardsdatascience.com/counting-people-on-your-webcam-using-gluoncv-and-mxnet-d1a9f05c427d">Counting people on your webcam using GluonCV and MxNet</a></li></ul><blockquote>Happy Computer Vision, Happy Deep Learning and Happy Deployment. Until next time!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27846a5fe954" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/face-detection-on-the-browser-with-tensorflow-js-27846a5fe954">Optimizing Face Detection on your browser with Tensorflow.js</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Five PyTorch Tensor Operations you had absolutely no idea about]]></title>
            <link>https://medium.com/swlh/five-pytorch-tensor-operations-you-had-absolutely-no-idea-about-e25d3ae8db05?source=rss-6a0cec244490------2</link>
            <guid isPermaLink="false">https://medium.com/p/e25d3ae8db05</guid>
            <category><![CDATA[tensor]]></category>
            <category><![CDATA[deep-learning-framework]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Siddhant Baldota]]></dc:creator>
            <pubDate>Sat, 30 May 2020 20:20:41 GMT</pubDate>
            <atom:updated>2020-10-19T20:04:19.944Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*grIcZHZc1fUClygH" /><figcaption>Photo by <a href="https://unsplash.com/@frankvex?utm_source=medium&amp;utm_medium=referral">Frank Vessia</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote>Getting Started with PyTorch</blockquote><p><strong>Introduction</strong></p><p><strong><em>PyTorch</em></strong> is a relatively new and robust deep learning framework, having a dynamic computation graph and supports flexibility. It was designed primarily by Soumith Chinatala, Facebook AI Research. The new version (1.4+) supports deployment without need of Open Neural Network Exchange(ONNX), thus making it production ready. It supports scalable distributed training and performance optimization in research and production is enabled by the torch.distributed backend. It has a robust ecosystem for computer vision and natural language processing tasks among other deep learning applications. It is well supported on cloud platforms and recently Microsoft offered its cloud support to azure. Pytorch is more flexible than any other deep learning framework out there</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mB6J-4oYYBa9YHGD" /><figcaption>Photo by <a href="https://unsplash.com/@rapol?utm_source=medium&amp;utm_medium=referral">Rafael Pol</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Pytorch, as every other deep learning framework has a N-dimensional arrays for computation, which can be used on GPUs too. These N-dimensional arrays are called Tensors. Traditionally numpy arrays were not made for deep learning and won’t run on GPUs. This is the reason we use tensors which are similar to numpy arrays but are written with a CUDA and C++ backend so as to offer GPU computation. Pytorch provides a large variety of Tensor operations, similar to its contemporaries Tensorflow with Keras and MxNet with Gluon.</p><p>You might be familiar with the generic ones such as<a href="https://pytorch.org/docs/master/generated/torch.rand.html"> torch.rand()</a>, torch.tensor(), <a href="https://pytorch.org/docs/stable/torch.html#torch.abs">torch.abs()</a> , torch.ones() and much more, but there are a few top secret Tensor operations were hidden from us and we would have to dig deep down in the documentation to look at them.</p><ul><li><strong>torch.baddbmm</strong></li><li><strong>torch.bmm</strong></li><li><strong>torch.cholesky</strong></li><li><strong>torch.geqrf</strong></li><li><strong>torch.logdet</strong></li></ul><h4>Installation</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5nwOc8Zwr-2EXSHn" /><figcaption>Photo by <a href="https://unsplash.com/@owenbeard?utm_source=medium&amp;utm_medium=referral">Owen Beard</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>To install pytorch on your kernel/ notebook, go to <a href="https://pytorch.org/">https://pytorch.or</a>g .Under Quick Start Locally you’ll be provided with the installation preferences. I have installed the CUDA 10.1 version using Anaconda. Run this command once you’re in your conda virtual environment.</p><pre><strong>conda install pytorch torchvision cudatoolkit=10.1 -c pytorch</strong></pre><p>You can install locally via pip too. This downloads a wheel file and executes it</p><pre><strong>pip install torch==1.5.0+cu101 torchvision==0.6.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html</strong></pre><p><strong>Importing the torch library</strong></p><p>To import pytorch after installation, simply type the following line in your python file/shell or Jupyter notebook</p><pre><strong>import torch</strong></pre><p><strong>I hope that you have already installed Pytorch and are eager to know more about the non- generic functions. So let’s get started</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*A_NqCLMxUssiwtYm" /><figcaption>Photo by <a href="https://unsplash.com/@ikukevk?utm_source=medium&amp;utm_medium=referral">Kevin Ku</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>torch.baddbmm</h3><p><em>torch.baddbmm(input, batch1, batch2,*,beta=1,alpha=1,out=None)</em></p><p><strong>Returns:</strong></p><ul><li>a Tensor</li></ul><p><strong>Utility:</strong></p><p>Performs a matrix multiplication on tensors batch1 and batch 2, batchwise and adds the input tensor to the final product. alpha and beta act as the weights for the product and input tensor respectively</p><p><strong>Requirements:</strong></p><ul><li>batch 1 and batch 2 must be 3-D tensors, with the same number of matrices</li><li>if batch1’s shape is (b x n x m), batch2’s shape is (b x m x p) then input must be a (b x n x p) tensor, which makes the output to be the same shape as the input tensor</li></ul><p><strong>Params:</strong></p><ul><li>input — the input tensor</li><li>batch1 — the tensor of shape (b × n × m)</li><li>batch2 — the tensor of shape (b × m × p)</li><li>beta — (optional) — weight for input</li><li>alpha — (optional) — weight for the product matrix of batch1 and batch2</li><li>out — (optional) — the output tensor</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D3&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D3&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/27a5acb01b05ff1e282d27e5d5812c6f/href">https://medium.com/media/27a5acb01b05ff1e282d27e5d5812c6f/href</a></iframe><p><strong>A tensor of shape (5,20,10) is returned</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D5&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D5&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/1bea7bd3d7488edfe849f9a7292c1d15/href">https://medium.com/media/1bea7bd3d7488edfe849f9a7292c1d15/href</a></iframe><p>A tensor of shape (6,2,3) is assigned to the tensor ‘a’ which can be seen by performing a.size()</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D7&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D7&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/50dcfba55c7735b594a5d0e464aa3a39/href">https://medium.com/media/50dcfba55c7735b594a5d0e464aa3a39/href</a></iframe><p>Here the function returns an error as the first tensor is of the shape (b ×n × m) but the second tensor is not of the expected shape (b ×m ×p), thus they can’t be broadcast together</p><p>This function should be used to perform weighted batch-wise multiplication of tensors.</p><h3>torch.bmm</h3><p><em>torch.baddbmm(input, mat2,out=None)</em></p><p>This is a simplified version of torch.baddbmm</p><p><strong>Returns</strong></p><ul><li>a Tensor</li></ul><p><strong>Utility:</strong></p><ul><li>Performs a matrix multiplication on the input and mat2, batchwise</li></ul><p><strong>Requirements:</strong></p><ul><li>input and mat2 must be 3-D tensors, with the same number of matrices</li></ul><p><strong>Params:</strong></p><ul><li>input — the input tensor of shape (b ×n ×m)</li><li>mat2 — the tensor of shape (b ×m × p)</li><li>out — (optional) — the output tensor</li></ul><p>Note that this function does not broadcast</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D11&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D11&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/026df87e87a85190666b34910dc79891/href">https://medium.com/media/026df87e87a85190666b34910dc79891/href</a></iframe><p>A Tensor of shape (5,20,4) is returned</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D13&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D13&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/5320229b8c2e51de5cb290e68814a39f/href">https://medium.com/media/5320229b8c2e51de5cb290e68814a39f/href</a></iframe><p>A tensor of shape (6,2,4) is assigned to the tensor ‘a’ which can be seen by performing a.size()</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D15&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D15&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/e66dfb7e4fa571317dcfbbc7cc5f8e43/href">https://medium.com/media/e66dfb7e4fa571317dcfbbc7cc5f8e43/href</a></iframe><p>Here the function returns an error as the first tensor is of the shape (b × n ×m) but the second tensor is not of the expected shape (b ×m ×p).</p><p>To be used in Multi Layer Perceptrons</p><h3>torch.cholesky</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fanc8pJmT0tKJogw" /><figcaption>Photo by <a href="https://unsplash.com/@johnmoeses?utm_source=medium&amp;utm_medium=referral">John Moeses Bauan</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>torch.cholesky(input, upper=False, out=None)</em></p><p><strong>Returns:</strong></p><ul><li>a Tensor</li></ul><p><strong>Utility:</strong></p><ul><li>Performs the Cholesky decomposition of a symmetric positive-definitive matrix A or its batches</li></ul><p><strong>Requirements:</strong></p><ul><li>input tensor of shape (a,n,n) where a is whole number and n is a natural number ‘a’ denotes the batch size, if any, of the symmetric postive-definite matrices</li></ul><p><strong>Params:</strong></p><ul><li>input — the input tensor of shape (*,n,n)</li><li>upper — (optional) — boolean value, if set to True returns the upper triangular matrix, else returns the lower triangular matrix. The default values is False</li><li>out — (optional) — the output tensor</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D19&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D19&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/7eb892c5ab8df05d701bc6a9044ea12e/href">https://medium.com/media/7eb892c5ab8df05d701bc6a9044ea12e/href</a></iframe><p>We get a lower triangular matrix since the default value of upper is False</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D21&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D21&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/a96ab2dfd4953f4d604be544fab429a5/href">https://medium.com/media/a96ab2dfd4953f4d604be544fab429a5/href</a></iframe><p>We get a upper triangular matrix since we set the value of upper to True</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D23&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D23&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/d03447d719d619030a6caa841f3ebf76/href">https://medium.com/media/d03447d719d619030a6caa841f3ebf76/href</a></iframe><p>The Cholesky computation of a singular matrix cannot be calculated</p><p>Can be used in Single Value Decomposition</p><h3>torch.geqrf</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bbEh5nMwM2UHzlN8" /><figcaption>Photo by <a href="https://unsplash.com/@jontyson?utm_source=medium&amp;utm_medium=referral">Jon Tyson</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>torch.geqrf(input, output=None)</em></p><p><strong>Returns:</strong></p><ul><li>a tuple of Tensors (a,tau) of the form (Tensor,Tensor)</li></ul><p><strong>Utility:</strong></p><ul><li>Low level function for calling LAPACK directly. torch.qr() can also be used</li><li>Computes a QR decomposition of input, but without constructing QQ and RR as explicit separate matrices.</li></ul><p><strong>Requirements:</strong></p><ul><li>input matrix of a given shape</li></ul><p><strong>Params</strong></p><ul><li>input — the input tensor</li><li>out — (optional) — the output tensor</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D27&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D27&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c9e914a51a945275e3b8436594aba51/href">https://medium.com/media/3c9e914a51a945275e3b8436594aba51/href</a></iframe><p>A tuple of Tensors (a,tau) of the form (Tensor,Tensor) is returned.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D29&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D29&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/ccce43b115a4d3d0d2a63fbefb269843/href">https://medium.com/media/ccce43b115a4d3d0d2a63fbefb269843/href</a></iframe><p>The length of the tuple is 2</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D31&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D31&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb1ca47a867bb6ecb8541b41af234134/href">https://medium.com/media/fb1ca47a867bb6ecb8541b41af234134/href</a></iframe><p>Wrong data type passing can result in errors</p><p>Used in implict QR decomposition</p><h3>torch.logdet()</h3><p><em>torch.logdet(input)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*swgxmplilcgz8qEoGFtg6w.png" /><figcaption><strong>y = log(x) </strong>. Screenshot from my colab notebook</figcaption></figure><p><strong>Returns:</strong></p><ul><li>a Tensor</li></ul><p><strong>Utility:</strong></p><ul><li>Performs the computation of the log determinant of a square matrix or its batches</li></ul><p><strong>Requirements:</strong></p><ul><li>input tensor of shape (a,n,n) where a is whole number and n is a natural number ‘a’ denotes the batch size, if any, of the square matrices</li></ul><p><strong>Params:</strong></p><ul><li>input — the input tensor of shape (*,n,n)</li></ul><p>Note: -inf is returned if determinant is zero and nan if determinant is negative</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D35&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D35&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/243f5b0cbf8c5222e929c949136f9c31/href">https://medium.com/media/243f5b0cbf8c5222e929c949136f9c31/href</a></iframe><p>The log of the determinant of the square matrix is performed correctly</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D37&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D37&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/ed4ab30bfc3504d17b3c92c9e3525c73/href">https://medium.com/media/ed4ab30bfc3504d17b3c92c9e3525c73/href</a></iframe><p>The log of the determinant of the square matrix is performed correctly</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fjovian.ml%2Fembed%3Furl%3Dhttps%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D39&amp;dntp=1&amp;display_name=Jovian&amp;url=https%3A%2F%2Fjovian.ml%2Fsid0312%2F01-tensor-operations%2Fv%2F3%26cellId%3D39&amp;image=https%3A%2F%2Fjovianml.s3.amazonaws.com%2Fpreview-large.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;scroll=auto&amp;schema=jovian" width="800" height="600" frameborder="0" scrolling="no"><a href="https://medium.com/media/301e19a92db119786e028db9b0bf0e15/href">https://medium.com/media/301e19a92db119786e028db9b0bf0e15/href</a></iframe><p>This occurs because the matrix <strong>c </strong>is not a square matrix</p><p>Single Value Decomposition results are used when input is non invertible.</p><h3>Conclusion</h3><p>You have learnt about five awesome Pytorch Tensor operations, their return types, their arguments, their requirements, their utilities and how to use and not use them in Python code.</p><h3>Author’s Note</h3><p>This is just the beginning in your journey for deep learning. A lot remains to be discovered. So what are you waiting for ? Go, fall in love with Pytorch and dive deep into the ocean called Deep Learning</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qRMrUx7kd-U8t3Ac" /><figcaption>Photo by <a href="https://unsplash.com/@m_gatus?utm_source=medium&amp;utm_medium=referral">Gatis Marcinkevics</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e25d3ae8db05" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/five-pytorch-tensor-operations-you-had-absolutely-no-idea-about-e25d3ae8db05">Five PyTorch Tensor Operations you had absolutely no idea about</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OCT Image Segmentation using U-Nets]]></title>
            <link>https://medium.com/data-science/segmentation-of-optical-coherence-tomography-images-with-diabetic-macular-edema-the-gluon-82093cfd8e24?source=rss-6a0cec244490------2</link>
            <guid isPermaLink="false">https://medium.com/p/82093cfd8e24</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[inside-ai]]></category>
            <dc:creator><![CDATA[Siddhant Baldota]]></dc:creator>
            <pubDate>Fri, 29 May 2020 06:02:21 GMT</pubDate>
            <atom:updated>2020-06-21T20:37:07.017Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/inside-ai/home?gi=d53b80ec612">Inside AI</a></h4><h4>A deep learning approach for diagnosis of diabetic macular edema using Optical Coherence Tomography reports based on the Apache Mxnet and Gluon Framework</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*r7ZEGfln7WTZPVaS" /><figcaption>Photo by <a href="https://unsplash.com/@makuph?utm_source=medium&amp;utm_medium=referral">Claudia Ramírez</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4><strong>Diabetic Macular Edema (DME)</strong></h4><p>Diabetic Macular Edema is an aggravation caused due to a type- II diabetes called diabetic retinopathy. Diabetic retinopathy is one of the most commonly occurring diabetic eye disease and results in blindness more often than not. In a rough estimate, 7.7 million Americans are diagnosed with this diabetes and approximately 750,000 of them suffer from Diabetic Macular Edema .</p><h4>Optical Coherence Tomography (OCT)</h4><p>OCT is a diagnostic test for DME. This test uses a special light source along with a camera to obtain comprehensive inner cell layers of the retina. It perceives the thickness of the retina and is useful in determining the amount of swelling of the retina.</p><h4>U-Nets</h4><p>On the 18th of May, 2015, Olaf Ronneberger, Philipp Fischer, and Thomas Brox released a paper on Convoluted Neural Networks (CNNs) for Biomedical Image Segmentation. Traditionally CNNs were used for image classification and object detection. This approach wasn’t suitable for though as the desired output was that of localization where a class label was to be assigned to each pixel. Since biomedical tasks didn’t have a huge amount of positive samples in comparison to a image classification problem or an object detection problem. Ciresan et al. attempted training a network, using the sliding window setup to predict the class label of each pixel by using a region of interest or patch around that pixel as the input features. This method was though novel, was quite sedate and resulted in high amounts of redundancies because the patches overlapped. Moreover there was a trade-off between patch size and localization accuracy. Greater the patch-size, higher was the context but this resulted in the increase of max pooling layers which diminished the spatial information resulting in lower localization accuracy. If the patch-size was reduced, then the context reduced. This resulted in the creation of U-Nets, which are<em> psuedo-fully connected networks</em></p><p>The primary proposition was a providing a normal convolving network with consecutive layers sequentially. The max-pooling layers were substituted by upsampling layers, resulting in increase of pixel density and ultimately, resolution. These higher resolution features from the convolving path were concatenated with the upsampled output. Successive convolutional layers were added as the expanding path which resulted in an increase in the number of feature channels to enable the network pass on spatial information to the high resolution layers. This model doesn’t contain any fully connected layers and uses the most important part of each feature extractor (convolution). The overall structure was U-shaped and hence the name U-net. In case of larger images, the missing context was produced by flipping the input image and by performing different augmentations on the input image which didn’t change the meaning of the image but was helpful in extraction of more features per input image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*pYqOu9ctS98v0paIAZ0yQg.png" /><figcaption>source: ‘U-Net: Convolutional Networks for Biomedical Image Segmentation ‘</figcaption></figure><p>In reference to the above image, the authors of the paper used successive Conv3D layers with a ReLu activation function. The lowest resolution was taken as 32x32 pixels. Each sequential block of convolutional layers was max pooled and stacked on each other. To form the base of the U-Net, a plateau block consisting of 3 convolutional layers was created for data consistency which had the equal number of input and output channels. They were later up-sampled by transposing the convolutional layers. In turn, these layers were concatenated with the corresponding depth convolutional block. The output of the concatenation was passed on to the ascending limb of the U-Net or the expansive blocks. At each depth level, spatial information was continuously passed using concatenation.</p><p>You can check out the original paper at <a href="https://arxiv.org/abs/1505.04597">https://arxiv.org/abs/1505.04597</a> for further details.</p><h4>Dataset</h4><p>I have used the Segmentation of OCT images :<em> Optical coherence tomography (OCT) and diabetic macular edema (DME) dataset</em> which is made available on kaggle by Paul Mooney. You can take a look for yourself at <a href="https://www.kaggle.com/paultimothymooney/chiu-2015">https://www.kaggle.com/paultimothymooney/chiu-2015</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BAg9ILH25HkO91wESPRC-g.png" /><figcaption>screenshot from ‘kaggle.com’</figcaption></figure><p>The dataset consists of the following files</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*cVQFGt_0CkO5iRhIGCOt7w.png" /><figcaption>screenshot from my file explorer</figcaption></figure><h4>Complete Source Code</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WSQ2gIGdX2SRCFAB" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>You can find the complete source code of my implementation at my GitHub repository.</p><p><a href="https://github.com/sid0312/OCTImageSegmentation">sid0312/OCTImageSegmentation</a></p><p><strong>UNet using MxNet</strong></p><p>I built an U-Net from scratch using the <strong>Apache MxNet framework and the Gluon API</strong> which is analogous to Keras for the Tensorflow framework</p><p>The model.py in my repository is as follows</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0d833066166ea6a9708d2b2634d7d711/href">https://medium.com/media/0d833066166ea6a9708d2b2634d7d711/href</a></iframe><p><strong>Let us now look at our model.py file line by line</strong></p><ul><li><strong>Importing our dependencies</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/146f4cb6a065765f7f7aba8ded703716/href">https://medium.com/media/146f4cb6a065765f7f7aba8ded703716/href</a></iframe><ul><li><strong>Creating the class for the U-Net</strong></li></ul><p>The U-Net is implemented as a hybrid gluon block.</p><pre>class network(gluon.nn.HybridBlock):</pre><ul><li><strong>Creating the convoluting part function</strong></li></ul><p>This acts as a single contracting block which can be used many times in the convolving path. It consists of 3 Conv2D layers,each with a kernel size of 3, ReLu activation functions along with a BatchNorm layer to normalize the inputs, and one max pooling layer at the end. It is hybrid sequential block as information needs to be passed sequentially.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2471c3456a1672ceda05fa248fa01ed6/href">https://medium.com/media/2471c3456a1672ceda05fa248fa01ed6/href</a></iframe><ul><li><strong>Creating the deconvoluting part function</strong></li></ul><p>This acts as a single component of the expansive limb of the U-Net. It also can be used many times in the U-Net architecture. It consists of Conv2D layers,each with a kernel size of 3, ReLu activation functions along with a BatchNorm layer to normalize the inputs, and a Conv2DTranspose layer in order to expand the input size( in other words to increase the size of the feature map)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b36e59ce22b76deb40d530614971b27b/href">https://medium.com/media/b36e59ce22b76deb40d530614971b27b/href</a></iframe><ul><li><strong>Creating the plateau block function</strong></li></ul><p>This layer acts as a bottle neck for the U-Net. It was created for data consistency which had the equal number of input and output channels.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aed0acc50abb7a1d98deb1f516a08e22/href">https://medium.com/media/aed0acc50abb7a1d98deb1f516a08e22/href</a></iframe><ul><li><strong>Creating the output layer function</strong></li></ul><p>This layer acts as the output segmentation map for the U-Net. It occurs immediately after the expansive path. Note that this is almost similar to the convoluting part function, but it contains an extra parameter for robust feature extraction.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0d2b446cf394a8c0ca27fbd21b01c12/href">https://medium.com/media/e0d2b446cf394a8c0ca27fbd21b01c12/href</a></iframe><ul><li><strong>Creating the concatenate function</strong></li></ul><p>This is one of the most important functions in the model.py file. It concatenates the higher resolution features obtained from the convolving path were concatenated with the up-sampled output layer from the plateau block initially. It also pads the layers to give resultant layer of appropriate shape. It took me a long time to code this function and it had a lot of bugs due to shape inconsistency.</p><p>Nevertheless, this function works perfectly fine now. We have used edge padding in order to pad the feature map with the edge values of the input array. A brief look at the official documentation of padding in mxnet at <a href="http://beta.mxnet.io/r/api/mx.nd.pad.html">http://beta.mxnet.io/r/api/mx.nd.pad.html</a> would do a world of good in understanding the following code. Also try to look up the mxnet.nd.concat documentation at <a href="http://beta.mxnet.io/r/api/mx.nd.concat.html">http://beta.mxnet.io/r/api/mx.nd.concat.html</a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/234042a4f7b812e8d45b4658579608cb/href">https://medium.com/media/234042a4f7b812e8d45b4658579608cb/href</a></iframe><ul><li>Creating the __init__ function to assign the layers</li></ul><p>We have created the functions for each layer but to actually create the layers, we need to create the layers in the __init__ method. Note that the **kwargs is important for the hybrid sequential block arguments.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0fb58e24f9bec2ca31914dceecbabd0/href">https://medium.com/media/e0fb58e24f9bec2ca31914dceecbabd0/href</a></iframe><ul><li>Creating the hybrid_forward method</li></ul><p>We have assigned the layers but we need to sequentially stack them, pass the input ndarray and obtain the output ndarray.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b32c46484761d430eff67e5f3485ad33/href">https://medium.com/media/b32c46484761d430eff67e5f3485ad33/href</a></iframe><ul><li>Creating an object for the network class</li></ul><pre>net = network(input_channels=1,output_channels=2)</pre><p><strong>We have gone through the whole model.py. </strong>It is time to visualize our model</p><pre>print(net)</pre><p>Output :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/615/1*u2zfquemN_RjDJJr6JKHHQ.jpeg" /><figcaption>screenshot from my google colaboratory notebook</figcaption></figure><p>Let us intialize the weights of the network using Xavier initialization and check the model summary with a sample ndarray of shape (5,1,284,284) after obtaining the context (whether the code runs on a gpu or cpu)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/de0a8e20e50c62f0849a8e11ff0ab92b/href">https://medium.com/media/de0a8e20e50c62f0849a8e11ff0ab92b/href</a></iframe><p>Output :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*017mVv_NB_5_6GH53j43BA.jpeg" /><figcaption>screenshot from my google colab notebook</figcaption></figure><p>Enough Visualization. Now let us prepare our data using prepare_data.py</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2053c46d0eecf791dca5fbbe69303eed/href">https://medium.com/media/2053c46d0eecf791dca5fbbe69303eed/href</a></iframe><p>The create_pipeline method helps creating a input pipeline i.e. features and labels using the .mat files of the dataset.</p><p>The load_dataset method loads the input pipeline into two splits, train and validation</p><p>Now that we have our training features, training labels, validation features and validation, it’s time to train our model !!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/85c26f8534ba225b6583559f168dbcfd/href">https://medium.com/media/85c26f8534ba225b6583559f168dbcfd/href</a></iframe><p>We have a batch size of 6, run the training for 100 epochs.</p><p>The loss function used here is mxnet.gluon.loss.SoftmaxCrossEntropyLoss(). You can know more about losses at <a href="https://mxnet.incubator.apache.org/api/python/docs/api/gluon/loss/index.html">https://mxnet.incubator.apache.org/api/python/docs/api/gluon/loss/index.html</a></p><p>The optimizer used is a Gluon Trainer object. The optimization algorithm used is stochastic gradient descent with a learning rate of 0.0004. You can learn more about gluon trainers at <a href="https://mxnet.incubator.apache.org/api/python/docs/api/gluon/trainer.html">https://mxnet.incubator.apache.org/api/python/docs/api/gluon/trainer.html</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/904/1*iN9KgI4mIWaxfnL4oCRLpw.png" /><figcaption>Some logs from my training</figcaption></figure><p>I obtained a validation accuracy of 88 % which is pretty good.</p><p>After training our model, we obtain our model parameters, which we save in a file , let’s say, net.params</p><p>We can load our model for further predictions using the model.params file</p><p>Let us plot some results using the results.py file</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/646dec3e46ddf5187a6c78fe12d07844/href">https://medium.com/media/646dec3e46ddf5187a6c78fe12d07844/href</a></iframe><p>In order to understand this file requires some knowledge of matplotlib.pyplot. You can obtain this knowledge of data visualization at <a href="https://matplotlib.org/users/pyplot_tutorial.html">https://matplotlib.org/users/pyplot_tutorial.html</a></p><p><strong>Results</strong></p><p>Here are some results I obtained after using my trained model</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/1*Ypq-KMT_IEoqA43iltSJ_w.png" /><figcaption>Plotting the training examples and the results in my google colaboratory notebook</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*y0gdi0WvStnuAkf5rsXSPA.png" /><figcaption>Performance on our validation examples</figcaption></figure><p><strong>Conclusion</strong></p><p>U-Nets give better results than the sliding window segmentation model anyday. They also work on minimal data.</p><p><strong>Further Improvements</strong></p><p>Note that I have used Conv2D operations on this dataset. Conv3D would work even better on 4 dimensional datasets, which is generally the case in optical coherence tomography images beacuse the 3-D spatial information would be retained.</p><p><strong>Author’s Note</strong></p><p>Accuracy is not an appropriate parameter (Sensitivity and Specificity are) to judge the robustness of a model made for medical purposes but this notebook intends to explain the UNet Architecture using the Apache MXNet framework and not the evaluation metric as of now. Stayed tuned into the repository for further updates regarding the metrics</p><p>Happy Computer Vision and Happy Deep Learning. Thank you so much for your time. Really motivates me to develop more and write more.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=82093cfd8e24" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/segmentation-of-optical-coherence-tomography-images-with-diabetic-macular-edema-the-gluon-82093cfd8e24">OCT Image Segmentation using U-Nets</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Counting people on your webcam using GluonCV and MxNet]]></title>
            <link>https://medium.com/data-science/counting-people-on-your-webcam-using-gluoncv-and-mxnet-d1a9f05c427d?source=rss-6a0cec244490------2</link>
            <guid isPermaLink="false">https://medium.com/p/d1a9f05c427d</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[gluoncv]]></category>
            <category><![CDATA[ds-in-the-real-world]]></category>
            <dc:creator><![CDATA[Siddhant Baldota]]></dc:creator>
            <pubDate>Mon, 18 May 2020 17:02:30 GMT</pubDate>
            <atom:updated>2020-06-21T20:59:31.597Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/data-science-in-the-real-world/home?gi=ea0e55878a54">Data science in the real world</a></h4><h4>Based on the YOLOv3 object detection model</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oip10T3V6wlbKKyg" /><figcaption>Photo by <a href="https://unsplash.com/@stephanebernardphotography?utm_source=medium&amp;utm_medium=referral">Stéphane Bernard</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4><strong>Introduction</strong></h4><p>Counting people in images is one of the most common tasks of computer vision and has many use cases. In this tutorial, we have used the GluonCV toolkit to enable us in doing so. GluonCV provides cutting edge implementations of deep learning models in computer vision tasks. It consists of State-Of-The-Art pretrained models for Image Classification, Object Detection, Semantic Segmentation, Instance Segmentation, Pose Estimation, Object Tracking and Action Recognition. You can check out other models at the GluonCV model zoo.</p><pre><a href="https://gluon-cv.mxnet.io/model_zoo/index.html">https://gluon-cv.mxnet.io/model_zoo/index.html</a></pre><p>Generally in an object detection task, we have classes to be detected, a confidence score for each class and bounding boxes for the detected object in the image.</p><h4><strong>Detection</strong></h4><p>For example, consider the given image containing the dog, bicycle and truck-like car as prominent objects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nf0WWc_58nL7y67Xt6iDZA.png" /><figcaption>screenshot from my colab notebook</figcaption></figure><p>When we apply an object detection model to the image, our network checks for objects in the image, compares them to the ground truth label and returns a confidence score. The bounding box coordinates are in the form of a tuple consisting of the top left coordinates of the bounding box, height of the box and its width.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_qteoVK113HDMYVMfrDv1A.jpeg" /><figcaption>screenshot from my colab notebook</figcaption></figure><p>In order to detect people in an image, we check for the person class in the set of COCO ( Common Objects in Context) classes. The COCO <a href="http://mscoco.org/">dataset</a> is a fantastic object detection dataset with 80 classes, 80,000 training images and 40,000 validation images. Our model ‘yolo3_darknet53_coco’ is pretrained on this dataset.</p><p>In order to stream our webcam data, we use an Open Source Computer Vision Library, OpenCV</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/1*4gKsiOYn9VRnefXNvBVD3Q.png" /><figcaption>OpenCV is an open source machine learning library used for computer vision tasks , source: ‘OpenCV’, via: wikipedia</figcaption></figure><h4><strong>Prerequisites</strong></h4><p>Before installing the following, create a virtual environment in Anaconnda (preferred)</p><ol><li>OpenCV</li></ol><p>To install OpenCV in your system, type<strong> pip install opencv-python </strong>in your terminal or command prompt. I have used Spyder in Anaconda for this task, so I have installed the dependencies in my conda virtual environment.</p><pre>pip install opencv-python</pre><p>2. MXNet and Gluon</p><p>To install mxnet and gluon, type <strong>pip install mxnet-cu101 gluoncv </strong>in your terminal or prompt. I have done the same in my virtual environment. You can install the non CUDA based mxnet if you don’t have CUDA in your virtual environment</p><pre>pip install mxnet-cu101 </pre><pre>pip install gluoncv</pre><h4>GitHub</h4><p>If you’re familiar with GitHub, check out the following repository.</p><p><a href="https://github.com/sid0312/people_counter">sid0312/people_counter</a></p><p>Let us have a look at my source code for the given object detection task</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a4026420ad8910f3ee4c43eea21d18ab/href">https://medium.com/media/a4026420ad8910f3ee4c43eea21d18ab/href</a></iframe><ul><li><strong>Import our dependencies</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b0aef2ea80bd6735159613b2dcfcc71f/href">https://medium.com/media/b0aef2ea80bd6735159613b2dcfcc71f/href</a></iframe><ul><li><strong>Define the model to be used</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c8827912407f11515c26d076f115f01c/href">https://medium.com/media/c8827912407f11515c26d076f115f01c/href</a></iframe><ul><li><strong>Add utility functions</strong></li></ul><p>a) To convert our frame to which is a NumPy ndarray to a mxnet ndarray</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/64498f47b391e92515e309296036a6f3/href">https://medium.com/media/64498f47b391e92515e309296036a6f3/href</a></iframe><p>b) To apply transformations to the mxnet ndarray to supply it to the network</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8b8feac5464ced71f044703bcdeddc4a/href">https://medium.com/media/8b8feac5464ced71f044703bcdeddc4a/href</a></iframe><p>c) To get the class_ids, confidence scores and bounding boxes by applying the network to the data</p><pre>def detect(network,data):<br>  prediction = network(data)<br>  class_ids, scores, bounding_boxes = prediction<br>  return class_ids, scores, bounding_boxes</pre><p>d) The count_object function which counts the number of objects of a given object label over a given threshold</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/afc0ec2f6ddc879054fa2075af86839e/href">https://medium.com/media/afc0ec2f6ddc879054fa2075af86839e/href</a></iframe><ul><li><strong>Create a People_Counter class to sequentially use all utility functions and return predictions, as well as people, count in separate functions</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3922b0fd6be91c7afaf11ebffe6eea0c/href">https://medium.com/media/3922b0fd6be91c7afaf11ebffe6eea0c/href</a></iframe><p>The __init__ method imports the network from the GluonCV model zoo with pretrained as ‘true’ because we are using a pretrained model. Used hybridize() to speed up the network.</p><p>The set_threshold method sets the threshold for the confidence score of the objects. Detections having confidence score below the threshold won’t be considered.</p><p>The count_people method counts the number of people in an frame with the help of the utility functions</p><p>The detect_people method returns the detections on the frame.</p><p>The class_names returns all the 80 classes in the COCO dataset which are also the classes of the network since it is pretrained on that dataset.</p><ul><li><strong>Creating an object for the People_Counter class and setting the threshold to 0.5 . Detections below a confidence score of 0.5 will be taken into account.</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5e520ebb42756177ebc7b05bca71018e/href">https://medium.com/media/5e520ebb42756177ebc7b05bca71018e/href</a></iframe><ul><li><strong>Load the webcam handler and let the webcam autofocus for a second</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/11fbfa00898312dc01f69dcdbc689936/href">https://medium.com/media/11fbfa00898312dc01f69dcdbc689936/href</a></iframe><ul><li><strong>Set up the loop for performing detection and counting on each frame and displaying the results in a window using the gluoncv.utils.viz library</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cf9655aaef04b624b0236dcc965ddfbc/href">https://medium.com/media/cf9655aaef04b624b0236dcc965ddfbc/href</a></iframe><p>video.read() returns two values, of which the first is discarded and the second becomes our frame.</p><p>Use the count() and detect_people() of the counter object</p><p>Get the predictions and plot the bounding boxes on each frame. Put the text of the count on the frame. Display the frame using cv2.imshow()</p><p>If the key ‘q’ is pressed in the context of the window, break out of the loop.</p><ul><li><strong>Release the webcam and destroy all windows</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c0755e66a783d85afcd9ec2f6d4d7c3a/href">https://medium.com/media/c0755e66a783d85afcd9ec2f6d4d7c3a/href</a></iframe><p><strong>Results</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*N27BYe-fRMW4MhccmUVvQw.png" /><figcaption>A single frame of the detection</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*TXKJCGJUqEV-yaiLWJQ1EA.png" /><figcaption>Some logs over a particular time until I pressed ‘q’</figcaption></figure><blockquote><strong><em>Thank you so much for your patience. Happy Computer Vision and Happy Deep Learning.</em></strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d1a9f05c427d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/counting-people-on-your-webcam-using-gluoncv-and-mxnet-d1a9f05c427d">Counting people on your webcam using GluonCV and MxNet</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>