<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ivan Itzcovich on Medium]]></title>
        <description><![CDATA[Stories by Ivan Itzcovich on Medium]]></description>
        <link>https://medium.com/@itzco?source=rss-29dc149dca0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*lI1Uxrfm1rgNTS1wQ-lj1w.png</url>
            <title>Stories by Ivan Itzcovich on Medium</title>
            <link>https://medium.com/@itzco?source=rss-29dc149dca0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 20:09:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@itzco/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[faced: CPU Real Time face detection using Deep Learning]]></title>
            <link>https://medium.com/data-science/faced-cpu-real-time-face-detection-using-deep-learning-1488681c1602?source=rss-29dc149dca0------2</link>
            <guid isPermaLink="false">https://medium.com/p/1488681c1602</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[facedetection]]></category>
            <category><![CDATA[convolutional-network]]></category>
            <dc:creator><![CDATA[Ivan Itzcovich]]></dc:creator>
            <pubDate>Wed, 26 Sep 2018 18:08:15 GMT</pubDate>
            <atom:updated>2018-09-26T18:58:44.758Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Is it possible to implement object detection models with real-time performance without GPU? </em>faced is a proof of concept that it is possible to build a custom object detection model for a single class object (in this case, faces) running in <strong>real time</strong> <strong>on a CPU.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tdz9Pm3pTT75ix1yZDCSLg.png" /><figcaption><a href="https://www.kairos.com/blog/face-detection-explained">Face detection task</a></figcaption></figure><h3>What is the problem?</h3><p>There are many scenarios where a single class object detection is needed. This means that we want to detect the location of all objects that belong to a specific class in an image. For example, we could be detecting faces for a face identification system or people for pedestrian tracking.</p><p>What is more, most of the time we would like to run these models <strong>in real time. </strong>In order to achieve this, we have a feed of images providing samples at rate <em>x </em>and we need a model to run in less than rate <em>x</em> for each of the samples. <strong>Then, we can process images as soon as they are available.</strong></p><p>The most accessible and used solution nowadays to solve this task (and many others in computer vision) is to perform <a href="https://en.wikipedia.org/wiki/Transfer_learning"><em>transfer learning</em></a> on previously trained models (in general standard models trained on huge datasets like those found in <a href="https://www.tensorflow.org/hub/">Tensorflow Hub</a> or in <a href="https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md">TF Object Detection API</a>)</p><p>There are plenty of trained object detection architectures (e.g. FasterRCNN, SSD or YOLO) that achieve impressive accuracy within real-time performance <strong>running on GPUs</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/1*vPSB9em5LqVFw10hIUqe9w.png" /><figcaption>Extracted from SSD paper <a href="https://arxiv.org/pdf/1512.02325.pdf">here</a></figcaption></figure><p>GPUs are expensive but necessary in the training phase. However, in inference <strong>having a dedicated GPU to achieve real-time performance is not viable. All of the general object detection models (as those mentioned above) fail to run in real time without a GPU.</strong></p><blockquote><em>Then, how can we revisit the object detection problem for single class objects to achieve real-time performance but on CPU?</em></blockquote><h3>Main idea: simpler tasks require less learnable features</h3><p>All of the above mentioned architectures were designed to detect multiple object classes (trained on <a href="https://cocodataset.org/">COCO</a> or <a href="http://host.robots.ox.ac.uk/pascal/VOC//">PASCAL VOC</a> datasets). In order to be able to classify each bounding box to it’s appropriate class, these architectures require a massive amount of feature extraction. This translates to huge amount of learnable parameters, huge amount of filters, huge amount of layers. In other words, <strong>this networks are big</strong>.</p><p>If we define simpler tasks (rather than multiple-class bounding box classification) then we can think of the network needing to learn less features to perform the task. Detecting a face in an image is obviously more simple than detecting cars, people, traffic signs and dogs (all within the same model). <strong>The amount of features required by a Deep Learning model in order to recognize faces (or any single class object) will be less than the amount of features for detecting tens of classes at the same time. The required information to perform the first task is less than the latter task.</strong></p><blockquote><strong>Single class object detection models will need less learnable features. Less parameters mean that the network will be smaller. Smaller networks run faster because it requires less computations.</strong></blockquote><blockquote><strong>Then, the question is: how small can we go to achieve real time performance on CPU but keeping accuracy?</strong></blockquote><p>This is faced main concept:<strong> building the smallest possible network to (hopefully) run in real time in CPU while keeping accuracy.</strong></p><h3>The architecture</h3><p>faced is an ensemble of 2 neural networks, both implemented using <a href="http://tensorflow.org/">Tensorflow</a>.</p><h4>Main network</h4><p>faced main architecture is heavily based on <a href="https://pjreddie.com/darknet/yolo/">YOLO</a>’s architecture. Basically, it’s a Fully Convolutional Network (FCN) that runs a 288x288 input image through a series of convolutional and pooling layers (no other layer types are involved).</p><p>Convolutional layers are in charge of extracting space-aware features. Pooling layers increase the receptive field of consequent convolutional layers.</p><p>The architecture’s output is a 9x9 grid (versus 13x13 grid in YOLO). Each grid cell is in charge of predicting whether a face is inside that cell (versus YOLO where each cell can detect up to 5 different object).</p><p>Each grid cell has 5 associated values. The first one is the probability <em>p </em>of that cell containing the center of a face. The other 4 values are the (<em>x_center, y_center, width, height) </em>of the detected face (relative to the cell).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-w9qXAOq8NsRDcN8E8-0vQ.png" /><figcaption>Main architecture</figcaption></figure><p>The exact architecture is defined as follows:</p><ul><li>2x [8 filter convolutional layer on 288x288 image]</li><li>Max pooling (288x288 to 144x144 feature map)</li><li>2x [16 filter convolutional layer on 144x144 feature map]</li><li>Max pooling (144x144 to 72x72 feature map)</li><li>2x [32 filter convolutional layer on 72x72 feature map]</li><li>Max pooling (72x72 to 36x36 feature map)</li><li>2x [64 filter convolutional layer on 36x36 feature map]</li><li>Max pooling (36x36 to 18x18 feature map)</li><li>2x [128 filter convolutional layer on 18x18 feature map]</li><li>Max pooling (18x18 to 9x9 feature map)</li><li>4x [192 filter convolutional layer on 9x9 feature map]</li><li>5 filter convolutional layer on 9x9 feature map for the final grid</li></ul><p>All activation function are leaky_relu.</p><blockquote>faced has 6,993,517 parameters. YOLOv2 has 51,000,657 parameters. It’s size is 13% of YOLO’s size!</blockquote><h4>Auxiliary network</h4><p>(<em>x_center, y_center, width, height) </em>outputs of the main network were not as accurate as expected. Hence, a small CNN network was implemented to take as input a small image containing a face (cropped with the main architecture outputs) and to output a regression on the ground truth bounding box of the face.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*ZxfOG9eSKfja65EzhU7fZg.png" /><figcaption>The network takes a crop containing a face and predicts the correct bounding box</figcaption></figure><p>It’s only task is to complement and improve the output coordinates of the main architecture.</p><p>The specific architecture of this network is not relevant.</p><h3>Dataset</h3><p>Both networks were trained on the <a href="http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/">WIDER FACE dataset</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0J_Sl81iLs5XRCLIzsBQhQ.jpeg" /><figcaption>WIDER FACE multiple scenarios</figcaption></figure><blockquote>“WIDER FACE dataset is a face detection benchmark dataset […]. We choose <strong>32,203</strong> images and label <strong>393,703</strong> faces with a high degree of variability in scale, pose and occlusion as depicted in the sample images.”</blockquote><h3>Training</h3><p>Training was done on an <a href="https://www.nvidia.com/en-us/titan/titan-xp/">Nvidia Titan XP</a> GPU. Training time took ~20 hours. <a href="https://arxiv.org/abs/1502.03167">Batch Normalization</a> technique was used to help convergence and dropout was used (at 40% rate) as regularization method to avoid overfitting.</p><h3>Inference and Non max suppression</h3><p>When using faced for inference, first the image is resized to 288x288 in order to be fed into the network. The image goes under the FCN giving the 9x9 grid output described above.</p><p>Each cell has a probability <em>p </em>of containing an image. Cells are filtered by a configurable threshold (i.e. only cells with <em>p &gt; t</em> are kept). For those kept cells, the face is located using the cell’s (<em>x_center, y_center, width, height).</em></p><p>There are some cases where multiple cells can compete for the same face. Let’s suppose that a face center is located in the exact location where 4 cells intersect. Those 4 cells could have a high <em>p</em> (probability of containing a face center inside the cell). If we kept all cells and project the face coordinates of each cell, then we would see the same face with 4 similar bounding boxes around it. This problem is fixed through a technique called <a href="https://www.pyimagesearch.com/2014/11/17/non-maximum-suppression-object-detection-python/"><strong>non max suppression</strong></a>. The result is shown in the following image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/285/1*Mi6IPSd0gSdtCvKUBzZi9Q.jpeg" /><figcaption>Non max suppression example.</figcaption></figure><h3>Performance</h3><p>faced is able to achieve the following speed on inference:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/758/1*5wkRfA8gUcfwDdFoa1XXGA.png" /></figure><p>Pretty good considering that YOLOv2 cannot achieve even 1FPS on an i5 2015 MBP.</p><h3>Results</h3><p>Let’s see some results!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*gwC5tknaFN8SOKRqVpbyYA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*A3ke5xblWCos0wWkYtraSw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*iCNwCWlaHqn3TT9vpEpI4Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*afYSdEHidlC-UzmTAxa8_Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/414/1*hidCftzR20n1A6RTXKtNrQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uBEAhu5ngPMnEuX3rk5N3g.png" /><figcaption>faced on images</figcaption></figure><p>Now let’s see a comparison between faced and <a href="https://docs.opencv.org/3.4/d7/d8b/tutorial_py_face_detection.html">Haar Cascades</a>, which is a computer vision traditional approach that does not use Deep Learning. Both methods run under similar speed performance. faced shows significant more accuracy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*bQVE1o-cSwi4EZiFiYBd5A.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*_K2pmLW2rgQMe8YYd3-SWw.gif" /><figcaption>Haar Cascade [left] vs faced [right]</figcaption></figure><h3>How to use faced?</h3><p>faced is a really simple program that can be used both embeded in Python code or as command line program.</p><p>Go to the github repo for further instructions:</p><p><a href="https://github.com/iitzco/faced">iitzco/faced</a></p><p><em>Liked the project? </em>Leave a ⭐ on the project’s repo!</p><h3>Conclusion</h3><p>faced is a proof of concept that you don’t always need to rely on general purpose trained models in scenarios were these models are an overkill to your problem and performance issues are involved. <strong>Don’t overestimate the power of spending time designing custom neural network architectures </strong>that are specific to your problem. These specific networks will be a much better solution than the general ones.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1488681c1602" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/faced-cpu-real-time-face-detection-using-deep-learning-1488681c1602">faced: CPU Real Time face detection using Deep Learning</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing TFServe: Simple and easy HTTP server for tensorflow model inference]]></title>
            <link>https://medium.com/data-science/introducing-tfserve-simple-and-easy-http-server-for-tensorflow-model-inference-582ea1b07da8?source=rss-29dc149dca0------2</link>
            <guid isPermaLink="false">https://medium.com/p/582ea1b07da8</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[http-server]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[Ivan Itzcovich]]></dc:creator>
            <pubDate>Wed, 12 Sep 2018 22:10:58 GMT</pubDate>
            <atom:updated>2018-09-12T22:10:58.161Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DrKc7wEs4O5q6mwUCy0dCg.jpeg" /></figure><p>The most interesting (and fun) part of Machine Learning is to make use of the models you spent time training. While there are lots of resources on how to train your models, we often find it difficult to figure out how to deploy a system that uses the trained model for inference (i.e making predictions). <strong>TFServe is a framework designed to serve tensorflow models in a simple and easy way through an HTTP server.</strong></p><p>TFServe is a micro-framework built on top of <a href="https://github.com/encode/apistar">apistar</a>. To install it, just run:</p><pre>$ pip install tfserve</pre><h3>How to use TFServe?</h3><p>Let’s start with this frequent scenario. You’ve found an amazing Tensorflow Inception CNN model for Image Classification trained by Google on <a href="http://www.image-net.org/">ImageNet</a>. <strong>You</strong> <strong>wish to build an HTTP service that uses this model for inference.</strong></p><p>From the description you figure out that this model receives 224x224 normalized RGB images and returns the most probable class (from the 1000 classes available in ImageNet). You download the model, getting a frozen_graph.pb (or similar) file.</p><p><em>Now what? </em>Instructions on how to make it run involve knowledge about the model arquitecture (which you don’t know anything about), TF Serving or other tools. <strong>How difficult can it be to run a HTTP server for inference? Here is where </strong><strong>tfserve comes in action:</strong></p><h4>You will need 5 parts:</h4><ol><li><strong>Model file</strong>: it can be a .pb file or a model directory containing ckpt files.</li><li><strong>Input tensor names</strong>: name of the input tensors of the graph.</li><li><strong>Output tensor names</strong>: name of the output tensors of the graph.</li><li>encode: python function that receives the request body data and outputs a dict mapping input tensor names to input <em>numpy</em> values.</li><li>decode: python function that receives a dict mapping output tensor names to output <em>numpy</em> values and returns the HTTP response.</li></ol><h4>Input and Output tensor names?</h4><p>As you probably know, each tensorflow graph tensor is identified by a unique name. This name is declared when creating the graph. You will need to specify to tfserve what the names of the input tensors are. Same goes for output tensors.</p><blockquote>But I’ve downloaded this model from some Tensorflow repo and I have no idea of the input/output tensor names!</blockquote><p>Don’t panic. Just use tfserve.helper.estimate_io_tensors function to learn about possible i/o tensors for you to pick.</p><p>For example,</p><pre>&gt; import tfserve import helper<br>&gt; helper.estimate_io_tensors(&quot;frozen_model.pb&quot;)</pre><pre>Possible INPUT tensors:<br>        import/img:0<br><br>Possible OUTPUT tensors:<br>        pred/out:0<br>        pred/softmax_out:0</pre><blockquote><em>Great!</em> I will need import/img:0 for my input tensor and pred/softmax_out:0 for my output tensor (that will be a multi-class probability distribution).</blockquote><h4>Encode function</h4><p>Implement a function that receives the HTTP request body data and outputs a dict mapping input tensor names to input numpy values.</p><p>Following the Image Classification example, the encode function will receive the binary data provided in the HTTP request body and should output a dict mapping “import/img:0” to a 224x224 normalized RGB numpy image:</p><pre>def encode(request_data):<br>    with tempfile.NamedTemporaryFile(mode=&quot;wb&quot;, suffix=&quot;.jpg&quot;) as f:<br>        f.write(request_data)<br>        img = PIL.Image.open(f.name).resize((224, 224)) <br>        img = np.asarray(img) / 255.<br><br>    return {&quot;import/img:0&quot;: img}</pre><h4>Decode function</h4><p>Implement a function that receives a dict mapping output tensor names to output numpy values and returns the HTTP response.</p><p>Following the Image Classification example, the decode function will receive a dict mapping “pred/softmax_out:0” to a 1000-size array with the class probabilities. I wish to return a JSON object like:</p><pre>{<br>    class: &quot;german shepard&quot;,<br>    prob: 0.98,<br>}</pre><p>That contains information about the most likely class.</p><p>Then, the decode function will be:</p><pre>def decode(outputs):<br>    p = outputs[&quot;pred/softmax_out:0&quot;]<br>    index = np.argmax(p)<br>    return {<br>               &quot;class&quot;: index_to_class_name(index),<br>               &quot;prob&quot;: float(p[index])<br>           }</pre><h4>Running the server</h4><p>You can now run the server providing all 5 parts described above (<em>model path, input tensors, output tensors, </em><em>encode and </em><em>decode functions</em>):</p><pre>from tfserve import TFServeApp</pre><pre>app = TFServeApp(&quot;frozen_graph.pb&quot;, [&quot;import/img:0&quot;],<br>                 [&quot;pred/softmax_out:0&quot;], encode, decode)</pre><pre>app.run(&#39;127.0.0.1&#39;, 5000, debug=True)</pre><h3>How to send an actual image to the model?</h3><p>Server will be up and running in 127.0.0.1:5000. In order to run the model, you should send a <strong>POST</strong> request to / with the binary image as the request body. You should get the JSON response implemented in the decode function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YRGYRAwR0loT4fiWs4zH8w.gif" /><figcaption>Using the server through <a href="https://www.getpostman.com/">Postman</a></figcaption></figure><p>You could provide any kind of data in the request body. You just need to handle it correctly in the encode function in order to feed the input tensors of the graph.</p><p>Also, you can return any kind of data in the HTTP response (besides JSON objects, it could contain binary images with segmentation information for example). You just need to build the response in the decode function.</p><h3>Conclusion</h3><p>With tfserve, running a tensorflow trained model is really simple and easy. Just provide the model path, the input/output tensor names, the encode function that transforms the request data into the input tensors and the decode function that transforms the output tensors into the response. That’s it! tfserve will handle the rest.</p><h3>For more information, go to the project site!</h3><p><a href="https://github.com/iitzco/tfserve">iitzco/tfserve</a></p><p><em>Liked the project? </em>Leave a ⭐ on the github repo!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=582ea1b07da8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/introducing-tfserve-simple-and-easy-http-server-for-tensorflow-model-inference-582ea1b07da8">Introducing TFServe: Simple and easy HTTP server for tensorflow model inference</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>