<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Moonvision - Medium]]></title>
        <description><![CDATA[MoonVision automates visual inspection tasks with proprietary computer vision and deep learning tools bundled in the Moonvision Toolbox. We focus on edge deployment, active learning and training data management and enable domain experts to work with next generation technology. - Medium]]></description>
        <link>https://medium.com/moonvision?source=rss----a341496eb4cf---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Moonvision - Medium</title>
            <link>https://medium.com/moonvision?source=rss----a341496eb4cf---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 15:39:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/moonvision" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Computer Vision in Production — Podcast]]></title>
            <link>https://medium.com/moonvision/computer-vision-in-production-podcast-db3e2ec975a5?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/db3e2ec975a5</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[podcast]]></category>
            <category><![CDATA[research]]></category>
            <dc:creator><![CDATA[Alexander Hirner]]></dc:creator>
            <pubDate>Wed, 09 Jun 2021 07:23:26 GMT</pubDate>
            <atom:updated>2021-06-14T05:58:40.347Z</atom:updated>
            <content:encoded><![CDATA[<h3>Computer Vision in Production — Podcast</h3><p>Earlier this year, I joined Anthony Kelly from <a href="https://www.trustinsoda.com/">SODA</a> for their podcast series. Anthony focuses on recruiting Computer Vision specialists.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*QiIqkdaB1ExcptXBEnCYYQ.jpeg" /><figcaption>Also check other episodes on <a href="https://soundcloud.com/anthony-kelly-633324952">Soundcloud</a></figcaption></figure><p>Against this background, we discussed the milestones of MoonVision, how our technology evolved and some advice on bringing cutting edge-research to products that need to learn continuously:</p><ul><li><em>0:26 </em>Alexander Hirner — CTO &amp; co-founder at The Moonvision</li><li><em>5:02</em> Inception of Moonvision</li><li><em>11:55</em> Introducing CV and Deep Learning: How did you get started</li><li><em>15:01</em> Biggest Leanings &amp; Takeaways</li><li><em>19:20</em> Data Annotation &amp; Q&amp;A for CV</li><li><em>29:05</em> What makes going from research to production so difficult</li></ul><p><a href="https://soundcloud.com/anthony-kelly-633324952/computer-vision-in-production-alexander-hirner">https://soundcloud.com/anthony-kelly-633324952/computer-vision-in-production-alexander-hirner</a></p><p>If you are curious on particular topics or want to work with us, don’t hesitate to reach out. See you next time!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db3e2ec975a5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/computer-vision-in-production-podcast-db3e2ec975a5">Computer Vision in Production — Podcast</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Anomaly Detection and Segmentation for Surface Inspection]]></title>
            <link>https://medium.com/moonvision/anomaly-detection-and-segmentation-for-surface-inspection-ea101f2f4e24?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/ea101f2f4e24</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[anomaly-detection]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Ankit Kumar]]></dc:creator>
            <pubDate>Sat, 08 Feb 2020 23:46:45 GMT</pubDate>
            <atom:updated>2020-02-08T23:46:45.547Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JmRUg-82N0vMJdByFX9IdA.jpeg" /></figure><h4>What is Anomaly Detection:</h4><p>Anomaly detection is the identification of items that do not conform to an expected pattern or to other items in a dataset that can only be detected by well-trained experts. Such anomalies can be translated into problems such as structural and surface inspection, medical image diagnosis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8NhBBsOdwmeWbzTskpfopg.png" /><figcaption>Anomaly detected and segmented in Wood and Metal Strip.</figcaption></figure><h4>Why it is important?</h4><p>Quality of goods has become increasingly important to save natural resources and costs. Modern businesses are beginning to understand the importance of manufacturing and delivering quality products. Quality checks and surface inspection performed by human experts are highly time-consuming, costly and less effective to detect defects or anomalies. Not only in the manufacturing industry but also in the medical domain, we face the same problem due to scarce expert know-how.</p><p><strong>Anomaly detection</strong> is key for solving such tasks automatically.</p><h4>Artificial Intelligence for Anomaly detection and segmentation:</h4><p>One of the major concerns of research groups and companies working on anomaly detection tasks in computer vision is to obtain large amounts of labeled training data and to deal with the problem of imbalanced data. As a result, many supervised and semi-supervised approaches do not perform well.</p><p>To deal with such a crucial issue, our team at MoonVision proposes a very effective solution using <strong>one-class supervised learning</strong>. Through an image completion approach comparatively little and much cheaper to obtain data is required. The method achieves state of the art in <strong>discovering structural anomalies</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/1*Fm7RPNckepfCgLKtkKfskQ.png" /><figcaption>Autoencoder and decoder image completion architecture.</figcaption></figure><h4>Our Approach:</h4><p>We used a <strong>one-class unsupervised learning image completion</strong> approach on fault-free samples by training a deep convolutional neural network ( an autoencoder and decoder) to complete images in which some regions are cut out. The network was trained exclusively on fault-free data, it completes the image patches with a fault-free version of the missing image region. The pixel-wise reconstruction error within the cut-out region is an anomaly image that is used to score the degree of anomal regions. Results on surface images of, e.g. painted or milled surface areas demonstrate that this approach is suitable for the detection of visible anomalies and moreover surpasses all other tested methods.</p><p><strong>Results and Inference:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rBL6MNwSe4B-QW9QbZ1_2g.png" /><figcaption>Actual Image(Metal strip with rust), Heatmap and Anomaly/Rust detected and segmented.</figcaption></figure><h4>Conclusion</h4><p>Our <strong>one-class unsupervised learning image completion</strong> approach is the state of the art approach for solving <strong>anomaly detection and segmentation</strong> tasks in artificial intelligence which require a small amount of data.</p><p><a href="https://www.moonvision.io/products/"><strong>Our Products:</strong></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m0u7xwDBBz0K1FkPzItbGA.png" /><figcaption>Wood Scanner.</figcaption></figure><p><em>Check out what we do at</em><a href="https://www.moonvision.io/"><em> </em><strong><em>https://www.moonvision.io/</em></strong></a><strong><em> </em></strong><em>and check our platform at</em><a href="https://app.moonvision.io/signup"><em> </em><strong><em>https://app.moonvision.io/signup</em></strong></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea101f2f4e24" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/anomaly-detection-and-segmentation-for-surface-inspection-ea101f2f4e24">Anomaly Detection and Segmentation for Surface Inspection</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Want better accuracy for classification models? Let’s use “Attention maps”!]]></title>
            <link>https://medium.com/moonvision/want-better-accuracy-for-classification-models-lets-use-attention-maps-6d687c8e7625?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/6d687c8e7625</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[accuracy]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Ankit Kumar]]></dc:creator>
            <pubDate>Sun, 11 Aug 2019 08:42:37 GMT</pubDate>
            <atom:updated>2019-08-14T07:25:24.396Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the major concerns of research groups and companies dealing with classification tasks in computer vision is to tackle the scenario where images belonging to different classes look very similar. As a result, the classification convolutional neural network architecture does not work very well with such datasets.</p><p>To deal with such a common yet crucial issue in supervised learning, MoonVision proposes a solution, using <a href="https://medium.com/moonvision/few-shot-object-detection-in-practice-4f8fa98cba57"><strong><em>few-shot classification</em></strong></a><strong><em> </em></strong>with attention mechanism, which helps the convolutional neural network architecture learn fine-grained local details of an image. As attention maps focus on the fine-grained details of an image, it helps the convolution neural architecture distinguish between the images which look very similar but belong to different categories. This potentially increases the accuracy of deep learning architectures.</p><p>Qualitative results for Cars196 dataset by using our <a href="https://medium.com/moonvision/few-shot-object-detection-in-practice-4f8fa98cba57"><strong><em>few-shot classification</em></strong></a><strong><em> </em></strong>with an attention mechanism.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Po6hRd6NK9llm14X5LYiAw.png" /></figure><h3><strong>Let’s see the magic of “Attention maps” for better Interpretability</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bHPT8D6T8vqTOm2FO2Swww.png" /></figure><p>Here we have shown where our model incorporates attention during training for <a href="http://www.vision.caltech.edu/visipedia/CUB-200.html"><strong>CUB200</strong></a> and <a href="https://ai.stanford.edu/~jkrause/cars/car_dataset.html"><strong>CARS196</strong></a> dataset compared to other methods that just visualize <a href="https://github.com/utkuozbulak/pytorch-cnn-visualizations"><strong>activations</strong></a> for different layers of the model.</p><p>Indeed, local features play a critical role in many fine-grained visual recognition tasks. Typical deep neural networks designed for image classification are good at extracting high-level global features, but the features of local details are often missing. This could limit the Attention maps in exploring local details to distinguish the difference between images from different classes. For example, without local details, the deep learning models could not learn about fine-grained details of an image that could not force a model to focus on most discriminative parts such as logos, lights, etc.. in identifying different cars.</p><h3><strong>Conclusion</strong></h3><p>Our few shot classifications with “<strong>Attention mechanism</strong>” is the state of the art image classifier which extracts high-level global features as well as low-level local features to get better results.</p><p><em>Check out what we do at </em><a href="https://www.moonvision.io/"><strong><em>https://www.moonvision.io/</em></strong></a><strong><em> </em></strong><em>and check our platform at </em><a href="https://app.moonvision.io/signup"><strong><em>https://app.moonvision.io/signup</em></strong></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d687c8e7625" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/want-better-accuracy-for-classification-models-lets-use-attention-maps-6d687c8e7625">Want better accuracy for classification models? Let’s use “Attention maps”!</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why we love Hackathons]]></title>
            <link>https://medium.com/moonvision/why-we-love-hackathons-1101a5742dcd?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/1101a5742dcd</guid>
            <category><![CDATA[hackathons]]></category>
            <category><![CDATA[events]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Tatiana Statsenko]]></dc:creator>
            <pubDate>Mon, 29 Jul 2019 08:47:46 GMT</pubDate>
            <atom:updated>2019-07-29T08:47:46.583Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*YZgmnZPeacBQ2k5T9J8ePA.jpeg" /><figcaption>source: <a href="http://www.ceskatelevize.cz/">www.ceskatelevize.cz</a></figcaption></figure><p>The term “hackathon” was first used just about 20 years ago. First meetings involved aspiring enthusiasts that came together to solve particular problems. Later, companies started to organise events at their premises to boost productivity of the employees in a limited amount of time. Hackathons became part of the software community as events where people could hang out and work on various projects. Nowadays, hackathons are organised not only for coders, but also for people from other backgrounds, such as product design, economics or natural sciences, and became events where creative ideas of all kinds emerge.</p><p>When a large company organises an open hackathon, there can be several purposes: to look for people that can solve their problems, crowdsource for solutions in specific fields or to bring more attention to the company itself. The attendees can win high value prizes, get hired by the company or end up as collaborators on follow-up projects. As a result, teams which brought smart ideas to life in 24 or 48 hours brainstorming sessions started to create their own companies based on the solutions that they have developed. MoonVision is one of such companies, emerged from <a href="https://www.industr.com/de/smart-factory-hackathon-1976760">Smart Factory Hackathon</a>, hosted by Audi in 2016.</p><p>While at large hackathons the pressure is extreme and the stakes are high, there is always space for smaller events where people tend to just have fun while not being super competitive. Such events bring new followers and stars to your GitHub account, and give the possibility to learn something new while having a good time with amazing people and free pizza. Sometimes, it is not even necessary to find a group of people to participate — single attendees can always become a part of another team or work on a project alone.</p><p>We love to attend hackathons as a company! The opportunities are endless:</p><ul><li>Creating visibility for the company on a business level</li><li>Making other people interested in working with us</li><li>Getting real money that can be spent on <a href="http://tiny.cc/nvr89y">fun activities</a> (and travelling to even more hackathons)</li><li>Learning how to get “the winning idea”, which is an incredibly relevant skill in the startup world</li><li>Team building and learning how to be efficient in short time and under pressure</li></ul><p>Hackathons that members of MoonVision team have attended include: NightCrawl Big Data Hackathon, Palfinger Hackathon, Hackdays BE5, Zeiss Hackathon, and others. Some of them brought valuable prizes, others resulted in new connections, and it has always been a lot of fun.</p><p>What would we recommend? Attend hackathons, host hackathons, do it for the projects you always wanted to accomplish (or just for the pizza).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D5ePc3zOIUIjIOOBuGRbpA.gif" /><figcaption>source: giphy</figcaption></figure><p>If you are already eager to search for hackathons, you can start with these webpages:</p><ul><li><a href="https://www.hackathon.com/">https://www.hackathon.com/</a></li><li><a href="https://www.hackathon.io/">https://www.hackathon.io/</a></li><li><a href="https://hackevents.co/">https://hackevents.co/</a></li><li><a href="https://mlh.io/seasons/eu-2019/events">https://mlh.io/seasons/eu-2019/events</a></li></ul><p>To see how our ideas developed into a real product, visit our <a href="https://www.moonvision.io/">webpage</a> and try the <a href="https://app.moonvision.io/">MoonVision Toolbox</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1101a5742dcd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/why-we-love-hackathons-1101a5742dcd">Why we love Hackathons</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep Learning Docker Images for Production]]></title>
            <link>https://medium.com/moonvision/moonbox-docker-images-c41cb7ad02c5?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/c41cb7ad02c5</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Jakob Klepp]]></dc:creator>
            <pubDate>Tue, 16 Jul 2019 10:09:04 GMT</pubDate>
            <atom:updated>2019-07-17T13:10:00.035Z</atom:updated>
            <content:encoded><![CDATA[<p>When a deep learning network has been trained the work of bringing it to production has just begun. Here at <a href="https://www.moonvision.io/">MoonVision</a> different projects have very different production environments. Some projects lead to cloud hosted, some are run in the data centers of our customers and others are run as appliances on embedded industrial computers.</p><p>What all these environments have in common? They run Docker!</p><p>But what about the differences? In some environments we have NVIDIA GPUs for CUDA hardware acceleration, in some environments we don’t.</p><h3>Why we re-engineered our container build pipeline</h3><p>As you might know, to get the most out of CUDA requires CUDA-specific versions of various tools and libraries. In the past, we solved that by maintaining two versions of most of our Dockerfiles, one with CUDA acceleration and one for CPU only execution.</p><p>Another issue is managing custom builds of the software tools and libraries we use. If you want to use the newest features of libraries like <a href="https://pytorch.org/">PyTorch</a> before they are released in new versions, you have to build them yourself. It was a mess to cover this problem, but with the new pipeline this issue was solved.</p><h3>What we changed</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z3RA4nN9EXHrQvblpTGqFw.png" /><figcaption>Solid lines are child image/base image relationships. Dotted lines are other dependencies. Boxes off the same color are built from the same Dockerfile.</figcaption></figure><p>Instead of writing one Dockerfile per image we switched to separating by concerns and injecting dependencies as build arguments. This reduces code duplication in Dockerfiles, improving maintainability but also allows us to extend and customize Docker images without having to modify existing Dockerfiles.</p><p>By storing build artifacts like Debian .deb packages or Python Wheel .whl packages in Docker images we can use those artifacts in future builds without the additional cost and complexity that comes from having to maintain package repositories. Instead, we just push our build artifacts to Docker Hub. To keep the image sizes of our custom-builds images small we utilize Docker multi-stage builds, copying just the artifacts into an otherwise empty image.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e1de7ddf0c4792f6aff519c5b0cbccaa/href">https://medium.com/media/e1de7ddf0c4792f6aff519c5b0cbccaa/href</a></iframe><h3>Easily run GenICam based machine vision apps</h3><p>For example, to run machine vision applications with modern deep learning frameworks, we extend the <em>python-base</em> image, add the PyTorch and <a href="https://ffmpeg.org/">FFmpeg</a> artifacts that were built in the <em>custom-builds</em> images and install the open source <a href="https://github.com/genicam/harvesters">GenICam Harvesters library</a>. The result is a Moonbox image (<em>moonvision/moonbox:basler-latest</em>) equipped with powerful capabilities to acquire, process and save image data streams from vendors like <a href="https://www.baslerweb.com/en/">Basler</a>.</p><p>So if you are using a Basler industrial camera you are ready to go.</p><p>If you want to use a GenICam compliant camera from a different vendor, you by extending the<em> moonvision/moonbox:genicam-latest </em>Docker<em> </em>image yourself.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5f864db7812191f25ec63d81c97c748c/href">https://medium.com/media/5f864db7812191f25ec63d81c97c748c/href</a></iframe><p>To see if the library was installed correctly and the connected camera is recognized you can query the cameras available to Harvesters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2c243ef7f843957b523619b09ea80802/href">https://medium.com/media/2c243ef7f843957b523619b09ea80802/href</a></iframe><h3>Try it out yourself</h3><p>If you want to use Moonbox docker images you can pull them here: <a href="https://hub.docker.com/r/moonvision/moonbox">https://hub.docker.com/r/moonvision/moonbox</a></p><p>If you want to find out how they are made or built them yourself you can checkout the source here: <a href="https://github.com/MoonVision/moonbox-docker">https://github.com/moonvision/moonbox-docker</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c41cb7ad02c5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/moonbox-docker-images-c41cb7ad02c5">Deep Learning Docker Images for Production</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transfer Learning: Massive training data reduction with pre-training]]></title>
            <link>https://medium.com/moonvision/massive-training-data-reduction-with-pre-training-338f3a17a554?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/338f3a17a554</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[transfer-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Carlos Uziel Perez Malla]]></dc:creator>
            <pubDate>Wed, 22 May 2019 12:57:39 GMT</pubDate>
            <atom:updated>2019-05-23T19:39:59.923Z</atom:updated>
            <content:encoded><![CDATA[<p>The main concern of companies that want to incorporate into their business any feature or product that makes use of machine learning algorithms is the need of big amounts of labelled data to train such algorithms. For this reason, one of the main goals of MoonVision is to achieve high data efficiency by solving the clients’ needs with just a fraction of the data our competitors ask for. For example, in detection problems, we have successfully delivered projects with just 10 images per object category. One important technique to achieve such data efficiency is the use of <a href="https://towardsdatascience.com/transfer-learning-from-pre-trained-models-f2393f124751">pre-trained models and transfer learning</a>.</p><p>When approaching a project, we first do research about what data sets are available that could contain object instances related to the problem we want to solve, and then use them to pre-train a model. For example, if our project entails detecting different types of dishes, we will look for a data set that includes multiple food categories.</p><p>Once we have pre-trained our model and are happy with the results and the choice of the model architecture, we use <a href="https://en.wikipedia.org/wiki/Transfer_learning">transfer learning</a> (also known as fine-tuning) with the client’s data to achieve much better results on the objective task than if we would have trained our model from scratch. In essence, transfer learning allows us to take advantage of the similarities between the client’s data objects and the ones in the data used for pre-training. Then, since the model has already seen similar object instances to the ones we are interested in, it only needs to adjust itself during the new training to accommodate the new object instances in the client’s data.</p><p>Even when using a pre-trained model, we still have a lot of flexibility regarding which layers of the pre-trained model we want to keep and any additional pre-processing that we want to apply to the client’s data.</p><h3>So, what are the advantages of following this approach?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TXb_t_Ywesrr_HTjWiwzTA.png" /><figcaption>The three potential advantages of using transfer learning.</figcaption></figure><p>For the client, this means less effort from their side and faster delivery of their orders. Time savings start at the <a href="https://www.youtube.com/watch?v=Jaggdp8Kd5s">labelling data generation</a> and continue with the creation of the model itself.</p><p>But not all is about quick delivery, we use some of this saved time to improve results. Having a pre-trained model on the right data means that we already start with high accuracy, and our job is to improve it further, always aiming at perfect results.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=338f3a17a554" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/massive-training-data-reduction-with-pre-training-338f3a17a554">Transfer Learning: Massive training data reduction with pre-training</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Django with Dask for Task Processing]]></title>
            <link>https://medium.com/moonvision/using-django-with-dask-for-task-processing-73f4b903bf64?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/73f4b903bf64</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[dask]]></category>
            <category><![CDATA[django]]></category>
            <category><![CDATA[django-rest-framework]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Matt Nicolls]]></dc:creator>
            <pubDate>Thu, 16 May 2019 12:58:38 GMT</pubDate>
            <atom:updated>2020-06-24T13:35:19.924Z</atom:updated>
            <content:encoded><![CDATA[<p>Full Source Code: <a href="https://github.com/MoonVision/django-dask-demo">https://github.com/MoonVision/django-dask-demo</a></p><h3>Overview</h3><p>Web servers require asynchronous task execution for longer running computations. A common solution to this problem is task queues like Celery. As it has become easier to be able to store more data in recent years, data scientists are needing more and more computation power to process all their data. Dask was introduced to help them with this. From the docs, “Dask is a flexible library for parallel computing in Python.” Let’s look at how we can use Dask behind our Django Rest Framework web server to process asynchronous tasks.</p><h3>Dask Setup</h3><p>Dask can run on a single machine or a distributed system with thousands of cores. To use in a cluster, it uses a scheduler to coordinate worker nodes in executing your task. This <a href="https://distributed.dask.org/en/latest/quickstart.html#setup-dask-distributed-the-hard-way">setup</a> allows you to supply a preload script for your worker that will be run before the worker starts. We will use this script to setup the django environment on our workers. At the time of writing you install Dask with the “dask” pip package and the distributed system parts of Dask with the “distributed” pip package. See the <a href="https://docs.dask.org/en/latest/">official docs</a> for current info.</p><h3>Setting Up the Cluster and Connecting to the Scheduler</h3><p>With Dask installed, we now have the <em>dask-scheduler</em> and <em>dask-worker</em> commands. To startup our cluster we will first open a terminal and start a scheduler with <em>dask-scheduler</em> and then in another terminal we can start a worker with:</p><pre>dask-worker 127.0.0.1:8786 --preload daskworkerinit.py</pre><p>We are running the worker command from the /demo/ folder of the repository so that the <em>preload daskworkerinit.py</em> path works and <em>127.0.0.1:8786</em> is the location of our local scheduler.</p><p>To connect to the scheduler in our django app, we will use a class called <em>DaskManager</em> (/demo/daskmanager/daskmanager.py) to keep all of our connection information in one place. The class uses the singleton design pattern so we have just one connection per server. You will also notice this class has a callback function that is called when a task finishes. Our client needs to be kept alive until our task finishes for us to get this callback.</p><h3>Tracking Tasks and Getting the Result</h3><p>Before we create and submit our first Dask task, we need to have a way to see the result after it is finished. We use the DaskTask (/demo/daskmanager/models.py) model to store and track every task we create and submit to our cluster. We also setup the view and serializer for this model so we can view the results through our API. We create the DaskTask object in the DaskManager after submitting to the cluster. Dask returns a future with a unique key that we can use to lookup the status at a later time. For viewing tasks in real time, Dask also has a web view into the scheduler using Bokeh and runs on port 8787 by default.</p><h3>Submitting a Task to the Scheduler</h3><p>Now that we can see the results of our tasks, let’s look at a small demo task to run on the cluster. The <em>numbersexample</em> app will be our demo. Here we have one model called Number where we just store an integer value. In the view, we have one list route where we will build our Dask graph and then submit to the cluster. In our sample graph we will get all Numbers from the database, square them, and then take the sum of the squares.</p><p>First thing is to create numbers in shell:</p><pre>from numbersexample.models import Number</pre><pre>[Number.objects.create(value=n) for n in range(500)]</pre><p>Then we start a local server with <em>python manage.py runserver</em> and visit the endpoint <em>http://localhost:8000/numbers/square_sum/</em>. Our response looks something like:</p><pre>{</pre><pre>  “url”: “http://localhost:8000/tasks/sum-12f79623-0dd4-412f-b6de-f7d3a2f426ab/&quot;,</pre><pre>  “task_key”: “sum-12f79623–0dd4–412f-b6de-f7d3a2f426ab”,</pre><pre>  “status”: “pending”</pre><pre>}</pre><p>Dask always seems to return pending even when tasks are running unfortunately, but we can see that we now have a url for the task we created and can check the status by going to it. The task should be finished as isn’t something that should take long and will look like:</p><pre>{</pre><pre>  “url”: “http://localhost:8000/tasks/sum-12f79623-0dd4-412f-b6de-f7d3a2f426ab/&quot;,</pre><pre>  “task_key”: “sum-12f79623–0dd4–412f-b6de-f7d3a2f426ab”,</pre><pre>  “status”: “finished”,</pre><pre>  “result”: 41541750</pre><pre>}</pre><p>We can see the result of our task has been populated and saved.</p><h3>Real World Graph</h3><p>At Moonvision, we do image processing. Let’s look at how we make sprites on the platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*kzodwdlWXZ4ELYxP_Nnu9g.png" /></figure><p>Our graph look for building this image looks like:</p><pre>cropped_images = []</pre><pre>for entity_id in entity_ids:</pre><pre>  image_result = delayed(crop_image_to_entity, pure=True)(entity_id)</pre><pre>  cropped_images.append(image_result)</pre><pre>sprite_image = delayed(images_to_sprite)(cropped_images)</pre><pre>saved_image = delayed(cv2_to_image_bytes)(sprite_image)</pre><pre>return saved_image</pre><p>This real world example follows the same layout as the sample one where we first do computation on a list of things and join the result. For this one, our input is a list of IDs from our database and the output is a base64 byte string. Dask makes this joining and waiting on results easy.</p><h3>Final Thoughts and Improvements</h3><p>As mentioned, tasks always have the status of pending until the task is finished. It would be nice if Dask returned a proper status and more detailed information. Also, client connection needs to be held open while task is running. Dask also has a fire and forget function to submit to the cluster but we don’t have a way to save the result then. Dask provides a way to write scheduler plugins to have access to tasks as they finish but the scheduler is a bottleneck of our distributed system so running save tasks would not be advisable. In the future, we would like to forward the results to an AMQP, server such as rabbitMQ, to save results and save task updates. We could use these updates to correctly save the status of the task and have more information of how far along it is.</p><p>Overall, Dask allows us to write modular computer vision nodes that can be composed into processing graphs. We haven’t encountered any major roadblocks in this approach and plan on continuing it in the near future. We take advantage of some of the more advanced features of Dask in our cluster such as:</p><ul><li>Using <a href="https://distributed.dask.org/en/latest/resources.html">resources</a> on our tasks to have GPU workers in our cluster and dedicated real time workers for</li><li><a href="https://distributed.dask.org/en/latest/api.html#distributed.Client.replicate">Replicating</a> models into distributed memory for faster real time processing of tasks that require model loading</li><li><a href="https://distributed.dask.org/en/latest/task-launch.html">Spawning tasks</a> within tasks</li></ul><p>Checkout <a href="https://app.moonvision.io/">https://app.moonvision.io</a> to see it in action!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73f4b903bf64" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/using-django-with-dask-for-task-processing-73f4b903bf64">Using Django with Dask for Task Processing</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How modularity makes everything better]]></title>
            <link>https://medium.com/moonvision/how-modularity-makes-everything-better-56e3277c149f?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/56e3277c149f</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Carlos Uziel Perez Malla]]></dc:creator>
            <pubDate>Tue, 07 May 2019 12:29:44 GMT</pubDate>
            <atom:updated>2019-05-07T16:56:02.630Z</atom:updated>
            <content:encoded><![CDATA[<p>If you have any experience with software development, you must have encountered in your journey the term <a href="https://en.wikipedia.org/wiki/Modular_programming">modular programming</a>. In essence, this involves partitioning a program in multiple modules with simple responsibilities (i.e. see <a href="https://en.wikipedia.org/wiki/SOLID">SOLID</a>) that interact with each other and can be interchanged, thus avoiding having a monolithic design. The main benefits of modularity are reusability of modules and easier maintainability.</p><p>This time, I will talk about a real project where having a modular system saved substantial time and resources at MoonVision.</p><h3>New feature? Just add a new component</h3><p>All of our computer vision pipelines are composed of many simple steps that have to be taken to obtain the final result from the original image. Each one of these steps is a component (a module) in our <a href="https://en.wikipedia.org/wiki/Directed_acyclic_graph">execution graph</a>, starting with a data provider and ending by storing the results, which can be done in several ways. But we also have different kinds of graphs, depending on whether we want to train a model, perform inference on a dataset or just record a video with the pipeline results, among many others.</p><p>Chaining modules in our pipelines is made possible by using strong typing in Python. Knowing beforehand the types each module has to handle guarantees correct processing of inputs and returning the right outputs, making interchangeability of modules a trivial thing.</p><p>For one of our recent projects, we needed to detect objects of very different sizes. As a reference, consider that the smallest object was approximately 20 times smaller than the largest object. For this reason, we decided that in order to maximize efficiency and accuracy, it was best to use multiple detection models instead of a single one. This way, we could train a detector for larger objects on scaled images by a very small factor and then train another detector for the smallest objects with bigger resolution images. Such a design had considerable speedup gains in inference times, since in a production environment inference is usually done in small-factor computers that rely solely on CPU power.</p><p>Using multiple detectors means that the results of each detector must eventually be merged into a single result. Additionally, scaling at various points also increases complexity. By adding simple components that would take care of this, we can easily make complex graphs where all components interact seamlessly with each other.</p><p>Feel free to take a look at Figure 1 to understand all of the above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/762/0*MOr0yUdfNZgsSILh" /></figure><p><em>Figure 1: Simplified inference graph for one of our latest projects. The modules highlighted in orange were the new additions that made this project a success in a very short period of time.</em></p><p>Another side-effect of representing traditional and deep learning inference in a single symbolic graph is serialization. We can share an entire application between devices running on a single camera stream and a compute cluster with auto-scaling.</p><h3>Conclusion</h3><p>Modularity guarantees major flexibility improvements in the system and makes changes on the go much easier to implement. This translated into a much faster deployment of new features and completion of clients’ projects in a timely manner.</p><p><em>Check out what we do at </em><a href="https://www.moonvision.io/"><em>https://www.moonvision.io/</em></a><em> and check our platform at </em><a href="http://app.moonvision.io/signup"><em>app.moonvision.io/signup</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56e3277c149f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/how-modularity-makes-everything-better-56e3277c149f">How modularity makes everything better</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Succeed at your AI project]]></title>
            <link>https://medium.com/moonvision/succeed-at-your-ai-project-323854d2f066?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/323854d2f066</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Tatiana Statsenko]]></dc:creator>
            <pubDate>Tue, 30 Apr 2019 09:06:07 GMT</pubDate>
            <atom:updated>2019-04-30T09:06:07.369Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kwPbbW3uEcnX8BMtnlVymw.jpeg" /><figcaption><a href="https://commons.wikimedia.org/wiki/File:Rocket.jpg">https://commons.wikimedia.org/wiki/File:Rocket.jpg</a></figcaption></figure><p>Nowadays all is about speed and efficiency — fast search, fast payments, fast commuting. The field of computer vision is catching up with the power of machine learning: take a photo of text to <a href="https://www.nature.com/news/deep-learning-boosts-google-translate-tool-1.20696">translate</a> it, point at a flower to find out its <a href="https://www.picturethisai.com/">species</a>, detect a tumor from<a href="https://ai.googleblog.com/2018/10/applying-deep-learning-to-metastatic.html"> a single computer tomography scan</a>. Machine learning applied to visual recognition provides fast and reliable processing of images, and successful use cases inspire companies to implement it in their business or build entire products with deep learning algorithms.</p><h3>Everyone goes AI</h3><p>The desire to be in trend can end up being extreme, and using artificial intelligence in a workflow or a product, “no matter what”, can come to a point where one of the most impactful technologies is simply misused.</p><p>Visual recognition can be an unforgettable experience for users and add value to an end product, but digitalisation is a non-trivial task, and awareness of potential pitfalls is crucial for a successful project accomplishment. One has to understand, which benefits deep learning brings and what questions are going to be solved by this technology. The technology should be seamlessly integrated into the product, so that the user would not even notice its presence.</p><h3>Accuracy of machine learning algorithms</h3><p>Accuracy of some machine learning tasks is astonishing — object recognition, quality inspection, classification, outliers detection. However, recognition performance may fail and result in undesirable consequences, which is linked to the data used in the machine learning algorithms, real testing conditions and, what can be very likely, the whole scope of the application.</p><p>Imagine having a perfect detector for musical instruments. Point at your dog and get a result of a ”mandolin”. Did the system fail? Actually no, since its main function was to recognise between musical instruments and it was not aimed at anything else. Another example, using fixed light source to take images of objects and then realise that the lightning conditions are not corresponding to production environment thus giving wrong results all the time.</p><p>On the other hand, limitations of computer vision can boost research on a particular topic and result in a new technology that would bring better results to the current solution. One can always say: “It can not be done by deep learning.. Yet”, because technology development never stops.</p><h3>Want to apply deep learning right now?</h3><p>When considering improving your system with the help of deep learning in computer vision tasks, these are good questions to ask yourself:</p><ul><li>Why are the current solutions not serving our needs? (speed, ease of use, accuracy)</li><li>Which aspect of my process I want to improve with machine learning (reducing manual work, speed up operations, increasing recognition accuracy)</li><li>Do I have the data for training a machine learning model that will cover my use case?</li><li>How reliable the system should be and can I use other methods to support visual recognition?</li><li>Will I need to expand the solution in the near future?</li></ul><p>Knowing the answers beforehand will speed up the project planning and bring confidence for its successful completion.</p><p>At <a href="http://moonvision.io/">MoonVision</a>, we are constantly aiming for improvement of existing technology. The MoonVision Toolbox already helps users to perform labelling tasks on images and put life into projects in various fields of application — surface quality inspection, object detection and recognition. We think about the end product and are always excited to bring custom solutions to the market using a small amount of training data, hence reducing the time between project start and production launch.</p><p>Check out our <a href="https://app.moonvision.io/signup">MoonVision Toolbox</a> and try the free features we offer!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=323854d2f066" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/succeed-at-your-ai-project-323854d2f066">Succeed at your AI project</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ONNX: the long and collaborative road to machine learning portability]]></title>
            <link>https://medium.com/moonvision/onnx-the-long-and-collaborative-road-to-machine-learning-portability-a6416a96e870?source=rss----a341496eb4cf---4</link>
            <guid isPermaLink="false">https://medium.com/p/a6416a96e870</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Jakob Klepp]]></dc:creator>
            <pubDate>Wed, 17 Apr 2019 09:45:22 GMT</pubDate>
            <atom:updated>2019-04-17T09:45:22.587Z</atom:updated>
            <content:encoded><![CDATA[<p>Some time ago after training a model using deep learning frameworks you were locked into using it for inference as well. Considering that the inference environment often drastically differs from the training environment, this is likely not what you want. You feel the pain when you use various cloud platforms with GPU clusters for training and want to deploy the resulting models on low powered industrial PCs, on smartphones, or in the web. To do that using the same framework during inference as during training might not always be the best option. Thanks to ONNX (Open Neural Network Exchange) you no longer have to.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/0*Ecuxa5JPiV0jBJXj" /></figure><p>Though ONNX has only been around for a little more than a year it is already supported by most of the widely used deep learning tools and frameworks — made possible by a community that needed a standard for better collaboration.</p><h3>Motivation</h3><p>If you ask yourself why you should even care about the ability to use different ONNX backends, think about integration! Being flexible with the backend is key to integrating deep learning solutions into existing systems! The fact that ONNX models are supported by so many backends opens up a wide range of new deployment scenarios that were previously not feasible.</p><p>Suddenly sharing models is an option, especially when the model is implemented by another entity than the creator. And this is not a startup thinking — big companies are collectively supporting the standard developed by Microsoft. With ONNX Runtime, a ONNX backend developed by Microsoft, it’s now possible to use most of your existing models not only from C++ or Python but also in .NET applications. <a href="https://www.youtube.com/watch?v=lamnwHvjEV0">[1]</a></p><p>NVIDIA’s platform for high-performance deep learning inference, TensorRT, uses ONNX to support a wide range of deep learning frameworks. <a href="https://docs.nvidia.com/deeplearning/sdk/tensorrt-sample-support-guide/index.html#onnx_mnist_sample">[2]</a> TensorRT applies specific optimisation to allow for higher throughput and lower latency on their Tesla GPUs in data centers, as well as their Jetson embedded platforms for edge deployments.</p><p>Qualcomm’s Snapdragon NPE (Neural Processing Engine) SDK adds support for neural network evaluation to mobile devices. <a href="https://developer.qualcomm.com/blog/run-your-onnx-ai-models-faster-snapdragon">[3]</a> It can be used to implement inference directly into Android apps without having to rely on cloud services. It is optimized to perform well on Snapdragon chipsets. While only the Caffe, Caffe2 and TensorFlow model formats are directly supported by NPE a large number of deep learning frameworks are indirectly supported via the ONNX format.</p><h3>How to use it</h3><p>ONNX models are created from your existing deep learning models. The exact process varies from framework to framework but most of the popular deep learning frameworks either have a way to directly export the model in the ONNX format or there are converters which convert the framework specific models to the ONNX format.</p><p>Lets dive into the details of the ONNX format? An ONNX file is a Protobuf encoded tensor graph build from ONNX operations. The ONNX operations are versioned as operation sets, short opsets. <a href="https://github.com/onnx/onnx/blob/master/docs/Versioning.md">[4]</a> Each runtime has to specify which opsets it supports and then support those opsets completely. This improves compatibility across implementations.</p><p>The operations include most of the typical deep learning primitives, linear operations, convolutions and activation functions. The framework specific model is mapped to the ONNX format by executing the model with, often just random, input data and tracing the execution. The operations executed are mapped to ONNX operations and so the entire model graph is mapped into the ONNX format. After this the ONNX model is then saved as .onnx Protobuf file which can be read and executed by a wide and growing range of ONNX runtimes.</p><h3>How we use it — Initial Success</h3><p>Back in January we were already successfully using PyTorch, ONNX and ONNX Runtime together in a production environment — but we still moving on tough terrain. Pytorch was releasing many breaking changes and got rid of plenty of legacy code.</p><p>I’m not really sure how universally useful this is going to be. These commits are from a time where pytorch was introducing a lot of features with breaking changes but also removing a lot of legacy code. I think the pytorch commit that worked for use was one of those two.</p><ul><li><strong>PyTorch<br></strong>Last build on 2018–10–27: <a href="https://github.com/pytorch/pytorch/commit/4d62eef505770f1c866fa2205a5a05ab47865dda">Github [Commit 4d62eef5],</a> <a href="https://ci.pytorch.org/jenkins/job/pytorch-master/4824/">Jenkins [Build 4824]</a><br>Only build on 2018–10–28: <a href="https://github.com/pytorch/pytorch/commit/1fe8278559f466fc368f5ab03fc77b5267fa0494">Github [Commit 1fe82785],</a> <a href="https://ci.pytorch.org/jenkins/job/pytorch-master/4825/">Jenkins [Build 4825]</a></li><li><strong>ONNX</strong><em><br></em>We used this ONNX commit: <a href="https://github.com/onnx/onnx/commit/2a857ac039d9ff0a1ba2579afbb2f54fc603c6d0">Github [Commit 2a857ac0]</a></li><li><strong>ONNX Runtime</strong><em><br></em>And we used ONNX runtime onnxruntime==0.1.3 installed via pip. Hard to tell which commit because there are no tags from back then maybe <a href="https://github.com/Microsoft/onnxruntime/commit/d8fb92b62a1baa319ed2b69eeae90dd336f27f43">Github [Commit </a><a href="https://github.com/Microsoft/onnxruntime/commit/d8fb92b62a1baa319ed2b69eeae90dd336f27f43">d8fb92b6</a><a href="https://github.com/Microsoft/onnxruntime/commit/d8fb92b62a1baa319ed2b69eeae90dd336f27f43">]</a>.</li></ul><p>In order to find the most feasible combination of libraries, we developed tests to measure the numerical and runtime difference for several important models. The final logits deviated quite strongly from the PyTorch output for the ONNX Runtime. Surprisingly, this translated to rather low deviations in the qualitative output (bounding boxes or pixel-wise segmentations). We still have to figure out if those numerical differences are largely linear such that final argmax and softmax operations that we do in numpy don’t affect the final outcome.</p><p>For a glimpse into our tests, we listed the runtime and qualitative differences for a segmentation model between Pytorch GPU, Pytorch CPU, Caffe2 CPU and ONNX Runtime CPU in their aforementioned versions (8 hyper-threads, GTX 1080Ti respectively).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/0*SDiH_lBZvp3VoSwQ" /><figcaption>Caffe2 native on CPU, no bilinear Upsampling, Detection time: 2.45s.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/0*kfQuRVLpNvsbsngL" /><figcaption>Caffe2 on CPU, no bilinear Upsampling, Detection time: 2.57s.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/0*tACABgZ4z-xXci7W" /><figcaption>ONNX Runtime on CPU, with bilinear Upsampling, Detection time: 0.761s.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/0*4OlkpA4gHpgLNUn0" /><figcaption>PyTorch on GPU, with bilinear Upsampling, Detection time: 0.038s.</figcaption></figure><h3>How we rapidly will use it to our advantage</h3><p>All this is good and well but there are still a couple of issues hindering us from using ONNX in our docker edge deployment cycle. Opsets are changing fast, while this means a lot of improvements, it also means that what worked a month ago, might not work in the present any more. Among the many ONNX backends, few support the current opset version 9, let alone the upcoming version 10. And even when the new opset versions are supported, it takes a while until they make it into official releases of the various inference frameworks. This means that a lot of time is spent on trying to compile, mix and match master branches of various deep learning tools, libraries and their dependencies → this is basically my job description here at <a href="https://www.moonvision.io/">MoonVision</a>.</p><p>In particular we had issues with the Upsample operator. Based on our investigation and involvement we think that the specification for the Upsample operator was perhaps unclear and was misinterpreted by various parties which lead to a number of conflicts. Since it seems that we aren’t the only ones having issues with the Upsample operator we compiled an easy follow version history as well as a collection of relevant issues.</p><h3><strong>Upsample Changelog</strong></h3><h4>Add an upsample2d operator as experimental</h4><p>PR created on: 2017–11–02<br>PR merged on: 2017–11–15<br>Opset: 1<br>URL: <a href="https://github.com/onnx/onnx/pull/180">github.com/onnx/onnx/pull/180</a></p><h4>Change ResizeNearest op to Upsample op</h4><p>PR created on: 2017–11–18<br>PR merged on: 2017–11–27<br>Opset: 1–6<br>URL: <a href="https://github.com/onnx/onnx/pull/290">github.com/onnx/onnx/pull/290</a></p><h4>Promote Upsample op</h4><p>PR created on: 2018–05–01<br>PR merged on: 2018–05–10<br>Opset: 7–8<br>URL: <a href="https://github.com/onnx/onnx/pull/861">github.com/onnx/onnx/pull/861</a></p><h4>Change upsample operator to allow dynamic ‘scales’</h4><p>PR created on: 2018–10–02<br>PR merged on: 2018–10–26<br>Opset: 9<br>URL: <a href="https://github.com/onnx/onnx/pull/1467">github.com/onnx/onnx/pull/1467</a></p><h4>Support down sampling for Upsample with scales &lt; 1</h4><p>PR created on: 2019–01–25<br>PR merged on: 2019–03–20<br>Opset: 10<br>URL: <a href="https://github.com/onnx/onnx/pull/1773">github.com/onnx/onnx/pull/1773</a></p><h3>Related Issues</h3><h4>Add linear mode case for upsample operator</h4><p>Issue created on:<br>Used opset:<br>URL: <a href="https://github.com/onnx/onnx/pull/1295#discussion_r228605032">github.com/onnx/onnx/pull/1295#discussion_r228605032</a></p><h4>Upsample ONNX op uses wrong mode</h4><p>Issue created on:<br>Used opset:<br>URL:<a href="https://github.com/pytorch/pytorch/issues/12647">github.com/pytorch/pytorch/issues/12647</a></p><h4>Modify the ONNX Upsample mode “linear” -&gt; “bilinear”</h4><p>Issue created on:<br>Used opset:<br>URL: <a href="https://github.com/pytorch/pytorch/pull/13382">github.com/pytorch/pytorch/pull/13382</a></p><h4>Upsample node does not support align_corners attribute</h4><p>Issue created on:<br>Used opset:<br>URL: <a href="https://github.com/onnx/onnx/issues/1612">github.com/onnx/onnx/issues/1612</a></p><h4>Upsample problem</h4><p>Issue created on:<br>Used opset:<br>URL: <a href="https://github.com/onnx/onnx/issues/1807">github.com/onnx/onnx/issues/1807</a></p><h4>Error converting upsample operator from pytorch to tensorflow</h4><p>Issue created on:<br>Used opset:<br>URL: <a href="https://github.com/onnx/onnx/issues/1827">github.com/onnx/onnx/issues/1827</a></p><h4>Upsample only checks its “scales” input from float_data field</h4><p>Issue created on:<br>Used opset:<br>URL: <a href="https://github.com/onnx/onnx/issues/1852">github.com/onnx/onnx/issues/1852</a></p><h4>PyTorch export of Upsample</h4><p>Issue created on:<br>Used opset:<br>URL: <a href="https://github.com/onnx/onnx/issues/1918">github.com/onnx/onnx/issues/1918</a></p><h3>Further Reading</h3><p>[1]: Learning Machine Learning with .NET, PyTorch and the ONNX Runtime, Microsoft Developer, <a href="https://www.youtube.com/watch?v=lamnwHvjEV0">https://www.youtube.com/watch?v=lamnwHvjEV0</a></p><p>[2]: NVIDIA TensorRT — Integrated with All Major Frameworks, NVIDIA Developer, <a href="https://docs.nvidia.com/deeplearning/sdk/tensorrt-sample-support-guide/index.html#onnx_mnist_sample">https://docs.nvidia.com/deeplearning/sdk/tensorrt-sample-support-guide/index.html#onnx_mnist_sample</a></p><p>[3]: Run your ONNX AI Models Faster on Snapdragon, Qualcomm Developer Network, <a href="https://developer.qualcomm.com/blog/run-your-onnx-ai-models-faster-snapdragon">https://developer.qualcomm.com/blog/run-your-onnx-ai-models-faster-snapdragon</a></p><p>[4]: ONNX versioning, onnx, <a href="https://github.com/onnx/onnx/blob/master/docs/Versioning.md">https://github.com/onnx/onnx/blob/master/docs/Versioning.md</a></p><p>This article will be updated in the way with sharing new findings along that long and (still) lonesome road. Give us a comment if you have similar struggles or share your experience with the new format.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6416a96e870" width="1" height="1" alt=""><hr><p><a href="https://medium.com/moonvision/onnx-the-long-and-collaborative-road-to-machine-learning-portability-a6416a96e870">ONNX: the long and collaborative road to machine learning portability</a> was originally published in <a href="https://medium.com/moonvision">Moonvision</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>