<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by AI4kids on Medium]]></title>
        <description><![CDATA[Stories by AI4kids on Medium]]></description>
        <link>https://medium.com/@service_20673?source=rss-ef68a2e569ea------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*SVAcX0mp-KI28CsQ64fMRQ.png</url>
            <title>Stories by AI4kids on Medium</title>
            <link>https://medium.com/@service_20673?source=rss-ef68a2e569ea------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 05:57:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@service_20673/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[一文讀懂在 SIGGRAPH 上技驚四座的 Nvidia Omniverse： 以 Metaverse、AI 引領各領域工作流程之創新]]></title>
            <link>https://medium.com/@service_20673/%E4%B8%80%E6%96%87%E8%AE%80%E6%87%82%E5%9C%A8-siggraph-%E4%B8%8A%E6%8A%80%E9%A9%9A%E5%9B%9B%E5%BA%A7%E7%9A%84-nvidia-omniverse-%E4%BB%A5-metaverse-ai-%E5%BC%95%E9%A0%98%E5%90%84%E9%A0%98%E5%9F%9F%E5%B7%A5%E4%BD%9C%E6%B5%81%E7%A8%8B%E4%B9%8B%E5%89%B5%E6%96%B0-c06f31764c62?source=rss-ef68a2e569ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/c06f31764c62</guid>
            <dc:creator><![CDATA[AI4kids]]></dc:creator>
            <pubDate>Thu, 26 Aug 2021 06:02:49 GMT</pubDate>
            <atom:updated>2022-02-10T07:14:32.289Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*TIkJy3RoxmPX02L5.png" /><figcaption>遊戲變局者 - Nvidia Omniverse</figcaption></figure><h3>前言</h3><p>Nvidia 在最近頻頻出現在各類媒體、新聞上，於全球掀起一片熱烈討論。原因是藉由自家的開源軟體 Omniverse ，於最具盛名的計算機圖學年度會議 SIGGRAPH 2021 以讓人瞠目結舌的成果獲得最佳展示獎。</p><p>本文將帶領讀者一起看看 Nvidia 獲獎的三個案例成果，Omniverse 整合「虛擬環境模擬」、「AI」 跨足不同領域、帶動流程創新。</p><h3>Metaverse</h3><blockquote>The Metaverse is “a virtual environment where you can be present with people in digital spaces” - Zuckerberg</blockquote><p>還記得電影「一級玩家」當中，主角徜徉的虛擬世界嗎？人類建造了一個全虛擬的世界，在這個世界裡，除了能將真實世界當中的一切重現以外，現實當中不存在的事物也可以被實現；創造者有如造物主一般擁有此世界的所有掌控。</p><p>「Metaverse」做為未來科技發展重點之一，將會大幅改變現行的社會型態。本文將簡介由 Nvidia 開源的軟體 Omniverse，全球首個以 Metaverse 概念建立的模擬平台，如何讓用戶配合 AI 工具形塑、模擬自己設計的世界、影響各領域的發展。</p><h3>Omniverse 簡介</h3><p><a href="https://urldefense.com/v3/__https:/www.nvidia.com/en-us/omniverse/__;!!FMox2LFwyA!4YgG_SWJg4OnUgoggY6eaSYO2YhkeHOJ-ydfV8NiaPmt2orj5VenJDl-sqVcyKiAI5IATEA$">NVIDIA Omniverse</a> 是一個基於 CUDA、RTX 光線追蹤技術的數位創作平台，用戶除了可以輕易地建造栩栩如生的數位 3D 模型、以真實世界中的物理現象進行模擬以外，得利於 CUDA 的強大平行運算能力，所有呈現都幾乎是即時的。</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FxC6cho2VL6c%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DxC6cho2VL6c&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FxC6cho2VL6c%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/79a41b7bdf7abf17c48208cd34ff37fa/href">https://medium.com/media/79a41b7bdf7abf17c48208cd34ff37fa/href</a></iframe><p>除了彩現、模擬之外，Omniverse 亦整合進許多實用的 「AI 工具」用以加速數位創作的過程；這個創舉無疑是其獲得最佳展示獎的原因，我們將在文章當中了解這些 AI 工具。</p><h3>Demo-1 維妙維肖的數位替身</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*Q1DpC7M11oBt-DQ_.gif" /><figcaption>GTC 2021 大會上的黃仁勳是 3D 虛擬人物</figcaption></figure><p>讓 Nvidia 在 SIGGRAPH 2021 上備受矚目的原因，是因為公布了 CEO 黃仁勳在 GTC 2021 時演講片段的幕後花絮；其中一段是由極其真實的數位替身代替真人進行演說。</p><p>在 GTC 2021 會議當下，幾乎沒人有人看出眼前的黃仁勳居然是虛擬的 3D 人物，因此花絮一經公開馬上攻佔了網路、媒體的版面，也藉此拿下了大會最佳展示獎。</p><h3>專案技術流程拆解</h3><p>此專案動用了高達 34 位的cg 藝術家、12 位研究人員進行開發，製作規模浩大。專案大部分採用 3D 動畫業界標準流程，3D 建模、骨架綁定、動畫製作、彩現..等等，搭配 Omniverse 的 RTX 光線追蹤技術，彩現效果非常驚人。</p><h4>數位替身模型建立</h4><p>為了製作 CEO 黃仁勳 的數位替身，使用的技術為「Photogrammetry」，透過大量的攝影機群從不同角度拍攝黃仁勳本人，得以用電腦視覺的演算法重建出他的 3D 形體。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IhbVGQ1n2CHLZdRUsiYaIQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k0Il55s3CiNFXydFmXoedQ.png" /></figure><p>在模型建立完畢以後，依照 3D 行業的傳統流程，得為 3D 人物建立控制項，用於驅動人物動作與身體外觀之間的形體變化。控制項通常分為身體、臉部，過程需要經驗豐富的 3D 藝術家花費大量的時間進行人工設置。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*QHOui9r4sxoIgvmv.gif" /><figcaption>傳統方法：透過手動設置人臉控制項，調整3D角色臉部表情</figcaption></figure><h3>AI 翻轉影像創作流程</h3><h4>Audio2face — 語音驅動的數位人物表情生成</h4><p>在數位替身臉部控制項設定與動畫的流程，Omniverse 顛覆了傳統人工作法，以名稱為「audio2face」的內建 AI 工具替代；輸入一段使用者輸入的語音，可以立即為 3D 數位人物產生臉部表情以及臉部肌肉活動的樣貌。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*krimeWY_PIz60Y0af7A38w.gif" /><figcaption>使用 語音驅動的 AI 工具自動產生的人臉表情與臉部肌肉變化</figcaption></figure><h4>Face-vid2vid — 生成照片等級的人臉動畫</h4><p>由 audio2face 產生的臉部表情非常細緻，但是缺乏真實感，因此 artist 與 Nvidia AI team 進行合作，使用深度學習的生成模型「face-vid2vid」將原本不真實的臉部動畫轉變成栩栩如生的黃仁勳本人。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*REz1ZNFxjYdBDva1OLuHOQ.gif" /><figcaption>face-vid2vid 臉部風格轉換</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oLsWB_AUQYt71iYrnpMJow.gif" /><figcaption>以 face-vid2vid 產生之結果進行臉部合成、替換</figcaption></figure><p>至此 Omniverse 的 AI 強大的功能證明可以取代傳統動畫的繁瑣流程。而數位人物身體動畫則是依照傳統做法，以動作捕捉的方式進行人物動作的錄製、讓黃仁勳的數位替身演員做出一樣的動作。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SvZ78G8GOI-Xcrj5UMuqkg.gif" /><figcaption>捕捉演員之動作</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G2dFlMIFFOYkC7Amk7TcmQ.gif" /><figcaption>將動態擷取之結果呈現於 Omniverse</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7pgoRLGX1uipHEg-kR4VFA.gif" /><figcaption>讓數位替身演員做出與捕捉之人物相同之動態</figcaption></figure><p>最後，透過 Omniverse 當中的 RTX 光線追蹤技術，數位替身演員的演出呈現出極為真實的結果。 由 Face-vid2vid 生成的黃仁勳臉部動畫、Omniverse 產生的人物身體動畫彩現，最終在合成軟體當中結合，便產生了 GTC 2021 大會上的假 CEO 演說片段。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*C3UoYt3bBqxuZJQt.gif" /></figure><h3>結語</h3><p>Nvidia 此次的展示，是替影像創作頁敲響了第一記變革的鐘聲，證明 AI 與科技的創新作法，在影像創作領域已經可以代換已經相當成熟的標準流程；甚至於程度上更有效率。</p><h3>下期內容預告</h3><p>下一篇文章，我們將會解析 Omniverse 如何被應用在智慧工廠的產線模擬，以及在虛擬環境當中驗證 AI、機器人開發的必要性。</p><p>在團隊努力寫文章時，歡迎大家先看看其他相關資訊</p><ul><li><a href="https://ai4kids.ai/ai-camp/">高中寒暑期營隊-AI人工智慧專題實作營</a></li><li><a href="https://ai4kids.ai/program-language-resource/">自學程式語言該如何入門？3大資源就算沒背景也能輕鬆上手！</a></li><li><a href="https://ai4kids.ai/coding-resource-kid/">【兒童程式設計推薦】4款適合小學的學習資源，讓孩子愛上寫程式</a></li></ul><h3>訂閱我們</h3><p>我們是 AI4Kids，由 Nvidia Deep Learning Institute 認證的 AI 大使。作為大使，我們定期發布 AI 最新的工具、應用，讓讀者能夠時時接觸到最新、實用的內容。訂閱我們， 隨時與新知同行不掉隊！</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c06f31764c62" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Iguana detection and monitoring with Nvidia Jetson Nano]]></title>
            <link>https://medium.com/@service_20673/iguana-detection-and-monitoring-with-nvidia-jetson-nano-5c006882d3e0?source=rss-ef68a2e569ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c006882d3e0</guid>
            <dc:creator><![CDATA[AI4kids]]></dc:creator>
            <pubDate>Wed, 23 Jun 2021 17:20:55 GMT</pubDate>
            <atom:updated>2021-06-24T01:38:43.927Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pdqzzG038X2slD2W.jpg" /><figcaption>實作綠鬣蜥偵防儀表板</figcaption></figure><h3>1. Intro</h3><pre>This blog is an introduction to the open source project &quot;Green Iguana Detection and Monitoring&quot; in the Nvidia Jetson community contributed by us. </pre><pre>This project uses Jetson Nano with the object detection model of computer vision to detect and visualize green iguanas in real time.</pre><h3>2. Motivation</h3><h4>What are iguanas？</h4><pre>The American iguana, also known as the green iguana, is a large lizard that lives on trees. Its head and tail can reach 1 to 2 meters in length, and can live for more than 10 years. It is based on plant leaves, buds, flowers, and fruits. The food is daily-traveling reptiles, which can lay 24–45 eggs per litter, with strong fertility and environmental adaptability.</pre><h4>The severity of the ecological disaster brought by the green iguanas</h4><pre>In Taiwan, abandoned green iguanas reproduce rapidly in the wild. The Forestry Bureau of the National Council of Agriculture once stated that the problem of green iguanas not only harms the ecology, but also causes loss of crops by farmers due to the green iguana’s burrowing. It is a habit to dig holes beside river banks and fish farms to destroy facilities. As far as the government knows under current surveillance, the whereabouts of the green iguana have spread to many counties and cities.</pre><h4>A system that detect and monitor iguanas</h4><pre>Thanks to the development of AI and Edge device in recent years, we have been able to develop and deploy real-time computer vision applications to detect and monitor the whereabouts of the green iguana. Through the system real-time warning and notification, we can take measures to avoid asset losses caused by the green iguana. problem.</pre><h4>Software architecture</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fdy8rDn9eXCc%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Ddy8rDn9eXCc&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fdy8rDn9eXCc%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/edd7ea15bed4ec7f6436e3f51f7a7768/href">https://medium.com/media/edd7ea15bed4ec7f6436e3f51f7a7768/href</a></iframe><h3>3. Data collection</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Cr8BAPLCsJVihlIT.jpg" /><figcaption>綠鬣蜥圖片標籤示意</figcaption></figure><h4>Web image scraper</h4><pre>Through the selenium dynamic web crawler, we collected about 5,000 green iguana pictures and labeled them one by one. Here we also open the download of pictures and label information, please see the picture labeling tutorial.</pre><h4>Image labeling</h4><pre>We use the web-based image labeling tool <strong>makesense.ai</strong> to label iguana image we collect, because of its simple and easy-to-use feature, and no software is required to be installed.</pre><h3>4. Training iguana detection model with Nvidia TLT</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HWXNXrLISd5UIXGx.png" /><figcaption>Nvidia Transfer Learning Toolkit</figcaption></figure><h4>Nvidia Transfer Learning Toolkit (TLT) intro</h4><pre>Creating an AI/ML model from scratch to solve business problems is an extremely costly process, so &quot;transfer learning&quot; is usually used to accelerate the development of the project. Transfer learning is a research field of machine learning. It focuses on storing existing problem-solving models, such as using pre-trained models, and using them for other different but related problems.</pre><pre>The TLT (Transfer Learning Toolkit) provided by NVIDIA, as the name suggests, is a toolkit for transfer learning, providing popular pre-trained models for image and natural language processing for transfer learning. The most important thing about TLT is for deployment purposes; the size reduction and optimization of the model after training are all integrated into the TLT function.</pre><pre>In the most ideal case, using TLT for model training only needs to set the training parameters; there is no need to write code from scratch. TLT realizes the end-to-end process of training, optimization, and deployment, which is very different from the general development process and greatly reduces the time and technical cost of project development.</pre><h4>Hardware requirements</h4><pre>The official recommended hardware configuration for running Nvidia TLT:</pre><ul><li>32 GB system RAM</li><li>32 GB of GPU RAM</li><li>8 core CPU</li><li>1 NVIDIA GPU</li><li>100 GB of SSD space</li></ul><h4>Software requirements</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*GJSeZjFgi4rqnOhn8IdYOw.png" /></figure><h4>NVIDIA TLT installation tutorial</h4><pre>Here we provide a detailed NVIDIA TLT installation instruction, please see our tutorial.</pre><h4>TLT for model training and optimization</h4><pre>The object detection model for this project is yolo v4. Please refer to this yolo v4 training example. After a little modification, the model can be trained using the green iguana pictures and tags.<br>Run this code and wait for the program to be executed. An .etlt file will be automatically exported, which is the model file we will eventually deploy.。</pre><h3>5. Model inference on Jetson device</h3><h4>Deepstream intro</h4><pre>After the model is trained, the next step is to deploy it on the Jetson device for real-time inference. At the same time, the images and inference results need to be streamed. We are using Nvidia Deepstream, a framework designed for inference and data streaming.</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*o0GjdRSGm7i5EMwd.png" /></figure><pre>DeepStream is an end-to-end process that can perform deep learning inference, image and sensor processing, and send insights to the cloud in a streaming application. You can use containers to build cloud-native DeepStream applications, and perform coordination through the Kubernetes platform for large-scale deployment. When deployed at the edge, applications can communicate between IoT devices and cloud standard message brokers (such as Kafka and MQTT) for large-scale wide-area deployment.</pre><h4>TensorRT intro</h4><pre>NVIDIA TensorRT™ is an SDK for deep learning inference. The SDK contains the runtime environment of the deep learning learning optimizer, which enables deep learning learning applications to provide low latency and high throughput. Before deploying the model, TensorRT is required to convert the model to speed up the running speed of inference.</pre><pre>TLT is a deployment purpose-oriented model training framework, which naturally includes conversion functions for the trained model. As long as tlt-converter is used, TensorRT&#39;s inference engine can be built on the TLT-trained .etlt output model. In addition, the hardware and software deployed for the device need to be converted with the corresponding tlt-converter version.<br> <br><strong>The following are all configured options:</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/1*HYSrwz-SXtKdujEVN5SiJQ.png" /></figure><h4>Deepstream environment setup</h4><pre>We document the installation steps of the Deepstream environment and the process of building the TensorRT engine of the .etlt model. It also contains methods for installing other packages needed for this project, such as MQTT client... etc. For detailed installation steps, please refer to the environment setup tutorial we prepared.</pre><h4>Running Deepstream app</h4><pre>deepstream_app/deepstream_mqtt_rtsp_out.py is the running configuration of deepsteam python binding. When performing object detection inference, this program will stream the image in RTSP mode, and send the data of the inference result to the MQTT broker and store it in the database for later visualization.</pre><h3>6. Data monitoring</h3><h4>Plotly Dash</h4><pre>Plotly Dash is a framework for interactive visualization of data. A non-front-end language can be used to write a front-end page for data visualization. At the same time, other sub-packages extended by this package simplify the process of web page style and layout. Suitable for rapid development of web-based data visualization.</pre><pre>In addition, the live-update function provided by the kit allows web pages to be continuously updated dynamically, which is particularly suitable for the dashboard requirements of this project.</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*K0lrcUWfGIZ2WQE-.png" /></figure><h4>Dash Bootstrap Components</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/0*QkTs-0It6nTqB5EF.png" /></figure><pre>Bootstrap is a set of open source front-end frameworks for website and web application development. It includes HTML, CSS and JavaScript frameworks. It provides font typography, forms, buttons, navigation and various other components and Javascript extensions, designed to make dynamic web pages And web application development is easier.</pre><pre>Using Dash Bootstrap Components can directly inherit the framework defined by bootstrap, and set the layout of the webpage with quick keys.</pre><h4>MQTT broker</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/354/0*-eTGIHSoCnFFYtLE.png" /><figcaption>Eclipse Mosquitto</figcaption></figure><pre>Users can choose MQTT broker according to their own preferences. In this project, we use Eclipse Mosquitto. After installation, just start MQTT broker to start receiving data.</pre><h4>Database</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*hM2QZfX9iVRPGeU0.png" /><figcaption>mySQL</figcaption></figure><pre>This project uses mySQL to build a database, and users can choose their preferred database type. The purpose of the database is that the data stream sent to the MQTT broker from Jetson Nano inference can be written to the database, and the data will be updated regularly through Plotly Dash and the latest data will be displayed on the dashboard.</pre><h4>MQTT topic subscription and data saving</h4><pre>When Deepstream runs on Jetson Nano, it continuously sends data of inference results to specific topics of the MQTT broker through the MQTT communication protocol. By setting the subscribers of this specific topic, they can be notified when the latest data flows in a certain topic and write the data into the database. The location of this program is mqtt_topic_subscribe/mqtt_msg_to_db.py.</pre><h4>Run the web-based monitor</h4><ul><li>Deepstream up and running</li><li>MQTT broker up and running</li><li>Database up and running</li></ul><pre>With the above conditions, the web dashboard can start to run. Please note that the configuration of Deepstream is changeable, so if you want to replace the model with another object detection model in the future, you only need to modify the configuration file of Deepstream and the execution file of Deepstram python.</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c006882d3e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[用 Nvidia 軟硬體生態系 實現 Jetson Nano 物件偵測與數據串流儀表板]]></title>
            <link>https://medium.com/@service_20673/%E7%94%A8-nvidia-%E8%BB%9F%E7%A1%AC%E9%AB%94%E7%94%9F%E6%85%8B%E7%B3%BB-%E5%AF%A6%E7%8F%BE-jetson-nano-%E7%89%A9%E4%BB%B6%E5%81%B5%E6%B8%AC%E8%88%87%E6%95%B8%E6%93%9A%E4%B8%B2%E6%B5%81%E5%84%80%E8%A1%A8%E6%9D%BF-f1e4c587156d?source=rss-ef68a2e569ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/f1e4c587156d</guid>
            <dc:creator><![CDATA[AI4kids]]></dc:creator>
            <pubDate>Tue, 22 Jun 2021 11:30:24 GMT</pubDate>
            <atom:updated>2022-11-08T03:30:38.485Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pdqzzG038X2slD2W.jpg" /><figcaption>實作綠鬣蜥偵防儀表板</figcaption></figure><h4>0.整體系統架構圖</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*icZr5rQO5gG6VEn1rTFAgg.png" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fdy8rDn9eXCc%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Ddy8rDn9eXCc&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fdy8rDn9eXCc%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/43759cc6aa2cf710d070c6162ec6c11c/href">https://medium.com/media/43759cc6aa2cf710d070c6162ec6c11c/href</a></iframe><h3>目錄</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/1*0nYgsDj73YhIoNX-oXkucA.png" /></figure><h3>1. 專案大綱</h3><p>本頁面是 Nvidia Jetson 社群中的公益開源專案「綠鬣蜥個體偵測與監控」的 專案簡介。此專案使用 Jetson Nano 搭配電腦視覺之物件偵測模型即時偵測綠鬣蜥的行蹤與呈現。</p><p>我們同時提供此開源專案的 <a href="https://github.com/ai4kids-tw/Iguana-detection-on-Nvidia-Jetson-Nano-for-monitoring">Github repo</a>，讓讀者能夠在進行類似專案開發時複製模式進行快速開發。</p><h3>2. 專案動機</h3><h4>什麼是綠鬣蜥？</h4><p>美洲鬣蜥又名綠鬣蜥，是一種生活在樹上的大型蜥蜴，頭尾全長可達1～2公尺，壽命可長達10年以上，以植物的葉、嫩芽、花、果實為食物，為日行性爬蟲類，每胎可產24–45顆卵，繁殖力、環境適應能力極強。</p><h4>綠鬣蜥帶來之生態浩劫嚴重性</h4><p>在台灣，棄養的綠鬣蜥在快速野外繁殖，本國農委會林務局曾表示，綠鬣蜥問題不不僅危害生態，其攝食植物造成農民的農作物損失、並且由於綠鬣蜥有挖洞之習慣，常在河堤、魚塭旁挖洞，破壞設施。以政府目前監控下所知，綠鬣蜥的行蹤已擴散多個縣市。</p><h4>建立綠鬣蜥監控系統</h4><p>得力於近年來 Edge device 的發展，我們得以開發、佈署實時電腦視覺應用，藉以偵測、監控綠鬣蜥的行蹤，通過系統即時告警與通報，能採取措施避免綠鬣蜥帶來的資產損失問題。</p><h3>3. 資料蒐集</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Cr8BAPLCsJVihlIT.jpg" /><figcaption>綠鬣蜥圖片標籤示意</figcaption></figure><h4>圖片爬蟲</h4><p>透過 selenium 動態網路爬蟲，我們蒐集了5000張左右的綠鬣蜥圖片，並一一進行標籤。在此我們也開放圖片、標籤資料的下載，請見我們準備的<a href="https://github.com/ai4kids-tw/Iguana-detection-on-Nvidia-Jetson-Nano-for-monitoring/tree/main/3.image_labeling"><strong>圖片標籤教學</strong></a>。</p><h4>圖片標籤</h4><p>在多人標籤的作業當中，我們使用了<strong>網頁版的物件偵測標籤工具</strong> <a href="https://makesense.ai/"><strong>makesense.ai</strong></a> 來進行，因為其簡單易用的特性，也無須安裝任何軟體。</p><p>使用此工具的教學文檔見此<a href="https://hackmd.io/iq1tHBp8SfS6hAnj0ZKIEA">連結</a>。</p><h3>4. 使用 Nvidia TLT 訓練綠鬣蜥偵測模型</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HWXNXrLISd5UIXGx.png" /><figcaption>Nvidia Transfer Learning Toolkit</figcaption></figure><h4>Nvidia Transfer Learning Toolkit (TLT) 簡介</h4><p>從頭開始創建 AI/ML 模型來解決業務問題是一項成本極高的過程，因此通常專案開發時會使用「遷移學習」來加速開發。遷移學習是屬於機器學習的一種研究領域，它專注於存儲已有問題的解決模型，例如使用預訓練的模型，將其利用在其他不同但相關問題上。</p><p>NVIDIA 所提供的 TLT (Transfer Learning Toolkit) 顧名思義就是遷移學習的工具包，提供了圖像、自然語言處理的熱門預訓練模型做遷移學習。TLT 最重要的是面向佈署的目的；針對模型訓練完畢後的尺寸縮減、優化都被整合在 TLT 的功能當中。</p><p>在最理想的情況下使用 TLT 做模型的訓練僅需要設定訓練參數；不用從頭撰寫程式碼。TLT 實現了訓練、優化、佈署的端到端流程，這與一般的開發流程有極大的不同，大大減少了專案開發的時間、技術成本。</p><h4>硬體需求</h4><p>官方所建議的運行 Nvidia TLT 的硬體配置:</p><ul><li>32 GB system RAM</li><li>32 GB of GPU RAM</li><li>8 core CPU</li><li>1 NVIDIA GPU</li><li>100 GB of SSD space</li></ul><h4>軟體需求</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*GJSeZjFgi4rqnOhn8IdYOw.png" /></figure><h4>NVIDIA TLT 安裝教學</h4><p>在此我們提供了一份詳細的 NVIDIA TLT 安裝教學，請見我們準備的<a href="https://github.com/ai4kids-tw/Iguana-detection-on-Nvidia-Jetson-Nano-for-monitoring/tree/main/4.training_yolo_with_nvidia_tlt#readme"><strong>環境設置教學</strong></a>。</p><h4>TLT 模型訓練、優化</h4><p>此專案的物件偵測模型為 yolo v4，請參考此 <a href="https://github.com/ai4kids-tw/Iguana-detection-on-Nvidia-Jetson-Nano-for-monitoring/blob/main/4.training_yolo_with_nvidia_tlt/tlt_iguana_detector_training_yolo_v4_mobilenet_v2/yolo_v4.ipynb">yolo v4 訓練範例程式碼</a>稍作修改後，便能使用綠鬣蜥的圖片、標籤來訓練模型。</p><p>運行此的程式碼，等待程式全部執行完畢後，會自動匯出一 .etlt 檔案，便是我們最終要佈署的模型檔案。</p><p>PS. 想要快速上手 TLT 的 CV 使用範例程式碼皆可在此官方<a href="https://ngc.nvidia.com/catalog/resources/nvidia:tlt_cv_samples/quickStartGuide">連結</a>內找到，幾乎都是只要稍微調整參數即可一鍵訓練、優化、匯出。</p><p>如果是初學者想先掌握人工智慧模型訓練的基本原理與程式實作，可以參考我們的<a href="https://ai4kids.ai/courses/edge-computing-yolo/">YOLO影像辨識課程</a>。</p><h3>5. 佈署模型於 Jetson 裝置</h3><h4>Deepstream 簡介</h4><p>模型訓練完畢後，下一步是佈署於 Jetson 裝置進行即時推論，同時也需要將影像與推論結果進行串流。我們使用的是 Nvidia Deepstream，一個專為推論、資料串流而生的框架。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*o0GjdRSGm7i5EMwd.png" /></figure><p>DeepStream 是一套 end-to-end 的流程，可以進行深度學習推論、影像和感測器處理，並在串流應用程式中將洞見傳送至雲端。您可以使用容器建構雲端原生的 DeepStream 應用程式，並透過 Kubernetes 平台執行協調，以進行大規模的部署。當部署在邊緣端時，應用程式可以在 IoT 裝置與雲端標準訊息代理程式（例如 Kafka 和 MQTT）之間通訊，以進行大規模的廣域部署。</p><h4>TensorRT 簡介</h4><p>NVIDIA TensorRT™是用於深度學習推理的SDK。SDK包含深度學習學習優化器運行時環境，使深度學習學習應用提供低延遲和高吞吞。在進行模型佈署之前，需要 TensorRT 對模型進行轉換，加快推論時的運行速度。</p><p>TLT 是以佈署目的為導向的模型訓練框架，自然包含了針對訓練後模型的轉換功能。只要使用 tlt-converter 就可以對 TLT 訓練完畢的 .etlt 輸出模型進行 TensorRT 的推論引擎建置。另外，針對裝置佈署的硬體、軟體都需要用相應的 tlt-converter 版本進行轉換。以下為所有配置的選項：</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/1*HYSrwz-SXtKdujEVN5SiJQ.png" /></figure><h4>Deepstream 環境安裝</h4><p>我們記錄了 Deepstream 環境安裝的步驟，以及 .etlt 模型的 TensorRT 引擎建置過程。其中也包含安裝此專案所需要的其他套件的方法，如 MQTT client…等等，詳細安裝步驟請見我們準備的<a href="https://github.com/ai4kids-tw/Iguana-detection-on-Nvidia-Jetson-Nano-for-monitoring/tree/main/5.nvidia_deepstream_installation"><strong>環境設置教學</strong></a>。</p><h4>Deepstream 運行</h4><p>deepstream_app/deepstream_mqtt_rtsp_out.py 是 deepsteam python binding 的運行配置。此程式在進行物件偵測推論時，會將圖片以 RTSP 的方式進行串流，並且將推論結果之數據傳送到 MQTT broker、儲存至資料庫以方便之後的視覺化。</p><h3>6. 數據儀表板</h3><h4>Plotly Dash</h4><p>Plotly Dash 是一個資料可互動式視覺化的框架，可以用非前端的語言來撰寫一個資料視覺化的前端頁面，同時本套件延伸的其他子套件更是將網頁風格與排版的過程極簡化，非常適合快速開發基於網頁的資料視覺化。</p><p>另外，該套件提供的 live-update 功能，可以讓網頁持續動態更新，特別適合本專案的儀表板需求。</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*K0lrcUWfGIZ2WQE-.png" /></figure><h4>Dash Bootstrap Components</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/0*QkTs-0It6nTqB5EF.png" /></figure><p>Bootstrap是一組用於網站和網路應用程式開發的開源前端框架，包括HTML、CSS及JavaScript的框架，提供字體排印、表單、按鈕、導航及其他各種元件及Javascript擴充套件，旨在使動態網頁和Web應用的開發更加容易。</p><p>使用 Dash Bootstrap Components 可直接承襲 bootstrap 所定義好的框架，快速鍵置網頁的版面配置。</p><h4>MQTT broker</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/354/0*-eTGIHSoCnFFYtLE.png" /><figcaption>Eclipse Mosquitto</figcaption></figure><p>使用者可依照自己的喜好選用 MQTT broker，在此專案當中我們所使用的是 Eclipse Mosquitto。安裝好後只要啟動 MQTT broker 就能開始接收接收資料。</p><h4>資料庫建置</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*hM2QZfX9iVRPGeU0.png" /><figcaption>mySQL</figcaption></figure><p>本專案使用 mySQL 建置資料庫，使用者可挑選偏好的資料庫種類。資料庫的建目的是從 Jetson Nano 推論時發送至 MQTT broker 的資料流可以被寫入資料庫，透過 Plotly Dash 定時更新資料並且將最新的資料呈現於儀表板。</p><h4>MQTT 主題訂閱並與數據寫入資料庫</h4><p>Deepstream 在 Jetson Nano 上運行時通過 MQTT 通訊協定不斷向 MQTT broker 的特定主題發送推論結果之數據。透過設定此特定主題的訂閱者，就能在某主題有最新數據流入時收到通知、將數據寫入資料庫。此程式位置為 mqtt_topic_subscribe/mqtt_msg_to_db.py。</p><h4>運行網頁儀表板</h4><ul><li>Deepstream 佈署與運行</li><li>MQTT broker 運行於 data server</li><li>Database 運行於 data server</li></ul><p>具備以上的條件，網頁儀表板便可以開始運行了。請注意 Deepstream 的配置是可變動的，因此未來使用者若要將模型代換成其他物件偵測模型只要修改 Deepstream 的配置文件與 Deepstram python 執行檔案便可以。</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f1e4c587156d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>