<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Stephan Zhdanov on Medium]]></title>
        <description><![CDATA[Stories by Stephan Zhdanov on Medium]]></description>
        <link>https://medium.com/@ret7020?source=rss-c764fa3dece2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*gKDoAT-VQregqlEw3T67RQ.png</url>
            <title>Stories by Stephan Zhdanov on Medium</title>
            <link>https://medium.com/@ret7020?source=rss-c764fa3dece2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:08:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ret7020/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[LicheeRV Nano — Running YOLO]]></title>
            <link>https://medium.com/@ret7020/licheerv-nano-running-yolo-5de8d2a39776?source=rss-c764fa3dece2------2</link>
            <guid isPermaLink="false">https://medium.com/p/5de8d2a39776</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[edge-computing]]></category>
            <dc:creator><![CDATA[Stephan Zhdanov]]></dc:creator>
            <pubDate>Sun, 12 Oct 2025 18:09:56 GMT</pubDate>
            <atom:updated>2025-10-12T18:09:56.639Z</atom:updated>
            <content:encoded><![CDATA[<h3>LicheeRV Nano — Running YOLO</h3><p>Hello everyone! This is the third part of the series of articles on LicheeRV Nano. <a href="https://medium.com/@ret7020/licheerv-nano-ai-board-first-steps-d05e7999dd29">First part</a>, <a href="https://medium.com/@ret7020/licheerv-nano-board-programming-part-2-a10e33b0e110">second part</a> . This time we will talk about using CSI camera and running YOLO on NPU.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tWdXbAH3i_TgLQje.png" /></figure><p><strong>NPU</strong> <em>(Neural Processing Unit)</em> is a specialized co-processor designed to efficiently perform neural network-related computations. Unlike <strong>CPUs</strong> and <strong>GPUs</strong>, which can handle a wide range of tasks, <strong>NPUs</strong> are specifically optimized to accelerate neural network inference.</p><p><strong>Advantages of using an NPU:</strong></p><ul><li><strong>High performance</strong> — The specialized architecture allows processing neural network models with minimal latency.</li><li><strong>Low power consumption</strong> — Unlike GPUs, which require significant amounts of energy, NPUs perform computations more energy-efficiently.</li><li><strong>Minimal heat generation</strong> — Thanks to optimized resource management, NPUs generate less heat, making them ideal for embedded systems and mobile devices.</li><li><strong>Optimization for edge devices</strong> — NPUs are often used in IoT devices, robotics, and other systems that require autonomous operation and limited hardware resources.</li></ul><p>This article will demonstrate a real-time object detection solution using <strong>YOLOv8</strong> and <strong>YOLOv11</strong> on an NPU. Due to hardware acceleration, inference will be performed quickly and efficiently.</p><p><strong>Analytics</strong></p><p>Example code in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/Yolo_Benchmark">Projects/Yolo_Benchmark</a>.</p><p>A separate benchmark program was written to compare the performance of different <strong>YOLO</strong> models. It measures the inference speed of each model on images from a specified directory.</p><p>At the beginning of the article, a graph of the board’s power consumption during inference was provided, but only for <strong>YOLOv8n</strong>. This is because the model’s complexity does not significantly affect power consumption, especially considering the low accuracy of the USB tester.</p><p>To accelerate inference on the NPU, quantization is applied — reducing the bit width of the model weights. The NPU in the SG2002 only supports weights of two bit widths: INT8 and BF16.</p><ul><li><strong>INT8</strong> provides maximum performance — up to 1 TOPS.</li><li><strong>BF16</strong> offers higher quality but at a lower speed — about 0.5 TOPS.</li></ul><p>However, certain difficulties arose when using <strong>YOLO</strong> with BF16, so only the INT8 option is considered in the article.</p><p>Below is a summary table containing the results of inference speed measurements for various versions of <strong>YOLO</strong> models, datasets, and input image sizes.</p><p><strong>YOLOv8 INT8</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*LztthH_BByUTAc835aq_YA.png" /></figure><p><strong>YOLOv11 INT8</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*Vks6OyTnp-w4GRdEJ6j0mQ.png" /></figure><p><strong>YOLOv11 </strong>vs <strong>YOLOv8 </strong>plots</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1yRFdUynVKBXg8aK.png" /><figcaption>Inference</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9jiDV7S7gQgZm5d8.png" /><figcaption>Inference</figcaption></figure><p>Weights files sizes table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*Xv99LJbRPNqre5n1gXfDRw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*r3AhgQPDQz499uHY.png" /></figure><p>From the benchmark, two facts are clearly evident:</p><ol><li>Inference speed is inversely related to the size of the input images (approximately, but not perfectly linear).</li><li>The number of classes the model detects has a significant impact on the inference speed for the N and S models.</li></ol><p>For a quick test, you can use the COCO model YOLOv8n (640x640), available <a href="https://drive.google.com/file/d/1Z7Mo2jf0-PET8TMjbwUDA98kgG8G5doI/view?usp=drive_link">here</a>, and the binary for inference is available <a href="https://drive.google.com/file/d/1x0M_t1YqRvG775FoVjqOrx4YlYXQmXWs/view?usp=drive_link">here</a>. <a href="https://drive.google.com/file/d/1d-aIEVvb3XnlxcLygfvoHoD-Kl0tGhCE/view?usp=drive_link">Here is</a> an example image for detection with the correct dimensions.</p><p>Furthermore, working with the YOLOv11 model does not require any changes to the export, conversion, or inference code.</p><p><strong>Benchmark source code:</strong></p><pre>#include &lt;stdio.h&gt;<br>#include &lt;stdlib.h&gt;<br>#include &lt;string.h&gt;<br>#include &lt;time.h&gt;<br>#include &lt;chrono&gt;<br>#include &lt;functional&gt;<br>#include &lt;iostream&gt;<br>#include &lt;map&gt;<br>#include &lt;sstream&gt;<br>#include &lt;string&gt;<br>#include &lt;vector&gt;<br>#include &quot;core/cvi_tdl_types_mem_internal.h&quot;<br>#include &quot;core/utils/vpss_helper.h&quot;<br>#include &quot;cvi_tdl.h&quot;<br>#include &quot;cvi_tdl_media.h&quot;<br>#include &lt;dirent.h&gt;<br><br>#define MODEL_SCALE 0.0039216<br>#define MODEL_MEAN 0.0<br>#define MODEL_THRESH 0.5<br>#define MODEL_NMS_THRESH 0.5<br><br>#define READ_BUFF 512<br><br>typedef struct<br>{<br>    char * model_path;<br>    int classes;<br>    char * test_images_path;<br>} Model;<br><br>CVI_S32 init_param(const cvitdl_handle_t tdl_handle, int class_cnt)<br>{<br>    // setup preprocess<br>    YoloPreParam preprocess_cfg =<br>        CVI_TDL_Get_YOLO_Preparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION);<br><br>    for (int i = 0; i &lt; 3; i++)<br>    {<br>        printf(&quot;asign val %d \n&quot;, i);<br>        preprocess_cfg.factor[i] = MODEL_SCALE;<br>        preprocess_cfg.mean[i] = MODEL_MEAN;<br>    }<br>    preprocess_cfg.format = PIXEL_FORMAT_RGB_888_PLANAR;<br><br>    printf(&quot;setup yolov8 param \n&quot;);<br>    CVI_S32 ret = CVI_TDL_Set_YOLO_Preparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION,<br>                                            preprocess_cfg);<br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;Can not set yolov8 preprocess parameters %#x\n&quot;, ret);<br>        return ret;<br>    }<br><br>    // setup yolo algorithm preprocess<br>    YoloAlgParam yolov8_param =<br>        CVI_TDL_Get_YOLO_Algparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION);<br>    yolov8_param.cls = class_cnt;<br><br>    printf(&quot;setup yolov8 algorithm param \n&quot;);<br>    ret =<br>        CVI_TDL_Set_YOLO_Algparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, yolov8_param);<br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;Can not set yolov8 algorithm parameters %#x\n&quot;, ret);<br>        return ret;<br>    }<br><br>    // set theshold<br>    CVI_TDL_SetModelThreshold(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, MODEL_THRESH);<br>    CVI_TDL_SetModelNmsThreshold(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, MODEL_NMS_THRESH);<br><br>    printf(&quot;yolov8 algorithm parameters setup success!\n&quot;);<br>    return ret;<br>}<br><br>int main(int argc, char *argv[])<br>{<br>    int vpssgrp_width = 1920;<br>    int vpssgrp_height = 1080;<br>    CVI_S32 ret = MMF_INIT_HELPER2(vpssgrp_width, vpssgrp_height, PIXEL_FORMAT_RGB_888, 1,<br>                                   vpssgrp_width, vpssgrp_height, PIXEL_FORMAT_RGB_888, 1);<br>    if (ret != CVI_TDL_SUCCESS) { return ret; }<br><br>    cvitdl_handle_t tdl_handle = NULL;<br>    ret = CVI_TDL_CreateHandle(&amp;tdl_handle);<br>    if (ret != CVI_SUCCESS) {return ret;}<br><br>    imgprocess_t img_handle;<br>    CVI_TDL_Create_ImageProcessor(&amp;img_handle);<br><br>    char filename[] = &quot;config.txt&quot;;<br>    char buffer[READ_BUFF];<br>    FILE *fp = fopen(filename, &quot;r&quot;);<br>    if (fp)<br>    {<br>        while ((fgets(buffer, READ_BUFF, fp)) != NULL)<br>        {<br>            if (buffer[0] != &#39;#&#39;){<br>                char* token = strtok(buffer, &quot; &quot;);<br>                int index = 0;<br>                Model tmp = {filename, 2, filename};<br>                while (token != NULL) {<br><br>                    switch (index)<br>                    {<br>                        case 0:<br>                            tmp.model_path = token;<br>                            break;<br>                        case 1:<br>                            tmp.classes = atoi(token);<br>                            break;<br>                        case 2:<br>                            token[strlen(token) - 1] = &#39;\0&#39;;<br>                            tmp.test_images_path = token;<br>                    }<br>                    token = strtok(NULL, &quot; &quot;);<br>                    index++;<br>                }<br>                printf(&quot;%s:%d:%s\n&quot;, tmp.model_path, tmp.classes, tmp.test_images_path);<br>                ret = init_param(tdl_handle, tmp.classes);<br>                ret = CVI_TDL_OpenModel(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, tmp.model_path);<br>                if (ret != CVI_SUCCESS) { return ret; }<br><br>                DIR* dirp = opendir(tmp.test_images_path);<br>                if (dirp == NULL) {<br>                    perror(&quot;opendir failed&quot;);<br>                    return -1;<br>                }<br><br>                struct dirent * dp;<br>                double fps = 0.0;<br>                double fps_sum = 0.0;<br>                double fps_min = 1000.0;<br>                double fps_max = 0.0;<br>                int counter = 0;<br>                int frame_h = 0, frame_w = 0;<br>                <br>                while ((dp = readdir(dirp)) != NULL) {<br>                if (!strcmp(dp-&gt;d_name, &quot;..&quot;) || !strcmp(dp-&gt;d_name, &quot;.&quot;)) continue; // Skip ../ path<br>                    VIDEO_FRAME_INFO_S bg;<br>                    char absFilePath[512];<br>                    sprintf(absFilePath, &quot;%s/%s&quot;, &quot;coco128/&quot;, dp-&gt;d_name);<br>                    <br>                    ret = CVI_TDL_ReadImage(img_handle, absFilePath, &amp;bg, PIXEL_FORMAT_RGB_888_PLANAR);<br><br>                    cvtdl_object_t obj_meta = {0};<br>                    std::chrono::steady_clock::time_point begin = std::chrono::steady_clock::now();<br>                    CVI_TDL_YOLOV8_Detection(tdl_handle, &amp;bg, &amp;obj_meta);<br>                    std::chrono::steady_clock::time_point end = std::chrono::steady_clock::now();<br>                    fps = 1 / std::chrono::duration&lt;double&gt;(end - begin).count();<br>                    fps_sum += fps;<br>                    fps_max = std::max(fps_max, fps);<br>                    fps_min = std::min(fps_min, fps);<br>                    frame_h = bg.stVFrame.u32Height;<br>                    frame_w = bg.stVFrame.u32Width;<br><br>                    printf(&quot;IMG %s, %d: %lf\n&quot;, dp-&gt;d_name, counter, fps);<br>                    counter++;<br>                    CVI_TDL_ReleaseImage(img_handle, &amp;bg);<br>                }<br>                fps_sum = fps_sum / counter;<br>                printf(&quot;\n\n-------\nProcessed images: %d\nFrame size: %dx%d\nAVG FPS: %lf\nMin FPS: %lf\nMax FPS: %lf\n-------\n&quot;, counter, frame_h, frame_w, fps_sum, fps_min, fps_max);<br><br>            }<br>            <br>        }<br>        fclose(fp);<br>    }<br><br>    CVI_TDL_Destroy_ImageProcessor(img_handle);<br>    CVI_TDL_DestroyHandle(tdl_handle);<br>    <br>    <br>    return ret;<br>}</pre><p><strong>Model training and exporting for platform</strong></p><p><em>The model export process is also relevant for the Milk-V Duo 256, as this board uses the same SoC. If the board runs on a RISC-V core, you can even run binaries compiled for the LicheeRV Nano.</em></p><p>Base scheme of model exporting:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*H8QoqfiWhOHDZ_tX.png" /><figcaption>PyTorch model -&gt; ONNX model -&gt; MLIR model -&gt; calibration -&gt; INT8 cvimodel</figcaption></figure><p>For export, you will need the <strong>MLIR Toolkit</strong>. The official documentation recommends using it inside a Docker container. However, to make this guide accessible to those who cannot use Docker, I have prepared a <strong>Jupyter Notebook</strong> for Google Colab that allows you to export the model directly in your browser. The notebook provides an example of exporting a YOLOv8n model, but for <strong>YOLOv11</strong>, you will simply need to replace the model name/path.</p><p>The first two parts set up the environment: an older version of Python (3.10) and tpu_mlir are installed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*77zJ2j5KabnzVxgw.png" /><figcaption>Google Colab environment setup</figcaption></figure><p>Next, the <strong>YOLO</strong> weights are exported to the <strong>ONNX</strong> format. In this process, the model’s forward method is replaced, which is why a separate script is used for the conversion:</p><pre>from ultralytics import YOLO<br>import types<br>import sys<br><br>input_size = (int(sys.argv[2]), int(sys.argv[3]))<br><br>def forward2(self, x):<br>  x_reg = [self.cv2[i](x[i]) for i in range(self.nl)]<br>  x_cls = [self.cv3[i](x[i]) for i in range(self.nl)]<br>  return x_reg + x_cls<br><br>model_path = sys.argv[1]<br>model = YOLO(model_path)<br>model.model.model[-1].forward = types.MethodType(forward2, model.model.model[-1])<br>model.export(format=&#39;onnx&#39;, opset=11, imgsz=input_size)</pre><p>It takes the path to the weights as its first argument, which can be replaced with the path to your custom model’s weights. The next two arguments specify the size of the input image. I tested models with input sizes of <em>640x640</em> and <em>320x320</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gIKVK6Wifcrta9p-.png" /><figcaption>Export model from PyTorch weights to ONNX format</figcaption></figure><p>During the process of exporting the model from <strong>MLIR</strong> to cvimodel, one test image will be needed (for example, <a href="https://github.com/ret7020/YoloBenchmarks/blob/master/assets/images/bus.jpg?raw=true"><em>bus.jpg</em></a> for COCO models). You can replace it with your own if you are converting a custom <strong>YOLO</strong> model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CnZfA_hZ-lUDWGPw.png" /><figcaption>Prepare for model exporting</figcaption></figure><p>Additionally, the following are specified via environment variables: the image dimensions (as a string), the path to the calibration dataset (more on that later), the number of images to use for calibration, and the model name for export (without the file extension).</p><p>The next three cells convert the <strong>ONNX</strong> weights to <strong>MLIR</strong>, calibrate the model, and export it to <strong>cvimodel</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*m84P8dH0k0MmXoDI.png" /><figcaption>Export MLIR to CVIMODEL</figcaption></figure><p><strong>Arguments for the first cell’s command:</strong></p><ul><li>model_def: The path to the model weights in ONNX format.</li><li>input_shapes: The size of the input data (primarily, you only need to change the last two numbers, which represent the width and height of the input image).</li><li>mean and scale: Describe the color encoding. Each pixel color (r, g, b) is encoded as a float from 0 to 1, and the value 0.0039216 for scale equals 1/255 (1 byte per color).</li></ul><p>Here is a list of all parameters accepted by model_transform.py (taken from the <a href="https://drive.google.com/file/d/1ykCpWLsHdZ_BPN4EjEe1feRvFk55xAoB/view?usp=drive_link"><em>TPU MLIR Quick Start Guide</em></a>, where you can read more about the export process):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/0*8MqTa_2y3lbfquKS.png" /><figcaption>All export params</figcaption></figure><p>After executing model_transform, the <strong>ONNX</strong> model is converted to the <strong>MLIR</strong> format. Next, a calibration table must be created. For each image in the sample dataset, this process calculates the results from both the original model and the quantized model. These results are then compared to minimize the error introduced by reducing the weight precision.</p><p>For calibrating a COCO model, the <strong>COCO2017</strong> dataset is used, whose images are included by default in the <strong>TPU MLIR SDK</strong>. Its path is set via the CALIBRATION_DATASET_PATH environment variable, as mentioned above. The number of images used for calibration is configured in the CALIBRATION_IMAGES_COUNT variable. It is recommended to use the entire dataset for calibration, but to save time, you can select only a portion of it. It might also make sense to use a separate dataset for calibration that was not used during the model&#39;s training.</p><p>During calibration, model inference is performed on the CPU, without the option to offload to the GPU (CUDA). Therefore, this stage is the most time-consuming part of the model conversion process.</p><p>Finally, the last cell exports the quantized model to the file ./result/yolov8n_cv181x_int8_sym.cvimodel<em>(the filename depends on the initial model name)</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/421/0*JxRRI8cTNcnWXSNj.png" /><figcaption>Final file hierarchy</figcaption></figure><p>This is the model that will be run on the board. Here is the list of all parameters for the final command (model_deploy.py):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/941/0*lh-GgW17W-EG3pOq.png" /><figcaption>All export supported params</figcaption></figure><p>When exporting for the <strong>SG2002</strong> SoC (referred to by the <strong>NPU</strong> chip name cv181x in MLIR), you can use two types of quantization: INT8 and BF16. However, when exporting and running the BF16 <strong>YOLO</strong> model, it failed to detect any objects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/0*r6RyMnz2XBKs93hz.png" /><figcaption>Different platforms support different weights sizes/quantizations</figcaption></figure><p><strong>Inference on the Board</strong></p><p>The current example is located in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/Yolov8">Projects/Yolov8</a>.</p><p>A simple example of YOLO inference on an image from a file.</p><p><strong>Inference source code:</strong></p><pre>#include &lt;stdio.h&gt;<br>#include &lt;stdlib.h&gt;<br>#include &lt;time.h&gt;<br>#include &lt;chrono&gt;<br>#include &lt;functional&gt;<br>#include &lt;iostream&gt;<br>#include &lt;map&gt;<br>#include &lt;sstream&gt;<br>#include &lt;string&gt;<br>#include &lt;vector&gt;<br>#include &quot;core/cvi_tdl_types_mem_internal.h&quot;<br>#include &quot;core/utils/vpss_helper.h&quot;<br>#include &quot;cvi_tdl.h&quot;<br>#include &quot;cvi_tdl_media.h&quot;<br><br>#define MODEL_SCALE 0.0039216<br>#define MODEL_MEAN 0.0<br>#define MODEL_CLASS_CNT 80<br>#define MODEL_THRESH 0.8<br>#define MODEL_NMS_THRESH 0.8<br><br>// set preprocess and algorithm param for yolov8 detection<br>// if use official model, no need to change param (call this function)<br>CVI_S32 init_param(const cvitdl_handle_t tdl_handle)<br>{<br>    // setup preprocess<br>    YoloPreParam preprocess_cfg =<br>        CVI_TDL_Get_YOLO_Preparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION);<br><br>    for (int i = 0; i &lt; 3; i++)<br>    {<br>        printf(&quot;asign val %d \n&quot;, i);<br>        preprocess_cfg.factor[i] = MODEL_SCALE;<br>        preprocess_cfg.mean[i] = MODEL_MEAN;<br>    }<br>    preprocess_cfg.format = PIXEL_FORMAT_RGB_888_PLANAR;<br><br>    printf(&quot;setup yolov8 param \n&quot;);<br>    CVI_S32 ret = CVI_TDL_Set_YOLO_Preparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION,<br>                                            preprocess_cfg);<br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;Can not set yolov8 preprocess parameters %#x\n&quot;, ret);<br>        return ret;<br>    }<br><br>    // setup yolo algorithm preprocess<br>    YoloAlgParam yolov8_param =<br>        CVI_TDL_Get_YOLO_Algparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION);<br>    yolov8_param.cls = MODEL_CLASS_CNT;<br><br>    printf(&quot;setup yolov8 algorithm param \n&quot;);<br>    ret =<br>        CVI_TDL_Set_YOLO_Algparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, yolov8_param);<br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;Can not set yolov8 algorithm parameters %#x\n&quot;, ret);<br>        return ret;<br>    }<br><br>    // set theshold<br>    CVI_TDL_SetModelThreshold(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, MODEL_THRESH);<br>    CVI_TDL_SetModelNmsThreshold(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, MODEL_NMS_THRESH);<br><br>    printf(&quot;yolov8 algorithm parameters setup success!\n&quot;);<br>    return ret;<br>}<br><br>int main(int argc, char *argv[])<br>{<br>    int vpssgrp_width = 1920;<br>    int vpssgrp_height = 1080;<br>    CVI_S32 ret = MMF_INIT_HELPER2(vpssgrp_width, vpssgrp_height, PIXEL_FORMAT_RGB_888, 1,<br>                                   vpssgrp_width, vpssgrp_height, PIXEL_FORMAT_RGB_888, 1);<br>    if (ret != CVI_TDL_SUCCESS)<br>    {<br>        printf(&quot;Init sys failed with %#x!\n&quot;, ret);<br>        return ret;<br>    }<br><br>    cvitdl_handle_t tdl_handle = NULL;<br>    ret = CVI_TDL_CreateHandle(&amp;tdl_handle);<br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;Create tdl handle failed with %#x!\n&quot;, ret);<br>        return ret;<br>    }<br><br>    std::string strf1(argv[2]);<br><br>    // change param of yolov8_detection<br>    ret = init_param(tdl_handle);<br><br>    ret = CVI_TDL_OpenModel(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, argv[1]);<br><br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;open model failed with %#x!\n&quot;, ret);<br>        return ret;<br>    }<br>    imgprocess_t img_handle;<br>    CVI_TDL_Create_ImageProcessor(&amp;img_handle);<br><br>    VIDEO_FRAME_INFO_S bg;<br>    ret = CVI_TDL_ReadImage(img_handle, strf1.c_str(), &amp;bg, PIXEL_FORMAT_RGB_888_PLANAR);<br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;open img failed with %#x!\n&quot;, ret);<br>        return ret;<br>    }<br>    else<br>    {<br>        printf(&quot;image read,width:%d\n&quot;, bg.stVFrame.u32Width);<br>        printf(&quot;image read,hidth:%d\n&quot;, bg.stVFrame.u32Height);<br>    }<br><br>    cvtdl_object_t obj_meta = {0};<br>    std::chrono::steady_clock::time_point begin = std::chrono::steady_clock::now();<br>    CVI_TDL_YOLOV8_Detection(tdl_handle, &amp;bg, &amp;obj_meta);<br>    std::chrono::steady_clock::time_point end = std::chrono::steady_clock::now();<br>    double fps = 1 / std::chrono::duration&lt;double&gt;(end - begin).count();<br>    printf(&quot;\n\n----------\nDetection FPS: %lf\nDetected objects cnt: %d\n\nDetected objects:\n&quot;, fps, obj_meta.size);<br>    for (uint32_t i = 0; i &lt; obj_meta.size; i++)<br>    {<br>        printf(&quot;x1 = %lf, y1 = %lf, x2 = %lf, y2 = %lf, cls: %d, score: %lf\n&quot;, obj_meta.info[i].bbox.x1, obj_meta.info[i].bbox.y1, obj_meta.info[i].bbox.x2, obj_meta.info[i].bbox.y2, obj_meta.info[i].classes, obj_meta.info[i].bbox.score);<br>    }<br><br>    CVI_TDL_ReleaseImage(img_handle, &amp;bg);<br>    CVI_TDL_DestroyHandle(tdl_handle);<br>    CVI_TDL_Destroy_ImageProcessor(img_handle);<br>    return ret;<br>}</pre><p><strong>CMakeLists.txt</strong></p><pre>cmake_minimum_required(VERSION 3.10)<br>project(sample_yolov8)<br><br>set(CMAKE_C_COMPILER &quot;$ENV{COMPILER}/riscv64-unknown-linux-musl-g++&quot;)<br>set(CMAKE_CXX_FLAGS &quot;-march=rv64imafd -O3 -DNDEBUG -D_MIDDLEWARE_V2_ -DC906 -DUSE_TPU_IVE -fsigned-char -Werror=all -Wno-format-truncation -fdiagnostics-color=always -s&quot;)<br><br>include_directories(<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/include<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/include/cvi_tdl<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/middleware/v2/include<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/middleware/v2/include/linux<br>)<br><br>set(SOURCE_FILES main.c)<br>set(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR}/bin)<br>file(MAKE_DIRECTORY ${EXECUTABLE_OUTPUT_PATH})<br><br>add_executable(sample_yolov8 ${SOURCE_FILES})<br><br>target_link_libraries(sample_yolov8<br>    -mcpu=c906fdv<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/middleware/v2/lib<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/middleware/v2/lib/3rd<br>    -lini -lsns_full -lsample -lisp -lvdec -lvenc -lawb -lae -laf -lcvi_bin -lcvi_bin_isp -lmisc -lisp_algo -lsys -lvpu<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/opencv/lib<br>    -lopencv_core -lopencv_imgproc -lopencv_imgcodecs<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/tpu/lib<br>    -lcnpy -lcvikernel -lcvimath -lcviruntime -lz -lm<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/ive/lib<br>    -lcvi_ive_tpu<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/lib<br>    -lcvi_tdl<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/lib<br>    -lpthread -latomic<br>)</pre><p>The init_param function sets the model parameters: scale, mean, number of classes, model type (the <strong>YOLOv8</strong> model type is suitable for <strong>YOLOv11</strong>). At the beginning, there are several defines whose values are used during initialization:</p><ul><li>MODEL_SCALE = 0.0039216 - color scale, same as during export to TPU MLIR</li><li>MODEL_MEAN = 0.0 - color mean, same as during export to TPU MLIR</li><li>MODEL_CLASS_CNT = 80 - number of classes detected by the model (80 for COCO in this case)</li><li>MODEL_THRESH = 0.8 - object detection filter threshold</li><li>MODEL_NMS_THRESH = 0.8 - NMS filter threshold</li></ul><p>The example can be built locally on a Linux computer or in a <strong>Google Colab notebook</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*moUc93qMfWf00Utl.png" /><figcaption>Build example in Google Colab</figcaption></figure><p>Building requires the TDL SDK, which can be downloaded <a href="https://drive.google.com/file/d/1cp7E3I2t7Tj-ljuCz68jh6kW2QPu-sLa/view?usp=sharing">here</a>. It then needs to be extracted:</p><pre>tar xvf cvitek_tdl_sdk_1228.tar.gz -C CVI_SDK<br>export COMPILER_PATH=&quot;path to the cross-compiler&quot;<br>export SDK_PATH=&quot;path to the CVI_SDK directory containing cvitek_tdl_sdk&quot;</pre><p><strong>Compilation:</strong></p><pre>mkdir build &amp;&amp; cd build<br>cmake ..<br>make</pre><p><strong>Running on the board:</strong></p><pre># Set LD_LIBRARY_PATH (if not automated in /etc/profile)<br>export LD_LIBRARY_PATH=/root/libs_patch/lib:/root/libs_patch/middleware_v2:/root/libs_patch/middleware_v2_3rd:/root/libs_patch/tpu_sdk_libs:/root/libs_patch:/root/libs_patch/opencv<br><br>./sample_yolov8 yolov8n_coco_640.cvimodel bus_640.jpg</pre><p>The launch arguments must be the path to the weights and the path to the test image. Its aspect ratio must match the aspect ratio of the model’s input size <em>(e.g., a 640x640 image can be fed to a 320x320 model, but a 640x480 image cannot)</em>.</p><p><strong>Example output:</strong></p><pre># ./sample_yolov8 yolov8n_coco_640.cvimodel bus_640.jpg<br>---------------------openmodel-----------------------version: 1.4.0<br>yolov8n Build at 2025-01-17 12:48:09 For platform cv181x<br>Max SharedMem size:2457600<br>---------------------to do detection-----------------------<br>image read,width:640<br>image read,hidth:640<br><br><br>----------<br>Detection FPS: 18.177214<br>Detected objects cnt: 5<br><br>Detected objects:<br>x1 = 42.027672, y1 = 234.737930, x2 = 189.531281, y2 = 536.424988, cls: 0, score: 0.866496<br>x1 = 529.857666, y1 = 230.736252, x2 = 639.212769, y2 = 520.091125, cls: 0, score: 0.866496<br>x1 = 4.051300, y1 = 135.388489, x2 = 637.122925, y2 = 445.642975, cls: 5, score: 0.832456<br>x1 = 176.457748, y1 = 242.180679, x2 = 272.009369, y2 = 508.060699, cls: 0, score: 0.832456<br>x1 = 0.000000, y1 = 326.076141, x2 = 59.022076, y2 = 516.136963, cls: 0, score: 0.566403</pre><p>The next section will cover an example of real-time detection using a <strong>CSI camera </strong><em>(GC4653)</em>, with the results visualized in an <strong>MJPEG stream</strong>.</p><p><strong>Custom model</strong></p><p>The custom model will detect two classes: red and blue landing pads for FPV drones.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HvePUV4uc6au8kzB.png" /><figcaption>Red mat labeled example</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zsy6ydfYPDGmKRNx.png" /><figcaption>Blue mat labeled</figcaption></figure><p><strong>Dataset analysis:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nwtlH1w7E29reFzE.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dRUhh49FsCovY5gg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IEMAasICwyU9INlw.png" /></figure><p>The <strong>YOLOv11n</strong> model will be used as the base model because it performs slightly faster than <strong>YOLOv8n</strong>. The model was trained for <strong>50</strong> epochs using default hyperparameters.</p><p><strong>Training metric graphs:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fwbh3Xfs07LULTAs.png" /><figcaption>Train metrics</figcaption></figure><p><strong>Confusion matrix for class determination:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kyS-YmjphyAVK_Wk.png" /><figcaption>Correct class detection</figcaption></figure><p>The current example is located in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/YoloCamera">Projects/YoloCamera</a>.</p><p>This code example reads frames from a CSI camera and streams the detection visualization in <strong>MJPEG</strong> format:</p><pre>#include &lt;stdio.h&gt;<br>#include &lt;stdlib.h&gt;<br>#include &lt;time.h&gt;<br>#include &lt;chrono&gt;<br>#include &lt;functional&gt;<br>#include &lt;map&gt;<br>#include &lt;sstream&gt;<br>#include &lt;string&gt;<br>#include &lt;vector&gt;<br>#include &lt;signal.h&gt;<br>#include &quot;core/cvi_tdl_types_mem_internal.h&quot;<br>#include &quot;core/utils/vpss_helper.h&quot;<br>#include &quot;cvi_tdl.h&quot;<br>#include &quot;cvi_tdl_media.h&quot;<br><br>#include &lt;opencv2/core/core.hpp&gt;<br>#include &lt;opencv2/highgui/highgui.hpp&gt;<br>#include &lt;opencv2/imgproc/imgproc.hpp&gt;<br><br>#include &quot;MJPEGWriter.h&quot;<br><br>#define MODEL_SCALE 0.0039216<br>#define MODEL_MEAN 0.0<br>#define MODEL_CLASS_CNT 2<br>#define MODEL_THRESH 0.2<br>#define MODEL_NMS_THRESH 0.2<br>#define BLUE_MAT cv::Scalar(255, 0, 0)<br>#define RED_MAT cv::Scalar(0, 0, 255)<br><br>volatile uint8_t interrupted = 0;<br><br>void interrupt_handler(int signum)<br>{<br>    printf(&quot;Signal: %d\n&quot;, signum);<br>    interrupted = 1;<br>}<br><br>// set preprocess and algorithm param for yolov8 detection<br>// if use official model, no need to change param (call this function)<br>CVI_S32 init_param(const cvitdl_handle_t tdl_handle)<br>{<br>    // setup preprocess<br>    YoloPreParam preprocess_cfg =<br>        CVI_TDL_Get_YOLO_Preparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION);<br><br>    for (int i = 0; i &lt; 3; i++)<br>    {<br>        printf(&quot;asign val %d \n&quot;, i);<br>        preprocess_cfg.factor[i] = MODEL_SCALE;<br>        preprocess_cfg.mean[i] = MODEL_MEAN;<br>    }<br>    preprocess_cfg.format = PIXEL_FORMAT_RGB_888_PLANAR;<br><br>    printf(&quot;setup yolov8 param \n&quot;);<br>    CVI_S32 ret = CVI_TDL_Set_YOLO_Preparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION,<br>                                            preprocess_cfg);<br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;Can not set yolov8 preprocess parameters %#x\n&quot;, ret);<br>        return ret;<br>    }<br><br>    // setup yolo algorithm preprocess<br>    YoloAlgParam yolov8_param =<br>        CVI_TDL_Get_YOLO_Algparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION);<br>    yolov8_param.cls = MODEL_CLASS_CNT;<br><br>    printf(&quot;setup yolov8 algorithm param \n&quot;);<br>    ret =<br>        CVI_TDL_Set_YOLO_Algparam(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, yolov8_param);<br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;Can not set yolov8 algorithm parameters %#x\n&quot;, ret);<br>        return ret;<br>    }<br><br>    // set theshold<br>    CVI_TDL_SetModelThreshold(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, MODEL_THRESH);<br>    CVI_TDL_SetModelNmsThreshold(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, MODEL_NMS_THRESH);<br><br>    printf(&quot;yolov8 algorithm parameters setup success!\n&quot;);<br>    return ret;<br>}<br><br>int main(int argc, char *argv[])<br>{<br><br>    signal(SIGINT, interrupt_handler);<br>    MJPEGWriter test(7777);<br><br>    cv::VideoCapture cap;<br>    cv::Mat bgr;<br><br>    cap.open(0);<br>    // cap.set(cv::CAP_PROP_FRAME_WIDTH, 320);<br>    // cap.set(cv::CAP_PROP_FRAME_HEIGHT, 320);<br>    cap &gt;&gt; bgr;<br><br>    test.write(bgr);<br>    test.start();<br><br>    printf(&quot;Pointer for High-Level code: %p\n&quot;, cap.image_ptr);<br>    VIDEO_FRAME_INFO_S *frame_ptr = (VIDEO_FRAME_INFO_S *)cap.image_ptr;<br><br>    CVI_S32 ret;<br>    // VSSGRP already inited by VideoCapture from OpenCV Mobile, second init will do some strange and cause memory problems<br><br>    cvitdl_handle_t tdl_handle = NULL;<br>    ret = CVI_TDL_CreateHandle(&amp;tdl_handle);<br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;Create tdl handle failed with %#x!\n&quot;, ret);<br>        return ret;<br>    }<br><br>    cap &gt;&gt; bgr;<br>    // cv::imwrite(&quot;captured.jpg&quot;, bgr);<br>    VIDEO_FRAME_INFO_S frame = *frame_ptr;<br><br>    // change param of yolov8_detection<br>    ret = init_param(tdl_handle);<br><br>    ret = CVI_TDL_OpenModel(tdl_handle, CVI_TDL_SUPPORTED_MODEL_YOLOV8_DETECTION, argv[1]);<br><br>    if (ret != CVI_SUCCESS)<br>    {<br>        printf(&quot;open model failed with %#x!\n&quot;, ret);<br>        return ret;<br>    }<br><br>    printf(&quot;image read,width:%d\n&quot;, frame.stVFrame.u32Width);<br>    printf(&quot;image read,hidth:%d\n&quot;, frame.stVFrame.u32Height);<br><br>    while (!interrupted)<br>    {<br>        cap &gt;&gt; bgr;<br>        VIDEO_FRAME_INFO_S frame = *frame_ptr;<br>        cvtdl_object_t obj_meta = {0};<br>        // std::chrono::steady_clock::time_point begin = std::chrono::steady_clock::now();<br>        CVI_TDL_YOLOV8_Detection(tdl_handle, &amp;frame, &amp;obj_meta);<br>        // std::chrono::steady_clock::time_point end = std::chrono::steady_clock::now();<br>        // double fps = 1 / std::chrono::duration&lt;double&gt;(end - begin).count();<br>        // printf(&quot;\n\n----------\nDetection FPS: %lf\nDetected objects cnt: %d\n\nDetected objects:\n&quot;, fps, obj_meta.size);<br>        for (uint32_t i = 0; i &lt; obj_meta.size; i++)<br>        {<br>            // printf(&quot;x1 = %lf, y1 = %lf, x2 = %lf, y2 = %lf, cls: %d, score: %lf\n&quot;, obj_meta.info[i].bbox.x1, obj_meta.info[i].bbox.y1, obj_meta.info[i].bbox.x2, obj_meta.info[i].bbox.y2, obj_meta.info[i].classes, obj_meta.info[i].bbox.score);<br>            cv::Rect r = cv::Rect(obj_meta.info[i].bbox.x1, obj_meta.info[i].bbox.y1, obj_meta.info[i].bbox.x2 - obj_meta.info[i].bbox.x1, obj_meta.info[i].bbox.y2 - obj_meta.info[i].bbox.y1);<br><br>            if (obj_meta.info[i].classes == 0)<br>                cv::rectangle(bgr, r, BLUE_MAT, 1, 8, 0);<br>            else if (obj_meta.info[i].classes == 1)<br>                cv::rectangle(bgr, r, RED_MAT, 1, 8, 0);<br><br>            cv::putText(bgr,<br>                        &quot;Mat&quot;,<br>                        cv::Point(obj_meta.info[i].bbox.x1, obj_meta.info[i].bbox.y1 - 5),<br>                        cv::FONT_HERSHEY_DUPLEX,<br>                        1.0,<br>                        (obj_meta.info[i].classes == 0) ? BLUE_MAT : RED_MAT,<br>                        1);<br>        }<br>        test.write(bgr);<br>        bgr.release();<br>    }<br><br>    printf(&quot;Stopping stream:\n&quot;);<br>    test.stop();<br>    cap.release();<br><br>    CVI_TDL_DestroyHandle(tdl_handle);<br><br>    return ret;<br>}</pre><p><strong>CMakeLists.txt:</strong></p><pre>cmake_minimum_required(VERSION 3.10)<br>project(stream_yolov8)<br>set(CMAKE_CXX_STANDARD 11)<br>set(CMAKE_C_COMPILER &quot;$ENV{COMPILER}/riscv64-unknown-linux-musl-g++&quot;)<br>SET(CMAKE_CXX_COMPILER &quot;$ENV{COMPILER}/riscv64-unknown-linux-musl-g++&quot;)<br>SET(CMAKE_C_LINK_EXECUTABLE &quot;$ENV{COMPILER}/riscv64-unknown-linux-musl-ld&quot;)<br><br>set(CMAKE_CXX_FLAGS &quot;-march=rv64imafd -O3 -DNDEBUG -D_MIDDLEWARE_V2_ -DC906 -DUSE_TPU_IVE -fsigned-char -Werror=all -Wno-format-truncation -fdiagnostics-color=always -s&quot;)<br><br>set(OpenCV_DIR &quot;${CMAKE_CURRENT_SOURCE_DIR}/libs/opencv-mobile-4.10.0-licheerv-nano/lib/cmake/opencv4&quot;)<br>find_package(OpenCV REQUIRED)<br>include_directories(${OpenCV_INCLUDE_DIRS})<br><br><br>include_directories(<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/include<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/include/cvi_tdl<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/middleware/v2/include<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/middleware/v2/include/linux<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/middleware/v2/include/isp/cv181x<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/sample/utils<br>    $ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/rtsp/include/cvi_rtsp<br>    # ${OpenCV_INCLUDE_DIRS}<br>)<br><br>set(SOURCE_FILES main.c)<br>set(EXECUTABLE_OUTPUT_PATH ${CMAKE_BINARY_DIR}/bin)<br>file(MAKE_DIRECTORY ${EXECUTABLE_OUTPUT_PATH})<br><br>add_executable(stream_yolov8 MJPEGWriter.cpp ${SOURCE_FILES})<br><br>target_link_libraries(stream_yolov8<br>    -mcpu=c906fdv<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/middleware/v2/lib<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/middleware/v2/lib/3rd<br>    -lini -lsns_full -lsample -lisp -lvdec -lvenc -lawb -lae -laf -lcvi_bin -lcvi_bin_isp -lmisc -lisp_algo -lsys -lvpu<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/opencv/lib<br>    -lopencv_core -lopencv_imgproc -lopencv_imgcodecs<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/tpu/lib<br>    -lcnpy -lcvikernel -lcvimath -lcviruntime -lz -lm<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/ive/lib<br>    -lcvi_ive_tpu<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/lib<br>    -lcvi_tdl<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/3rd/lib<br>    -L$ENV{SDK_PATH}/cvitek_tdl_sdk/sample/utils<br>    -lpthread -latomic<br>    ${OpenCV_LIBS}<br>)</pre><p>An important difference from the inference example without <strong>OpenCV Mobile</strong> is the absence of <strong>VPSS</strong> initialization:</p><pre>CVI_S32 ret = MMF_INIT_HELPER2(vpssgrp_width, vpssgrp_height, PIXEL_FORMAT_RGB_888, 1,<br>                                vpssgrp_width, vpssgrp_height, PIXEL_FORMAT_RGB_888, 1);</pre><p>This initialization is handled internally by VideoCapture.open in <strong>OpenCV Mobile</strong>. Calling it again leads to issues and unstable behavior of the <strong>TDL SDK</strong>.</p><p>Under the hood, <strong>OpenCV Mobile</strong> reads camera frames using <strong>TDL SDK</strong> methods, which return frames as a VIDEO_FRAME_INFO_S structure (this format is required for inference). It then converts them to cv::Mat. Converting back from cv::Mat to VIDEO_FRAME_INFO_S after reading a frame seems inefficient. Therefore, I added a capability in <strong>OpenCV Mobile</strong> to retrieve a pointer to the original camera frame in VIDEO_FRAME_INFO_S format from the main code.</p><p>As a result, it’s now possible to work simultaneously with two identical frames: one is used for inference (VIDEO_FRAME_INFO_S), while the second (cv::Mat) is used for visualization using <strong>OpenCV Mobile</strong> tools.</p><pre>cv::VideoCapture cap;<br>cv::Mat bgr;<br>cap.open(0);<br>cap &gt;&gt; bgr;<br>VIDEO_FRAME_INFO_S *frame_ptr = (VIDEO_FRAME_INFO_S *)cap.image_ptr;<br>// …<br>VIDEO_FRAME_INFO_S frame = *frame_ptr;<br>CVI_TDL_YOLOV8_Detection(tdl_handle, &amp;frame, &amp;obj_meta);</pre><p>Finally, after program starts on board you can read MJPEG stream on port 7777 (can be opened via ffplay or generic web browser):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*Yiw3WGfK3UxTEDED.gif" /><figcaption>For demo I use 320x320 res</figcaption></figure><p>As a result, stable detection occurs at distances of up to 2–2.5 meters. Detection at greater distances can be achieved by expanding the dataset, as in the current dataset, all canvases were located at distances of up to 1 meter.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5de8d2a39776" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Speed Up YOLOv11 by 30% for Real-Time Edge AI]]></title>
            <link>https://medium.com/@ret7020/how-to-speed-up-yolov11-by-30-for-real-time-edge-ai-bedf5e1973e2?source=rss-c764fa3dece2------2</link>
            <guid isPermaLink="false">https://medium.com/p/bedf5e1973e2</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[edge-computing]]></category>
            <dc:creator><![CDATA[Stephan Zhdanov]]></dc:creator>
            <pubDate>Mon, 08 Sep 2025 17:11:59 GMT</pubDate>
            <atom:updated>2025-09-08T17:11:59.436Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vkbW5Bx3MiN5ZpIl5p7cGg.png" /><figcaption>DSC sample</figcaption></figure><h3>1. What is Depthwise Separable Convolution (DSC)?</h3><p>Depthwise Separable Convolution (DSC) is an efficient alternative to standard convolution that reduces computational cost while maintaining competitive accuracy. Instead of applying a 3D convolution kernel across all input channels simultaneously, DSC <strong>splits the operation into two steps</strong>:</p><ol><li><strong>Depthwise convolution</strong> — Each input channel is convolved separately using a single spatial filter. This step captures spatial features but does not mix information between channels.</li><li><strong>Pointwise convolution (1×1 conv) — </strong>A 1×1 convolution is applied to combine the channel-wise outputs from the depthwise step, enabling feature interaction across channels.</li></ol><p>This factorization significantly lowers the number of multiply-accumulate (MAC) operations compared to a standard convolution.</p><ul><li>On <strong>NPUs (Neural Processing Units)</strong>, this reduction directly translates into <strong>higher throughput and lower latency</strong>, since NPUs are designed for highly parallel and lightweight operations.</li><li>On <strong>GPUs/CPUs</strong>, the benefit is not as pronounced because these architectures are optimized for large, batched dense matrix multiplications. Thus, replacing standard convolutions with DSC may show little to no speedup (and sometimes even overhead) in generic runtimes.</li></ul><p>DSC is used in lightweight models such as <strong>MobileNet, FastDepth</strong>, and in this article we will implement it for YOLO.</p><h3>2. Computational Savings: Conv vs DSConv</h3><p>Let’s calculate the number of operations for a standard convolution vs. depthwise separable convolution on an input tensor of <strong>640 × 640 × 3</strong> with a kernel size of <strong>3 × 3</strong>, stride 1, and 32 output channels.</p><h3>Standard Convolution</h3><ul><li>Input: 640 × 640 × 3</li><li>Kernel: 3 × 3 × 3 × 32</li><li>MACs per output position: 3 × 3 × 3 × 32 = <strong>864</strong></li><li>Output size: 640 × 640 × 32 ≈ 13M positions</li><li>Total MACs = 864 × 13,107,200 ≈ <strong>11.3B operations</strong></li></ul><h3>Depthwise Separable Convolution</h3><p><strong>Depthwise step:</strong></p><ul><li>One 3 × 3 kernel per input channel (3 total)</li><li>MACs per output position: 3 × 3 × 3 = <strong>27</strong></li><li>Total = 27 × 13,107,200 ≈ <strong>354M operations</strong></li></ul><p><strong>Pointwise step:</strong></p><ul><li>1 × 1 × 3 × 32 filters</li><li>MACs per output position: 96</li><li>Total = 96 × 13,107,200 ≈ <strong>1.26B operations</strong></li></ul><p><strong>Combined total DSC = 354M + 1.26B = 1.61B operations</strong></p><p>➡️ <strong>Reduction: 11.3B → 1.61B (~7× fewer computations)</strong></p><p>This is why <strong>DSC</strong> is so attractive for edge <strong>NPUs</strong>.</p><h3>3. Implementing DSC in YOLOv11</h3><p>Source code — model configs and patches for <strong>YOLO</strong> code base can be found <a href="https://github.com/ret7020/YoloTuning">here</a>.</p><p>To use <strong>DSC</strong> inside <strong>YOLOv11 </strong>we need to tune two parts:</p><ul><li><strong>Model Architecture:</strong> Replace some of the standard convolutional layers (we will replace Convs and C3k2 blocks in backbone and head) with depthwise separable convolutions.</li><li><strong>Code changes: </strong>We need to implement two new blocks: <strong>DSConv</strong> and <strong>DSC3k2</strong>.</li></ul><p><strong>Model Architecture changes </strong><em>(file: configs/ds_cfg_backbone_and_head.yaml)</em><strong>:</strong></p><pre># Ultralytics 🚀 AGPL-3.0 License - https://ultralytics.com/license<br><br># Ultralytics YOLO11 object detection model with P3/8 - P5/32 outputs<br># Model docs: https://docs.ultralytics.com/models/yolo11<br># Task docs: https://docs.ultralytics.com/tasks/detect<br><br># Parameters<br>nc: 80 # number of classes<br>scales: # model compound scaling constants, i.e. &#39;model=yolo11n.yaml&#39; will call yolo11.yaml with scale &#39;n&#39;<br>  # [depth, width, max_channels]<br>  n: [0.50, 0.25, 1024] # summary: 181 layers, 2624080 parameters, 2624064 gradients, 6.6 GFLOPs<br>  s: [0.50, 0.50, 1024] # summary: 181 layers, 9458752 parameters, 9458736 gradients, 21.7 GFLOPs<br>  m: [0.50, 1.00, 512] # summary: 231 layers, 20114688 parameters, 20114672 gradients, 68.5 GFLOPs<br>  l: [1.00, 1.00, 512] # summary: 357 layers, 25372160 parameters, 25372144 gradients, 87.6 GFLOPs<br>  x: [1.00, 1.50, 512] # summary: 357 layers, 56966176 parameters, 56966160 gradients, 196.0 GFLOPs<br><br># YOLO11n backbone<br>backbone:<br>  # [from, repeats, module, args]<br>  - [-1, 1, DSConv, [64, 3, 2]] # 0-P1/2<br>  - [-1, 1, DSConv, [128, 3, 2]] # 1-P2/4<br>  - [-1, 2, DSC3k2, [256, False, 0.25]]<br>  - [-1, 1, DSConv, [256, 3, 2]] # 3-P3/8<br>  - [-1, 2, DSC3k2, [512, False, 0.25]]<br>  - [-1, 1, DSConv, [512, 3, 2]] # 5-P4/16<br>  - [-1, 2, DSC3k2, [512, True]]<br>  - [-1, 1, DSConv, [1024, 3, 2]] # 7-P5/32<br>  - [-1, 2, DSC3k2, [1024, True]]<br>  - [-1, 1, SPPF, [1024, 5]] # 9<br>  - [-1, 2, C2PSA, [1024]] # 10<br><br># YOLO11n head<br>head:<br>  - [-1, 1, nn.Upsample, [None, 2, &quot;nearest&quot;]]<br>  - [[-1, 6], 1, Concat, [1]] # cat backbone P4<br>  - [-1, 2, C3k2, [512, False]] # 13<br><br>  - [-1, 1, nn.Upsample, [None, 2, &quot;nearest&quot;]]<br>  - [[-1, 4], 1, Concat, [1]] # cat backbone P3<br>  - [-1, 2, DSC3k2, [256, False]] # 16 (P3/8-small)<br><br>  - [-1, 1, DSConv, [256, 3, 2]]<br>  - [[-1, 13], 1, Concat, [1]] # cat head P4<br>  - [-1, 2, DSC3k2, [512, False]] # 19 (P4/16-medium)<br><br>  - [-1, 1, DSConv, [512, 3, 2]]<br>  - [[-1, 10], 1, Concat, [1]] # cat head P5<br>  - [-1, 2, DSC3k2, [1024, True]] # 22 (P5/32-large)<br><br>  - [[16, 19, 22], 1, Detect, [nc]] # Detect(P3, P4, P5)</pre><p>So, if you compare this config with the original <strong>YOLO11</strong> config you can see that all <strong>Convs</strong> and <strong>C3k2</strong> were replaced for their <strong>Depthwise Separable Convolution</strong> alternatives.</p><p>But, we need to also implement these blocks with <strong>PyTorch</strong> inside <strong>YOLO</strong>’s code-base. So all changes you can see in patches in my repo, but main changes I will describe here. File <em>modules/block.py</em>:</p><pre>class DSConv(nn.Module):<br>    &quot;&quot;&quot;The Basic Depthwise Separable Convolution.&quot;&quot;&quot;<br>    def __init__(self, c_in, c_out, k=3, s=1, p=None, d=1, bias=False):<br>        super().__init__()<br>        if p is None:<br>            p = (d * (k - 1)) // 2<br>        self.dw = nn.Conv2d(<br>            c_in, c_in, kernel_size=k, stride=s,<br>            padding=p, dilation=d, groups=c_in, bias=bias<br>        )<br>        self.pw = nn.Conv2d(c_in, c_out, 1, 1, 0, bias=bias)<br>        self.bn = nn.BatchNorm2d(c_out)<br>        self.act = nn.SiLU()<br><br>    def forward(self, x):<br>        x = self.dw(x)<br>        x = self.pw(x)<br>        return self.act(self.bn(x))</pre><p>This is an implementation of a simple <strong>DSC</strong> which handles setup args same as the original <strong>Conv</strong> block in <strong>YOLO</strong>.</p><p><strong>DSC3k2</strong> implementation is divided into two stages:</p><ul><li><strong>DSBottleneck</strong> implementation</li></ul><pre>class DSBottleneck(nn.Module):<br>    def __init__(self, c1, c2, shortcut=True, e=0.5, k1=3, k2=5, d2=1):<br>        super().__init__()<br>        c_ = int(c2 * e)<br>        self.cv1 = DSConv(c1, c_, k1, s=1, p=None, d=1)   <br>        self.cv2 = DSConv(c_, c2, k2, s=1, p=None, d=d2)  <br>        self.add = shortcut and c1 == c2<br><br>    def forward(self, x):<br>        y = self.cv2(self.cv1(x))<br>        return x + y if self.add else y</pre><ul><li>And <strong>DSC3k2</strong></li></ul><pre>class DSC3k2(C2f):<br>    def __init__(<br>        self,<br>        c1,          <br>        c2,         <br>        n=1,          <br>        dsc3k=False,  <br>        e=0.5,       <br>        g=1,        <br>        shortcut=True,<br>        k1=3,       <br>        k2=7,       <br>        d2=1         <br>    ):<br>        super().__init__(c1, c2, n, shortcut, g, e)<br>        if dsc3k:<br>            self.m = nn.ModuleList(<br>                DSC3k(<br>                    self.c, self.c,<br>                    n=2,           <br>                    shortcut=shortcut,<br>                    g=g,<br>                    e=1.0,  <br>                    k1=k1,<br>                    k2=k2,<br>                    d2=d2<br>                )<br>                for _ in range(n)<br>            )<br>        else:<br>            self.m = nn.ModuleList(<br>                DSBottleneck(<br>                    self.c, self.c,<br>                    shortcut=shortcut,<br>                    e=1.0,<br>                    k1=k1,<br>                    k2=k2,<br>                    d2=d2<br>                )<br>                for _ in range(n)<br>            )</pre><h3>4. Performance Testing</h3><p>We will compare custom model metrics and inference speed on LicheeRV Nano — small SBC with <strong>SG2002</strong> SoC that have integrated NPU with performance of <strong>1 TOPS in INT8</strong> calculations.</p><p><em>This article will not cover the process of exporting a model for board, but it is no different from exporting a regular YOLO model.</em></p><p>Custom model have only <strong>1 class</strong> and are trained for <strong>50 epochs</strong> on both configs. <em>50 epochs is not enough for maximum accuracy, but anyway we will compare relative accuracy between DSConved and model and original.</em></p><p>Also for correct <strong>MACs</strong>/<strong>FLOPs</strong> calculation for <strong>DSC</strong> model I use <strong>thop</strong> with hook for new blocks.</p><pre>import torch<br>import torch.nn as nn<br>from thop import profile<br>from ultralytics import YOLO<br>from ultralytics.nn.modules import DSConv<br>from ultralytics.nn.modules.block import DSBottleneck, DSC3k, DSC3k2<br><br><br>def dsconv_flops_counter_hook(module, input, output):<br>    x = input[0]  # input tensor<br>    Cin = module.dw.in_channels<br>    Cout = module.pw.out_channels<br>    Hout, Wout = output.shape[2], output.shape[3]<br>    k = module.dw.kernel_size[0]<br><br>    macs_dw = Hout * Wout * Cin * (k * k)<br>    macs_pw = Hout * Wout * Cin * Cout<br>    module.__flops__ = 2 * (macs_dw + macs_pw)<br><br>def dsbottleneck_flops_counter_hook(module, input, output):<br>    flops = 0<br>    for m in [module.cv1, module.cv2]:<br>        x = input[0]<br>        Cin = m.dw.in_channels<br>        Cout = m.pw.out_channels<br>        Hout, Wout = output.shape[2], output.shape[3]<br>        k = m.dw.kernel_size[0]<br>        macs_dw = Hout * Wout * Cin * (k * k)<br>        macs_pw = Hout * Wout * Cin * Cout<br>        flops += 2 * (macs_dw + macs_pw)<br>    module.__flops__ = flops<br><br>def dsc3k_flops_counter_hook(module, input, output):<br>    module.__flops__ = sum(getattr(m, &quot;__flops__&quot;, 0) for m in module.modules())<br><br><br>config_name = &quot;ds_cfg_all.yaml&quot;<br>model = YOLO(config_name)<br>print(model.info())<br>model = model.model<br>dummy = torch.randn(1, 3, 640, 640)<br><br>flops, params = profile(model, inputs=(dummy,),<br>                        custom_ops={DSConv: dsconv_flops_counter_hook, <br>                                    DSBottleneck: dsbottleneck_flops_counter_hook,<br>                                    DSC3k: dsc3k_flops_counter_hook,<br>                                    DSC3k2: dsc3k_flops_counter_hook})<br><br>print(f&quot;Config: {config_name}:&quot;)<br>print(f&quot;FLOPs: {flops/1e9:.3f} GFLOPs&quot;)<br>print(f&quot;MACs: {flops/2/1e9:.3f} GMACs&quot;)</pre><p>We can see that there is no significant drop in accuracy despite the fact that default hyperparameters and a small number of epochs were used. At the same time, there is a noticeable increase in the inference speed on the board.</p><h3>5. Summary</h3><p>Depthwise separable convolution is a powerful optimization for YOLOv11 when deployed on NPUs or mobile hardware. While it doesn’t yield much benefit on GPUs/CPUs, it drastically reduces operations and memory access, making it ideal for edge AI applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bedf5e1973e2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LicheeRV Nano — Board programming (Part 2)]]></title>
            <link>https://medium.com/@ret7020/licheerv-nano-board-programming-part-2-a10e33b0e110?source=rss-c764fa3dece2------2</link>
            <guid isPermaLink="false">https://medium.com/p/a10e33b0e110</guid>
            <category><![CDATA[cpp-programming]]></category>
            <category><![CDATA[electronics]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Stephan Zhdanov]]></dc:creator>
            <pubDate>Sun, 18 May 2025 13:41:23 GMT</pubDate>
            <atom:updated>2025-05-18T13:41:23.527Z</atom:updated>
            <content:encoded><![CDATA[<h3>LicheeRV Nano — Board programming (Part 2)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MjqSh5re2YbXhQlV.jpeg" /></figure><p>Hello everyone! This is the second part of the series of articles on LicheeRV Nano. First article you can read <a href="https://medium.com/@ret7020/licheerv-nano-ai-board-first-steps-d05e7999dd29">here</a>. This time we will talk about using peripherals under Linux.</p><h4>Hello, world!</h4><p>All examples are provided as ready-to-use, structured projects with Makefile or CMake (for some) and can be found in my GitHub repository:</p><pre>git clone https://github.com/ret7020/LicheeRVNano</pre><p>The current example is located in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/HelloWorld">Projects/HelloWorld</a>.</p><p>Development for the board will be done in C/C++. The cross-compiler can be downloaded <a href="https://sophon-file.sophon.cn/sophon-prod-s3/drive/23/03/07/16/host-tools.tar.gz">here</a>. The archive contains three versions of the cross-compiler. The official Buildroot image uses the <strong>musl</strong> implementation of libc, so you should use the <strong>riscv64-linux-musl-x86_64</strong> version.</p><p>Inside the riscv64-linux-musl-x86_64/bin directory, you will find the GCC binary named riscv64-unknown-linux-musl-gcc.</p><p>The cross-compiler only works on x86 Linux, but I’ve prepared a special <a href="https://colab.research.google.com/drive/1wCq9WCEtpPNionaIoEON5OSE7HLfRTNS?usp=sharing">Jupyter notebook on Google Colab</a> that allows you to build projects for the board directly in your browser, on almost any system (as long as the browser works).</p><p>In some examples that use ioctl (e.g., working with sysfs - GPIO), you may encounter compilation issues. These can be resolved by editing ioctl.h (./gcc/riscv64-linux-musl-x86_64/sysroot/usr/include/sys/ioctl.h) — comment out line 9:</p><pre>//#include &lt;bits/alltypes.h&gt;</pre><p>In the Google Colab notebook, this patch is applied to the cross-compiler automatically.</p><p>All subsequent examples use the COMPILER environment variable to get the absolute path to the cross-compiler. You need to set it to the path to the compiler binaries:</p><pre>export COMPILER=/PATH/TO/COMPILER/gcc/riscv64-linux-musl-x86_64/bin</pre><p>Replace /PATH/TO/COMPILER with your actual path. <br><strong>Example source code:</strong></p><pre>#include &lt;stdio.h&gt;<br><br>int main(){<br>    printf(&quot;Hello, world from LicheeRV Nano!\n&quot;);<br>    return 0;<br>}</pre><p><strong>Makefile:</strong></p><pre>build:<br>    mkdir -p bin<br>    echo Using: ${COMPILER}<br>    ${COMPILER}/riscv64-unknown-linux-musl-gcc main.c -o ./bin/hello<br>deploy:<br>    scp ./bin/hello root@${LICHEERV_IP}:/root/hello</pre><p>If you set the board’s IP address in the LICHEERV_IP environment variable, you can use make deploy to automatically upload the binary to the board via scp after compilation. Most Linux file managers also support the <strong>SFTP</strong> protocol, which makes it convenient to work with files on the board.</p><p>When debugging software on the board, it’s helpful to check the logs. The main sources of logs include:</p><pre>dmesg<br>cat /var/log/messages     # especially useful when debugging projects using the TDL SDK (e.g., camera access, neural network inference)<br>cat /proc/cvitek/vi_dbg   # debug information for VI (video input), i.e., CSI camera<br>cat /proc/cvitek/vi       # information about the CSI camera<br>cat /proc/mipi-rx</pre><p>And of course, the logs sent to the <strong>debug UART0</strong> can also be very useful.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uEg_9qvFbNUEHr5E" /><figcaption>Many logs</figcaption></figure><h3>Peripheral with Linux</h3><p>This section explains how to use various hardware features of the board.</p><h4>GPIO</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XgXfxpT-urtoneuQ" /><figcaption>In the screenshot above, there’s a typo — GPIO pin A15 has number <strong>495</strong> in Linux.</figcaption></figure><p>The screenshot shows the board’s pinout. To work with the pins from Linux, we need to know their numbers (the second 3-digit number shown next to each pin in the diagram). All pins go through a multiplexer, so we need to select which SoC pin is routed to which physical pin on the board. This allows us to assign different functions to a pin — e.g., <strong>GPIO</strong>, <strong>UART</strong>, <strong>I2C</strong>, <strong>PWM</strong>, or <strong>SPI</strong> on pin <em>P18</em>.</p><p>The addresses of the pins in the multiplexer are listed in the old version (1.0) of the SG2002 datasheet in Chapter “10.1 PINMUX”. However, it’s better to use <a href="https://github.com/sophgo/sophgo-hardware/blob/master/SG200X/04_SG2002/04_SG2002_PINOUT.xlsx">this spreadsheet</a> (sheet “1. QFN”), which is much more convenient.</p><p>Also, note that some pins may already be used by other onboard peripherals:</p><ul><li>The <strong>Wi-Fi module</strong> uses pins P18 through P23 and pin A26.</li><li><strong>Bluetooth</strong> uses pins A18, A19, A28, and A29.</li></ul><p>Pin configuration can be done during board boot using <strong>U-Boot</strong>, or from within the running Linux OS using the devmem utility. Configuring via <strong>U-Boot</strong> requires rebuilding and updating the bootloader. We’ll go over both methods.</p><p><strong>Configuration in U-Boot<br></strong>Edit the cvi_board_init function in the following file: /workspace/build/boards/sg200x/sg2002_licheervnano_sd/u-boot/cvi_board_init.c</p><p>For example, if your board doesn’t have a Wi-Fi module or you don’t need it, you can comment out the following lines:</p><pre>// (comment next lines from 66-89)<br>// wifi power reset<br>mmio_write_32(0x0300104C, 0x3); // GPIOA 26<br>val = mmio_read_32(0x03020004); // GPIOA DIR<br>val |= (1 &lt;&lt; 26); // output<br>mmio_write_32(0x03020004, val);<br><br>val = mmio_read_32(0x03020000); // signal level<br>val &amp;= ~(1 &lt;&lt; 26); // set level to low<br>mmio_write_32(0x03020000, val);<br><br>suck_loop(50);<br>user_led_toggle();<br>val = mmio_read_32(0x03020000); // signal level<br>val |= (1 &lt;&lt; 26); // set level to high<br>mmio_write_32(0x03020000, val);<br><br>// wifi sdio pinmux<br>mmio_write_32(0x030010D0, 0x0); // D3<br>mmio_write_32(0x030010D4, 0x0); // D2<br>mmio_write_32(0x030010D8, 0x0); // D1<br>mmio_write_32(0x030010DC, 0x0); // D0<br>mmio_write_32(0x030010E0, 0x0); // CMD<br>mmio_write_32(0x030010E4, 0x0); // CLK</pre><p>To switch a pin to <strong>GPIO</strong> mode <em>(or any other)</em>, we need to find its address from the pinmux table. For example, take pin <strong>A24</strong>. In the table, look for “XGPIOA[24]”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ry3cdnWfwMdMRz9n" /><figcaption>Pinmux for pin A24</figcaption></figure><p>The pin address is in the third column from the left — in this case, it’s <strong>0x03001060</strong>, and the mode value is <strong>0x03</strong> (from the last column).</p><p>In the U-Boot configuration, add a call to mmio_write_32(ADDRESS, VALUE) inside cvi_board_init:</p><pre>// GPIO A24 PINMUX<br>mmio_write_32(0x03001060, 0x03);</pre><p>Then, rebuild U-Boot and update the bootloader image on the board.</p><p><strong>Runtime Configuration from Linux<br></strong>If you don’t have time to rebuild and reflash the bootloader, you can reconfigure the pinmux from a running Linux system. These changes will be lost after a reboot (as U-Boot settings take over again), but it’s sufficient for testing.</p><p>To do this, use the devmem utility. It’s not included in the official image by default, but you can download the precompiled binary for the board <a href="https://raw.githubusercontent.com/ret7020/LicheeRVNano/refs/heads/master/Tools/devmem">here</a>.</p><p>Example for configuring pin <strong>A24</strong>:</p><pre>./devmem 0x03001060 b 0x03</pre><p>If successful, the output will look like:</p><pre>/dev/mem opened.<br>Memory mapped at address 0x3fcaa97000.<br>Value at address 0x3001060 (0x3fcaa97060): 0x3<br>Written 0x3; readback 0x3</pre><p>The pin is now configured as <strong>GPIO</strong>.</p><p><strong>Controlling GPIO via Linux sysfs<br></strong>You can control the GPIO pin using the Linux GPIO sysfs interface, which means writing values to specific files in the /sys/class/gpio directory. In Linux, pin A24 is GPIO number <strong>504</strong> (as per the pinout diagram).</p><p>Terminal interaction example:</p><pre># Export the GPIO pin to userspace<br>echo 504 &gt; /sys/class/gpio/export<br><br># Set the direction to output<br>echo out &gt; /sys/class/gpio/gpio504/direction<br><br># Set the pin HIGH<br>echo 1 &gt; /sys/class/gpio/gpio504/value<br><br># Set the pin LOW<br>echo 0 &gt; /sys/class/gpio/gpio504/value<br><br># Unexport the pin<br>echo 504 &gt; /sys/class/gpio/unexport</pre><p>To test, you can use a simple Bash script to generate a square wave (toggle signal):</p><pre>while true; do<br>     echo 0 &gt; /sys/class/gpio/gpio504/value<br>     sleep 0.1<br>     echo 1 &gt; /sys/class/gpio/gpio504/value<br>     sleep 0.1<br>done</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*bRFXogA1gek1u8fZ" /><figcaption>Oscilloscope check</figcaption></figure><p>Similarly, you can programmatically control the pin from C code.<br>Example code in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/GPIO">Projects/GPIO</a></p><p>For convenience, I wrote separate functions and placed them to the <a href="https://github.com/ret7020/LicheeRVNano/blob/master/Projects/GPIO/gpio.h">gpio.h</a> file (the main functions are below):</p><pre>int exportPin(int pin)<br>{<br> FILE *exportFile = fopen(&quot;/sys/class/gpio/export&quot;, &quot;w&quot;);<br> if (exportFile == NULL)<br>     return -1;<br> fprintf(exportFile, &quot;%d&quot;, pin);<br> fclose(exportFile);<br><br> return 0;<br>}<br><br>int unexportPin(int pin)<br>{<br> FILE *exportFile = fopen(&quot;/sys/class/gpio/unexport&quot;, &quot;w&quot;);<br> if (exportFile == NULL)<br>     return -1;<br> fprintf(exportFile, &quot;%d&quot;, pin);<br> fclose(exportFile);<br><br> return 0;<br>}<br><br>int setDirPin(int pin, const char *dir)<br>{<br> char directionPath[50];<br> snprintf(directionPath, sizeof(directionPath), &quot;/sys/class/gpio/gpio%d/direction&quot;, pin);<br> FILE *directionFile = fopen(directionPath, &quot;w&quot;);<br> if (directionFile == NULL)<br>     return -1;<br> fprintf(directionFile, dir);<br> fclose(directionFile);<br><br> return 0;<br>}<br><br>int writePin(int pin, int value)<br>{<br> char valuePath[50];<br> snprintf(valuePath, sizeof(valuePath), &quot;/sys/class/gpio/gpio%d/value&quot;, pin);<br> FILE *valueFile = fopen(valuePath, &quot;w&quot;);<br> if (valueFile == NULL)<br>     return -1;<br><br> if (value)<br>     fprintf(valueFile, &quot;1&quot;);<br> else<br>     fprintf(valueFile, &quot;0&quot;);<br> fflush(valueFile);<br> fclose(valueFile);<br><br> return 0;<br>}<br><br>int setPin(int pin, int value) // wrapper<br>{<br> return exportPin(pin) + setDirPin(pin, &quot;out&quot;) + writePin(pin, value) + releasePin(pin);<br>}</pre><p><strong>Example usage</strong> <em>(digitalTest.c file)</em>:</p><pre>#include &lt;stdio.h&gt;<br>#include &quot;gpio.h&quot;<br><br>int main(){<br> int pinId = 0;<br> int pass = 0;<br> scanf(&quot;%d&quot;, &amp;pinId);<br> printf(&quot;Working with pin: %d\n&quot;, pinId);<br> exportPin(pinId);<br> setDirPin(pinId, &quot;out&quot;);<br> writePin(pinId, 1);<br> printf(&quot;Write 1\n&quot;);<br> scanf(&quot;%d&quot;, &amp;pass);<br><br> writePin(pinId, 0);<br> printf(&quot;Write 0\n&quot;);<br> unexportPin(pinId);<br><br> return 0;<br>}</pre><pre>make build_digital</pre><p>The program reads a GPIO pin number from <strong>stdin</strong> (you must input the Linux GPIO number), writes a 1 to it, waits for any key press, then turns the pin off (writes 0) and releases it (correctly ends interaction with it via sysfs).</p><h4>PWM</h4><p>Example code can be found in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/GPIO">Projects/GPIO</a>.<br>Pin multiplexer configuration for PWM pins is similar to that for GPIO pins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JD2T-g_4O6LQrI6o" /><figcaption>PWM pins pinmux</figcaption></figure><p>From Linux, PWM interaction also occurs through the <strong>sysfs</strong> interface.<br><strong>Pin multiplexer configuration:</strong></p><pre>./devmem 0x03001068 b 0x2 # PWM 6<br>./devmem 0x03001064 b 0x2 # PWM 7</pre><p>Inside the <strong>SG2002</strong>, there are <strong>4 PWM</strong> controllers, each with <strong>4 channels</strong>. As a result, PWM pins are distributed among them as follows:</p><pre>pwmchip0   -&gt; PWM[0, 1, 2, 3]  <br>pwmchip4   -&gt; PWM[4, 5, 6, 7]  <br>pwmchip8   -&gt; PWM[8, 9, 10, 11]  <br>pwmchip12  -&gt; PWM[12, 13, 14, 15]</pre><p>The right column of the table provides the <strong>absolute</strong> PWM channel numbers, but when exporting a pin via sysfs, you must specify the <strong>relative channel number</strong> within its controller.</p><p>For example:</p><ul><li><strong>PWM 7</strong> belongs to pwmchip4 and has relative channel number 7-4=3</li><li><strong>PWM 6</strong> has relative number 2</li></ul><p><strong>Bash control example (PWM 6 and PWM 7):</strong></p><pre># PWM 6<br>echo 2 &gt; /sys/class/pwm/pwmchip4/export<br>echo 10000 &gt; /sys/class/pwm/pwmchip4/pwm2/period<br>echo 5000 &gt; /sys/class/pwm/pwmchip4/pwm2/duty_cycle<br>echo 1 &gt; /sys/class/pwm/pwmchip4/pwm2/enable<br><br># PWM 7<br>echo 3 &gt; /sys/class/pwm/pwmchip4/export<br>echo 10000 &gt; /sys/class/pwm/pwmchip4/pwm3/period<br>echo 2500 &gt; /sys/class/pwm/pwmchip4/pwm3/duty_cycle<br>echo 1 &gt; /sys/class/pwm/pwmchip4/pwm3/enable</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*0rrxszZjtHsbPhin" /><figcaption>Oscilloscope check</figcaption></figure><p>To control pwm pins from code I wrote two functions in <strong>gpio.h</strong>:</p><pre>int pwmSetParam(int pin, int val, int type){<br> // type == 0: set pwm period<br> // type == 1: set pwm duty cycle<br> // type == 2: set enable/disable (1/0)<br> char typeMap[3][15] = {&quot;period&quot;, &quot;duty_cycle&quot;, &quot;enable&quot;};<br> char valPath[50];<br> snprintf(valPath, sizeof(valPath), &quot;/sys/class/pwm/pwmchip4/pwm%d/%s&quot;, pin, typeMap[type]);<br> printf(&quot;Path: %s\n&quot;, valPath);<br><br> FILE *exportFile = fopen(valPath, &quot;w&quot;);<br> if (exportFile == NULL)<br>     return -1;<br> fprintf(exportFile, &quot;%d&quot;, val);<br> fclose(exportFile);<br><br> return 0;<br>}<br><br>int pwmUnexport(int pin){<br> FILE *exportFile = fopen(&quot;/sys/class/pwm/pwmchip4/unexport&quot;, &quot;w&quot;);<br> if (exportFile == NULL)<br>     return -1;<br> fprintf(exportFile, &quot;%d&quot;, pin);<br> fclose(exportFile);<br><br> return 0;<br><br>}</pre><p>These functions are hardcoded under <strong>pwmchip4</strong>, because the pins of other controllers are occupied by other functions, so there is no need to control them.</p><p><strong>Example usage:</strong></p><pre>#include &lt;stdio.h&gt;<br>#include &quot;gpio.h&quot;<br><br>int main(){<br>    int pin = 0, period = 0, dutyCycle = 0, enableStatus = 0; <br>    while (1) {<br>        printf(&quot;Pin (PWM_X, supported: 3 (pwm7) and 2 (pwm6)): \n&quot;);<br>        scanf(&quot;%d&quot;, &amp;pin);<br>        printf(&quot;Period; duty cycle; enable/disable: \n&quot;);<br>        scanf(&quot;%d %d %d&quot;, &amp;period, &amp;dutyCycle, &amp;enableStatus);<br>        printf(&quot;Export status: %d\n&quot;, pwmExport(pin));<br>        printf(&quot;Setting period: %d\n&quot;, pwmSetParam(pin, period, 0));<br>        printf(&quot;Setting duty cycle: %d\n&quot;, pwmSetParam(pin, dutyCycle, 1));<br>        printf(&quot;Setting enable: %d\n&quot;, pwmSetParam(pin, enableStatus, 2));<br>        pwmUnexport(pin);<br>    }<br><br>    return 0;<br>}</pre><pre>make build_pwm</pre><p>After starting the program, you need to enter the channel number of the fourth <strong>PWM</strong> controller (2 or 3, corresponding to pins <strong>PWM6</strong> and <strong>PWM7</strong> on the board) and the modulation settings.</p><h4>Interrupts</h4><p>Example code can be found in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/Interrupts">Projects/Interrupts</a> .</p><p>This example will show interrupt processing using Linux API (poll/select), i.e. without setting up full-fledged interrupts in device tree. The response speed to such interrupts is lower than that of full-fledged ones, since the pin status is actually polled, but for most tasks this is enough.</p><p>An example of code that tracks “rising” interrupts on pin <strong>A15</strong> <em>(in the official image it is configured as GPIO, so reconfiguring the multiplexer is not required)</em>:</p><pre>#include &lt;stdio.h&gt;<br>#include &lt;poll.h&gt;<br>#include &lt;pthread.h&gt;<br>#include &quot;gpio.h&quot;<br><br>int main()<br>{<br>    exportPin(GPIOA_15);<br>    setDirPin(GPIOA_15, &quot;in&quot;);<br>    setInterruptType(GPIOA_15, &quot;rising&quot;);<br><br>    char valuePath[50];<br>    snprintf(valuePath, sizeof(valuePath), &quot;/sys/class/gpio/gpio%d/value&quot;, GPIOA_15);<br>    int fd = open(valuePath, O_RDONLY | O_NONBLOCK);<br>    if (fd &lt; 0)<br>    {<br>     perror(&quot;Unable to open GPIO value file&quot;);<br>     return 1;<br>    }<br>    struct pollfd pfd;<br>    pfd.fd = fd;<br>    pfd.events = POLLPRI;<br><br>    // Read first &#39;init&#39; interrupt<br>    poll(&amp;pfd, 1, -1);<br>    lseek(fd, 0, SEEK_SET);<br>    char tmp[3];<br>    read(fd, tmp, sizeof(tmp));<br><br>    while (1)<br>    {<br>     int ret = poll(&amp;pfd, 1, -1);<br>     if (ret &gt; 0)<br>     {<br>      char buf[3];<br>      lseek(fd, 0, SEEK_SET);<br>      read(fd, buf, sizeof(buf));<br>      printf(&quot;Interrupt detected!\n&quot;);<br>     }<br>    }<br><br>    close(fd);<br><br>    return 0;<br>}</pre><p>This code uses pin setup function from <strong>gpio.h</strong> file:</p><pre>int setInterruptType(int pin, const char *type)<br>{<br> char interruptPath[50];<br> snprintf(interruptPath, sizeof(interruptPath), &quot;/sys/class/gpio/gpio%d/edge&quot;, pin);<br> FILE *interruptFile = fopen(interruptPath, &quot;w&quot;);<br> if (interruptFile == NULL)<br>     return -1;<br> fprintf(interruptFile, type);<br> fclose(interruptFile);<br><br> return 0;<br>}</pre><h4>UART</h4><p>Example code can be found in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/Interfaces/UART">Projects/Interfaces/UART</a>.</p><p>There are 4 UARTs on the board: <strong>UART0</strong> is a debug UART, <strong>UART1</strong> and <strong>UART2</strong> <em>(if Bluetooth is not used)</em> can be used for your own tasks, and <strong>UART3</strong> is only available if <strong>WiFi</strong> is turned off. Let’s look at pins <strong>A28</strong> and <strong>A29</strong>. The multiplexer for these pins has an interesting feature: both (<strong>1</strong> and <strong>2</strong>) UARTs can work on them:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6Ar38_7ThFC0GRI_" /></figure><p>On the pinout, pins <strong>A28</strong> and <strong>A29</strong> are marked as <strong>UART2</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/0*xRJbqeXKMwc8Wc9o" /></figure><p>At the same time, no one forbids configuring them so that a signal from the hardware <strong>UART1</strong> goes to them. This fact must be taken into account when configuring multiplexer.</p><p>If there is a need to use both UARTs at once, then you also need addresses for pins <strong>A19</strong> and <strong>A18</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*D_gp8arS6iQ9IuG8" /><figcaption>BTW, only UART1 (and other peripherals) can use them.</figcaption></figure><p>Example of using <strong>UART1</strong> on pins <strong>A18</strong> (RX), <strong>A19</strong> (TX):</p><pre># Pinmux configuration<br>./devmem 0x03001068 b 0x6 # GPIOA 18 UART1 RX<br>./devmem 0x03001064 b 0x6 # GPIOA 19 UART1 TX</pre><p>Example uart data sending source code:</p><pre>#include &lt;stdio.h&gt;<br>#include &lt;stdlib.h&gt;<br>#include &lt;string.h&gt;<br>#include &lt;fcntl.h&gt;<br>#include &lt;termios.h&gt;<br>#include &lt;unistd.h&gt;<br><br>#define UART_PATH &quot;/dev/ttyS1&quot;<br><br>/* If you use official OS img<br> Configure system for UART usage<br>    devmem 0x03001068 b 0x6 # GPIOA 18 UART1 RX<br>    devmem 0x03001064 b 0x6 # GPIOA 19 UART1 TX<br><br>    You can get devmem tool from Tools/ directory of this repo<br> */<br><br>int main()<br>{<br>    printf(&quot;UART test %s\n&quot;, UART_PATH);<br>    char serialPort[] = UART_PATH;<br>    char txBuf[] = &quot;abcd&quot;;<br>    struct termios tty;<br>    ssize_t writeLen;<br>    int serialFd;<br>    char rxBuffer[256];<br>    int bytesRead;<br><br>    serialFd = open(serialPort, O_RDWR | O_NOCTTY);<br>    printf(&quot;%d\n&quot;, serialFd);<br><br>    memset(&amp;tty, 0, sizeof(tty));<br><br>    // Setting baud<br>    cfsetospeed(&amp;tty, B115200);<br>    cfsetispeed(&amp;tty, B115200);<br><br>    // Generic flags<br>    tty.c_cflag &amp;= ~PARENB;<br>    tty.c_cflag &amp;= ~CSTOPB;<br>    tty.c_cflag &amp;= ~CSIZE;<br>    tty.c_cflag |= CS8;<br><br>    while (1)<br>    {<br>     writeLen = write(serialFd, txBuf, sizeof(txBuf));<br>     printf(&quot;%d\n&quot;, writeLen);<br>     usleep(50000);<br>    }<br><br>    return 0;<br>}</pre><p>And check on oscilloscope or ttl-usb adapter:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*thdwRosObHwIVtB4" /></figure><h4>SPI</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*ShwfPBLWXHC5z-Rs" /><figcaption>SPI MOSI and CLK</figcaption></figure><p>Example code can be found in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/Interfaces/SPI">Projects/Interfaces/SPI</a>.</p><p><strong>SG2002</strong> has 3 hardware <strong>SPI</strong>. Only one is brought out on the board — <strong>SPI2</strong>, which is not usable when using WiFi. <strong>SPI4 </strong>can be emulated via the BitBang driver. This article will consider the use of hardware <strong>SPI2</strong>, with the WiFi module disabled.</p><p><strong>Pinmux configuration:</strong></p><pre>/etc/init.d/S30wifi stop # correct wifi module stop if using<br>./devmem 0x030010D0 b 0x1 # GPIO P18 CS<br>./devmem 0x030010DC b 0x1 # GPIO P21 MISO<br>./devmem 0x030010E0 b 0x1 # GPIO P22 MOSI<br>./devmem 0x030010E4 b 0x1 # GPIO P23 SCK</pre><p>If you have a board with WiFi, you can restore the functionality of the WiFi module without rebooting:</p><pre>./devmem 0x030010D0 b 0x0<br>./devmem 0x030010DC b 0x0<br>./devmem 0x030010E0 b 0x0<br>./devmem 0x030010E4 b 0x0<br>/etc/init.d/S30wifi start</pre><p>For a quick check, you can use the pre-installed <strong>spidev_test</strong> utility, also connecting <strong>MISO</strong> (P21) and <strong>MOSI</strong> (P22), creating some kind of physical “loopback”:</p><pre>spidev_test -D /dev/spidev2.0 -p &quot;hello, world!&quot; -v</pre><p>Result will be like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sfV6kVafSbf43Fe0" /><figcaption>Maximum speed at which messages processed correctly (TX and RX) — 93 MHz</figcaption></figure><p><strong>Example code for C/C++ interaction with SPI:</strong></p><pre>#include &lt;stdio.h&gt;<br>#include &lt;stdlib.h&gt;<br>#include &lt;stdint.h&gt;<br>#include &lt;fcntl.h&gt;<br>#include &lt;unistd.h&gt;<br>#include &lt;linux/spi/spidev.h&gt;<br>#include &lt;sys/ioctl.h&gt;<br><br>#define SPI_DEVICE_PATH &quot;/dev/spidev2.0&quot;<br><br>int main()<br>{<br>    int spi_file;<br>    uint8_t tx_buffer[5] = {1, 2, 3, 4, 5};<br>    uint8_t rx_buffer[5];<br><br>    // Open the SPI device<br>    if ((spi_file = open(SPI_DEVICE_PATH, O_RDWR)) &lt; 0)<br>    {<br>     perror(&quot;Failed to open SPI device&quot;);<br>     return -1;<br>    }<br><br>    uint8_t mode = SPI_MODE_0;<br>    uint8_t bits = 8;<br>    if (ioctl(spi_file, SPI_IOC_WR_MODE, &amp;mode) &lt; 0)<br>    {<br>     perror(&quot;Failed to set SPI mode&quot;);<br>     close(spi_file);<br>     return -1;<br>    }<br><br>    struct spi_ioc_transfer transfer = {<br>     .tx_buf = (unsigned long)tx_buffer,<br>     .rx_buf = (unsigned long)rx_buffer,<br>     .len = sizeof(tx_buffer),<br>     .speed_hz = 1000000,  // SPI speed in Hz<br>     .delay_usecs = 0,<br>     .bits_per_word = 8,<br>    };<br>    <br>    if (ioctl(spi_file, SPI_IOC_MESSAGE(1), &amp;transfer) &lt; 0)<br>    {<br>     perror(&quot;Failed to perform SPI transfer&quot;);<br>     close(spi_file);<br>     return -1;<br>    }<br>    for (uint8_t i = 0; i &lt; 5; i++) {printf(&quot;%d &quot;, rx_buffer[i]);}<br>    printf(&quot;\n&quot;);<br><br><br>    close(spi_file);<br><br>    return 0;<br>}</pre><p>The program sends 5 bytes <em>(1, 2, 3, 4, 5)</em> and outputs the answer. In the case of connected <strong>MOSI</strong> and <strong>MISO</strong> the answer will also be <em>“1 2 3 4 5”</em>.</p><h4>I2C</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FyFsN0_y15hy9lso" /></figure><p>Example code can be found in <a href="https://github.com/ret7020/LicheeRVNano/tree/master/Projects/Interfaces/I2C">Projects/Interfaces/I2C</a></p><p><strong>SG2002</strong> has 5 hardware <strong>I2C</strong>. Only 2 are brigned on the board: 1, 3. Their pins also intersect with the <strong>WiFi</strong> pins, so experiments with <strong>I2C</strong> will be carried out with the <strong>WiFi</strong> module turned off. <strong>I2C-5</strong> can be emulated in software via the BitBang driver.</p><p>As an example, we will demonstrate the work with the <strong>AHT20</strong> module (humidity and temperature sensor) + <strong>BMP280</strong> (barometer). It will be connected to <strong>I2C-1</strong>, the corresponding multiplexer configuration:</p><pre>/etc/init.d/S30wifi stop # correct WiFi stop<br><br>./devmem 0x030010D0 b 0x2 # GPIO P18 SCL<br>./devmem 0x030010DC b 0x2 # GPIO P21 SDA</pre><p>Now, using <strong>i2cdetect</strong>, you can find out the addresses of modules:</p><pre>i2cdetect -ry 1</pre><p>Output will be like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/0*v6A5EcMnGyu3W0EJ" /></figure><p><strong>0x38</strong> — AHT20 address<br><strong>0x77</strong> — BMP280 address</p><p>For manual testing you can use the following commands:</p><ul><li>i2cget — read a register value</li><li><strong>i2cset</strong> — write a value to a register</li><li><strong>i2cdump</strong> — read all device registers</li></ul><p>Example of C/C++ temperature reading with <strong>AHT20</strong>:</p><pre>#include &lt;stdio.h&gt;<br>#include &lt;stdlib.h&gt;<br>#include &lt;fcntl.h&gt;<br>#include &lt;unistd.h&gt;<br>#include &lt;linux/i2c-dev.h&gt;<br>#include &lt;sys/ioctl.h&gt;<br>#include &lt;stdint.h&gt;<br>#include &lt;time.h&gt;<br><br>#define I2C_BUS &quot;/dev/i2c-1&quot;  // bus<br>#define AHT20_ADDR 0x38    // AHT20 I2C address<br><br>#define CMD_TRIGGER_MEASURE 0xAC<br>#define CMD_INIT 0xBE<br>#define CMD_SOFT_RESET 0xBA<br><br>int open_i2c_device() {<br> int file = open(I2C_BUS, O_RDWR);<br> if (file &lt; 0) {<br>     exit(1);<br> }<br> if (ioctl(file, I2C_SLAVE, AHT20_ADDR) &lt; 0) {<br>     exit(1);<br> }<br> return file;<br>}<br><br>void aht20_reset(int file) {<br> uint8_t cmd = CMD_SOFT_RESET;<br> if (write(file, &amp;cmd, 1) != 1) {<br>     exit(1);<br> }<br> usleep(20000);<br>}<br><br>void aht20_init(int file) {<br> uint8_t cmd[3] = {CMD_INIT, 0x08, 0x00};<br> if (write(file, cmd, 3) != 3) {<br>     exit(1);<br> }<br> usleep(10000);<br>}<br><br>void aht20_start_measurement(int file) {<br> uint8_t cmd[3] = {CMD_TRIGGER_MEASURE, 0x33, 0x00};<br> if (write(file, cmd, 3) != 3) {<br>     exit(1);<br> }<br> usleep(80000);<br>}<br><br>uint32_t aht20_read_raw_temperature(int file) {<br> uint8_t data[6];<br><br> if (read(file, data, 6) != 6) {<br>     exit(1);<br> }<br> uint32_t raw_temp = ((data[3] &amp; 0x0F) &lt;&lt; 16) | (data[4] &lt;&lt; 8) | data[5];<br><br> return raw_temp;<br>}<br><br>float convert_temperature(uint32_t raw_temp) {<br> return ((raw_temp * 200.0) / 1048576.0) - 50.0;<br>}<br><br>int main() {<br> int file = open_i2c_device();<br><br> aht20_reset(file);<br> aht20_init(file);<br><br> aht20_start_measurement(file);<br><br> uint32_t raw_temp = aht20_read_raw_temperature(file);<br> float temperature = convert_temperature(raw_temp);<br><br> printf(&quot;Temperature: %.2f°C\n&quot;, temperature);<br><br> close(file);<br> return 0;<br>}</pre><h3>Next steps</h3><p>In next article we will work with CSI camera, connected to board.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a10e33b0e110" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LicheeRV Nano — AI board; First steps]]></title>
            <link>https://medium.com/@ret7020/licheerv-nano-ai-board-first-steps-d05e7999dd29?source=rss-c764fa3dece2------2</link>
            <guid isPermaLink="false">https://medium.com/p/d05e7999dd29</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[edge-ai]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[electronics]]></category>
            <category><![CDATA[iot]]></category>
            <dc:creator><![CDATA[Stephan Zhdanov]]></dc:creator>
            <pubDate>Sat, 08 Feb 2025 20:47:07 GMT</pubDate>
            <atom:updated>2025-02-08T20:48:44.341Z</atom:updated>
            <content:encoded><![CDATA[<h3>LicheeRV Nano — AI board; First steps</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QnaSUREYs5QayRhl" /></figure><p>This article discusses development for the LicheeRV Nano debug board — a compact device about the size of two coins yet packed with impressive capabilities.</p><p>The board can run both <strong>Linux</strong> and <strong>FreeRTOS</strong> simultaneously, perform neural network inference (including running YOLO and LLama2.c) thanks to its built-in NPU with 1 TOPS performance, and control various peripherals such as GPIO, I2C, UART, SPI, a CSI camera, Wi-Fi, Bluetooth, and Ethernet.</p><p>This is the first part of the article, serving some base info about the board and setup guide. The following articles will contain instructions on developing for the board (Interfaces, NPU and others).</p><h3>Overview</h3><h3>SG2002</h3><p>The board is built around the <strong>SG2002 SoC</strong>. A full list of hardware capabilities is illustrated in the diagrams below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_L_7_mWIE9EOAf8E" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*T_hCUuOQwA4H98o5" /></figure><p>The most striking feature is the presence of four separate cores: two RISC-V cores with different clock speeds (1 GHz and 700 MHz), one ARM Cortex-A53 core, and one 8051 core. Developers can choose the architecture of the “main” core (the one that will run Linux) — either ARM or RISC-V. These cores cannot operate simultaneously; the selection is made by pulling a specific chip pin to GND or 1.8V before startup. However, the developers of the LicheeRV Nano (Sipeed) have predetermined this choice: the corresponding pin is initially pulled up to a 1.8V line, causing the board to boot using RISC-V.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/0*1J3t7N9YGKHCr6x4" /></figure><p>For comparison, on the <strong>Milk-V Duo 256</strong> board, which uses the same SoC, the <strong>CPU_SEL</strong> line is routed to a separate pin, allowing manual processor selection. Recently, an official OS image for the ARM architecture was released, but it is not yet fully functional. Overall, the focus is on the RISC-V architecture, so all further development for the board in my articles will be based on RISC-V.</p><p><strong>Second RISC-V Core<br></strong>This core is intended for use with an RTOS and already has a FreeRTOS port available in official LicheeRV Nano Buildroot repo. This creates an architecture that is useful in robotics: the high-performance core running Linux and handles tasks such as navigation, localization, and video stream processing from a camera, while the FreeRTOS core manages PID controllers, odometry, and peripheral control — motors, servos, and other actuators. This setup enables a compact solution that replaces the traditional combination of a separate microprocessor and microcontroller (e.g., Raspberry Pi + STM32/Atmega).</p><p><strong>8051 Core<br></strong>This core operates at a low frequency and consumes significantly less power than the main cores. It has its own power management system and can control the power consumption of other cores, enabling a power-saving (sleep) mode.</p><p><strong>NPU and CSI<br></strong>The SoC is equipped with a neural processing unit (NPU) capable of 1 TOPS performance for INT8 computations. We will use it for running YOLO, a neural network that performs object detection on frames captured by a CSI camera.</p><h3>LicheeRV Nano</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kdRzKhn3-bD7Qat3" /></figure><p>There are four versions of the board: one with an Ethernet port, one with a Wi-Fi transceiver, one with both Wi-Fi and Ethernet, and a basic version. The board itself is the same across all versions, so if desired, you can solder a Wi-Fi transceiver or an Ethernet connector onto the basic version. I have the Wi-Fi version (which also includes Bluetooth).</p><p><strong>CPU</strong></p><p><strong>SG2002</strong>:</p><ul><li>1GHz RISC-V C906 / ARM A53</li><li>700MHz RISC-V C906</li><li>25–300 Mhz 8051</li></ul><p><strong>NPU</strong></p><p>1TOPS INT8, BF16 support</p><p><strong>Memory</strong></p><p>256 Mb DDR3</p><p><strong>IO</strong></p><p>2 x 14 pins <em>2.54 mm</em></p><p><strong>Size</strong></p><p>22.86 x 35.56 <em>mm</em></p><h3>Documentation and other information</h3><p>The documentation for the board can be found <a href="https://wiki.sipeed.com/hardware/en/lichee/RV_Nano/1_intro.html">here</a>. It’s worth noting that the Chinese version contains more detailed information in the “Peripherals” section (with more details on working with various interfaces). I am gradually translating this section into English, but the process of accepting Pull Requests and updating the website takes some time, so you can refer to my <a href="https://github.com/ret7020/sipeed_wiki/blob/main/docs/hardware/en/lichee/RV_Nano/5_peripheral.md">fork</a>. Alternatively, you can simply use the Chinese version along with a translator. And source for all my sample programs for board you can find in <a href="https://github.com/ret7020/LicheeRVNano">GitHub repo</a>.</p><p>There is also the <a href="https://milkv.io/ru/docs/duo/getting-started/duo256m">Milk-V Duo 256</a> board, which is also based on the SG2002 SoC. Their documentation and forum are more developed, so some useful information can be found there.</p><h3>Main frameworks mentioned in this article:</h3><ul><li><strong>CVI TDL SDK</strong> — A set of pre-built neural network algorithms that run inference on the NPU.</li><li><strong>TPU SDK</strong> — A lower-level SDK for interacting with the NPU and SoC peripherals.</li><li><strong>MMF (Multimedia Framework)</strong> — A framework with a unified API for working with video, images, and audio on the SoC.</li></ul><h3>Energy consumption</h3><p>The <strong>SG2002</strong> datasheet provides limited information on power consumption: <strong>“1080P + Video encode + AI: ~500mW.”</strong> Below, I will present power consumption graphs for the board in different usage scenarios when powered via USB Type-C.</p><p>Board boot-up and file download via wget over HTTP from another computer on the local network:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*Iyg-e26TIT61YBL_" /></figure><p>Sending large file via python http.server to computer from local network, that download file via wget:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*3M8dRV8rNWhzWNrx" /></figure><p><strong>CSI camera streaming into MJPEG</strong> via http for single client:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*TB2UG7Dg6dHbhigd" /></figure><p><strong>YOLOv8n </strong>COCO model inference on <strong>640x640 images</strong>. Other yolo models have same power consumption:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*FjBbtZDRB_yvLFac" /></figure><p><strong>LLama </strong>(<em>stories15M</em>) inference on <strong>RISC-V CPU</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*jAhi2EYm3wKE6mVR" /></figure><p><strong>WiFi</strong> transceiver turn off:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*P8g0se3B83opmcq9" /></figure><h3>First steps with a board</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wls9JU08HXpAcHPW" /></figure><h3>Installing OS</h3><p>Official Buildroot image can be downloaded from <a href="https://github.com/sipeed/LicheeRV-Nano-Build/releases">GitHub</a>. I used version <a href="https://github.com/sipeed/LicheeRV-Nano-Build/releases/tag/20241021"><strong>20241021</strong></a> (all examples were also tested on the newer <a href="https://github.com/sipeed/LicheeRV-Nano-Build/releases/tag/20250114"><strong>20250114</strong></a> version). There is also an experimental Debian-based image available for the board, but it lacks drivers for the NPU and CSI camera. This article will use the <strong>Buildroot</strong> image since it is easy to configure and is considered a standard for Embedded Linux.</p><p>The operating system boots from an <strong>SD card</strong>. To write the image from Linux, use the dd command:</p><pre>xzcat &quot;НАЗВАНИЕ.img.xz&quot; | sudo dd of=/dev/sdX conv=sync status=progress</pre><p>If the image archive is already extracted (just img file), use:</p><pre>cat &quot;НАЗВАНИЕ.img&quot; | sudo dd of=/dev/sdX conv=sync status=progress</pre><p>Replace <strong>/dev/sdX</strong> with the correct path to your SD card. The card must be <strong>fully formatted</strong> with an <strong>MBR (msdos) partition table</strong> and should contain <strong>no partitions</strong> (just unallocated space).</p><p>On <strong>Windows</strong>, you can use <strong>Rufus</strong> or <strong>Balena Etcher</strong>.</p><p>I have the <strong>Wi-Fi version</strong> of the board, so immediately after flashing the image, I can configure Wi-Fi credentials to enable <strong>SSH access</strong> (which starts automatically).</p><p>To do this, mount the <strong>rootfs</strong> partition (after flashing the SD card, it will have two partitions: <strong>boot</strong> and <strong>rootfs</strong>) and edit the /etc/wpa_supplicant.conf file as follows:</p><pre>ctrl_interface=/var/run/wpa_supplicant<br>ap_scan=1<br>network={<br>  ssid=&quot;NAME&quot;<br>  psk=&quot;PASSWORD&quot;<br>}</pre><p>Replace <strong>NAME</strong> and <strong>PASSWORD</strong> with your Wi-Fi credentials.</p><p>The board’s Wi-Fi transceiver supports <strong>5G networks</strong>, and the included antenna (<strong>N2425D</strong>) is dual-band, so you can use <strong>5G Wi-Fi</strong>.</p><p>Now you can <strong>remove the SD card</strong>, insert it into the board, and power it on.</p><ul><li>If everything is set up correctly, both LEDs will turn on initially (<strong>red = power, blue = user</strong>).</li><li>After a short time, the <strong>blue LED will start blinking</strong>.</li><li>If the board is booted <strong>without an SD card</strong> or with an <strong>incorrect image</strong>, the <strong>blue LED will remain solid</strong>.</li></ul><p>To find the board’s IP address on your local network, use nmap (or any other network scanner):</p><pre># Scan for all active devices in the network<br>sudo nmap -sN 192.168.1.0/24<br><br># Scan specifically for devices with an open SSH (port 22)<br>nmap -sV -p 22 192.168.1.0/24</pre><p>Once you find the IP address, connect via SSH (<strong>default username and password: root</strong>):</p><pre>ssh root@192.168.1.21</pre><p><strong>Using UART0 for Boards Without Ethernet or Wi-Fi</strong></p><p>If your board does <strong>not have Ethernet or Wi-Fi</strong>, you can interact with it via <strong>debug UART0</strong>.</p><p>To do this, connect a <strong>USB-to-TTL adapter</strong> to the following pins:</p><ul><li><strong>A17 (Board RX → Adapter TX)</strong></li><li><strong>A16 (Board TX → Adapter RX)</strong></li><li><strong>Ground (Common GND connection)</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/0*KXrgOasDWfjTAEr0" /></figure><p>Next, you can interact with the board using a terminal program (or screen on Linux) at a baud rate of <strong>115200</strong>:</p><pre>screen /dev/ttyUSB0 115200</pre><p>It is also possible to work with the board via <strong>ACM (/dev/ttyACM0)</strong> and over a <strong>local network via USB</strong>. However, I found <strong>SSH over Wi-Fi</strong> and <strong>debug UART0</strong> to be the most convenient methods. The board also sends additional debug information via UART0, which can sometimes be useful.</p><p>The board has only <strong>256MB of RAM</strong>, <strong>128MB</strong> of which is reserved for the <strong>MIPI-CSI camera and display</strong>, which may not be enough for some tasks. This issue can be solved by adding <strong>swap memory</strong>.</p><p>You can configure swap directly in <strong>Buildroot</strong>, so the final image is preconfigured with a swap file, but the easiest way is to <strong>set it up manually</strong> inside the running OS. The necessary utilities are included out-of-the-box with <strong>BusyBox</strong>:</p><pre>fallocate -l 1G /swapfile # Adjust the size as needed<br>chmod 600 /swapfile<br>mkswap /swapfile<br>swapon /swapfile</pre><p>To <strong>enable swap automatically</strong> on boot, add it to <strong>fstab</strong>:</p><pre>echo &#39;/swapfile none swap sw 0 0&#39; | tee -a /etc/fstab</pre><p>After rebooting, swap will be <strong>activated automatically</strong>.</p><p>The next sections of this article will include example programs using <strong>OpenCV Mobile</strong> and <strong>TDL SDK</strong>. These libraries are dynamically linked during compilation, so the correct versions must be present on the board.</p><p>Using incorrect libraries can cause issues such as:</p><ul><li><strong>TDL SDK running very slowly</strong></li><li><strong>Model processing failure for cv181x</strong></li><li><strong>Detection of over 2000 classes in a single image</strong></li><li><strong>Conflicts between OpenCV Mobile and TDL SDK</strong></li></ul><p>I have put together all the necessary libraries into <a href="https://drive.google.com/file/d/1nhWBeKPAJ9O-7zXrXu0uMwNdArOiBBLm/view?usp=drive_link">an archive</a>. To set them up:</p><pre># On your computer<br>scp required_libs.zip root@192.168.X.X:/root<br># On the board<br>cd /root<br>unzip required_libs.zip<br>export LD_LIBRARY_PATH=/root/libs_patch/lib:/root/libs_patch/middleware_v2:/root/libs_patch/middleware_v2_3rd:/root/libs_patch/tpu_sdk_libs:/root/libs_patch:/root/libs_patch/opencv</pre><p>To <strong>automate</strong> setting the correct LD_LIBRARY_PATH, add the export command to the end of <strong>/etc/profile</strong>:</p><pre>echo &quot;export LD_LIBRARY_PATH=/root/libs_patch/lib:/root/libs_patch/middleware_v2:/root/libs_patch/middleware_v2_3rd:/root/libs_patch/tpu_sdk_libs:/root/libs_patch:/root/libs_patch/opencv&quot; | tee -a /etc/profile</pre><h3>Custom Buildroot</h3><p>The <strong>official build</strong> is sufficient for initial simple tests with the board, but for more complex tasks, the image needs to be <strong>built from source</strong>. Additionally, the official image contains <strong>a lot of unnecessary components</strong> (it is <strong>1.2 GB</strong>, which is quite large). For example, it includes <strong>Python</strong> and a bunch of <strong>unnecessary libraries</strong> for it. Using Python on this board is <strong>not practical</strong> due to limited resources.</p><p>The image is also <strong>universal</strong> for all board versions, meaning that if you don’t use <strong>Wi-Fi or Ethernet</strong>, you still have drivers for them in the image. Ideally, <strong>Buildroot</strong> should be configured and built <strong>from scratch</strong> for a specific set of tasks, but this takes a lot of time. Therefore, in this article, we will focus on <strong>modifying the official image</strong> instead.</p><p>The source code for the official image is available in a <a href="https://github.com/sipeed/LicheeRV-Nano-Build"><strong>GitHub repository</strong></a>. The easiest way to build it is <strong>inside a Docker container</strong>, which helps avoid dependency issues related to the <strong>host system, compilers, and build tools</strong>.</p><p>Clone the source code and build the Docker container:</p><pre>git clone https://github.com/sipeed/LicheeRV-Nano-Build - depth=1<br>cd LicheeRV-Nano-Build<br>git clone https://github.com/sophgo/host-tools - depth=1<br>cd host/ubuntu<br>docker build -t licheervnano-build-ubuntu .</pre><p>Running the Docker container:</p><pre>docker run -v /path/to/LicheeRV-Nano-Build/:/workspace -it licheervnano-build-ubuntu /bin/bash</pre><p>Replace “/path/to/LicheeRV-Nano-Build/” with the actual path to the <strong>LicheeRV-Nano-Build</strong> Git repository you downloaded earlier.</p><p>Once the container starts, it will have a <strong>/workspace</strong> directory where <strong>Buildroot is built</strong>. This directory is <strong>synced</strong> with the LicheeRV-Nano-Build folder on your host system, so after stopping the container, <strong>all changes and cached binaries will be preserved</strong>.</p><p>To build the full image, run the following commands inside the <strong>Docker container</strong>:</p><pre>cd /workspace # change to dir with sources<br>source build/cvisetup.sh # env setup<br>defconfig sg2002_licheervnano_sd # select config for RISC-V core<br>git config - global - add safe.directory /workspace<br>build_all</pre><p>The <strong>first build will take a long time</strong>, but subsequent builds will be much <strong>faster</strong> since only modified components will be recompiled.</p><p>After the build is complete, the <strong>ready-to-flash image</strong> will be located in:</p><p>/workspace/install/soc_sg2002_licheervnano_sd/images</p><p>If you only need to rebuild specific components of the system, you can use the following commands:</p><pre>build_fsbl # fsbl(First Stage Bootloader), outputs fip.bin<br>build_uboot # Uboot<br>build_rtos # FreeRTOS compilation<br>build_kernel # Linux kernel compilation</pre><p>Configuration of individual parts of <strong>Buildroot</strong> will be covered in detail in later sections. However, here’s a quick guide on <strong>where to find and modify settings</strong>.</p><p>Inside the repository root (/workspace in the Docker container), you can open <strong>menuconfig</strong> for general system settings:</p><pre>menuconfig</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*h06fIzXU5zTPZv9E" /></figure><p>This allows you to <strong>customize</strong> the build settings, drivers, and included components before compiling the image.</p><p>Save changes into file:</p><p>/workspace/build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig</p><p>In <strong>/workspace/buildroot </strong>dir you can execute <em>make menuconfig </em>for configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vzsSqGl1YE8qKugg" /></figure><p>Save after edit into:</p><p><em>/workspace/buildroot/configs/cvitek_SG200X_musl_riscv64_defconfig</em></p><p>Kernel sources located in dir <strong>/workspace/linux_5.10</strong>, that you can configure via <strong>menuconfig</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LSlFvoNh8cKecR0p" /></figure><p>Save after edit into file: <br><em>/workspace/build/boards/sg200x/sg2002_licheervnano_sd/linux/sg2002_licheervnano_sd_defconfig</em></p><p>PINMUX configuration can be done before Linux starts via <strong>U-Boot</strong> in file <em>/workspace/build/boards/sg200x/sg2002_licheervnano_sd/u-boot/cvi_board_init.c</em>. More details about this process will be in the next articles about this board.</p><p>Device Tree description located in file: <em>/workspace/build/boards/sg200x/sg2002_licheervnano_sd/dts_riscv/sg2002_licheervnano_sd.dts.</em></p><h3>Next steps</h3><p>In next article I will show how to use GPIO pins, interact with UART, SPI, I2C from Linux.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d05e7999dd29" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>