<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[vicuesoft-techblog - Medium]]></title>
        <description><![CDATA[ViCueSoft.com Technology Blog - Medium]]></description>
        <link>https://medium.com/vicuesoft-techblog?source=rss----a7648e30fa7f---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>vicuesoft-techblog - Medium</title>
            <link>https://medium.com/vicuesoft-techblog?source=rss----a7648e30fa7f---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 12:12:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/vicuesoft-techblog" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Yet another 5 practices of CI for Web-App’s deployment.]]></title>
            <link>https://medium.com/vicuesoft-techblog/yet-another-5-practices-of-ci-for-web-apps-deployment-e98d1df57dfa?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/e98d1df57dfa</guid>
            <category><![CDATA[gitlab-ci]]></category>
            <category><![CDATA[cloud-services]]></category>
            <dc:creator><![CDATA[Valery Zimichev]]></dc:creator>
            <pubDate>Tue, 24 Jan 2023 06:42:48 GMT</pubDate>
            <atom:updated>2023-01-24T06:42:48.374Z</atom:updated>
            <content:encoded><![CDATA[<p>Hello, reader! In this article, I want to collect for myself the best practices of deployment that I found useful. Some of them are widely used, some I have not encountered in the wild and invented by myself. I hope this article will be useful for you as well, as it’s not often you get to show the inner workings of projects — here’s your chance!</p><blockquote>In this article, I am not insisting on the singularity of these practices; everything has its shortcomings.</blockquote><p><strong>So, Practice #1: </strong><em>Create an infrastructure project and use submodules.</em></p><p>Git has the ability to create nested projects. I find this super useful, especially when projects become more complex — when you deepen their structure. First you have a small Python web application, which you debug locally and bring to some condition. Then you need to manage the database directly from the admin page, and you have two projects. This means that there is already a specific relationship between them — someone has to run them. An infrastructure project appears in the group of projects.</p><p>It also allows you to tune CI once, only in an infrastructure project. The resulting structure is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/1*N52--Jyh65hXbnq-3Vn0rg.png" /><figcaption>Yes, I also want to launch my web application locally. That’s why docker-compose is here.</figcaption></figure><p><strong>Practice #2.</strong> <em>Update your submodules with CI.</em></p><p>It’s common for developers and testers to not be super cool with the infrastructural intricacies — make their life easier for them, make a button! Usually, one person is in charge of the submodule, and another person on the team organizes the rollout. Create an update trigger in a submodule, for example, in the frontend submodule. Then add a job in the yaml to the infrastructure project, as in the example.</p><p>For submodule project:</p><pre>#.gitlab-ci.yml for submodule (frontend example)<br>stages:<br> - upd-submodule <br><br>variables:<br>  GIT_STRATEGY: clone<br><br>.update:<br>  stage: upd-submodule<br>  variables:<br>    SUBMODULE_PATH: &#39;frontend/${CI_PROJECT_NAME}&#39;<br>    SUBMODULE_NAME: &#39;${CI_PROJECT_NAME}&#39;<br>    SUBMODULE_BRANCH: &#39;${CI_COMMIT_BRANCH}&#39;<br>    SUBMODULE_SHA: &#39;${CI_COMMIT_SHA}&#39;<br>    UPDATE_SUBMODULE: &#39;true&#39;<br>  trigger:<br>    branch: &#39;${TARGET_BRANCH}&#39;<br>    project: web_site/site-cluster<br>    strategy: depend<br>  when: manual<br><br>update-master:<br>  variables:<br>    TARGET_BRANCH: master<br>  extends:<br>    - .update</pre><p>And for infrastructure project:</p><pre>#.gitlab-ci.yml for infrastructure project<br>update:submodule:<br>  stage: pre-build<br>  rules:<br>    - if: $UPDATE_SUBMODULE == &quot;true&quot;<br>  tags:<br>    - linux<br>  before_script:<br>    - git config --global user.name &quot;${GITLAB_USER_NAME}&quot;<br>    - git config --global user.email &quot;${GITLAB_USER_EMAIL}&quot;<br>  script:<br>    - echo &quot;Updating ${SUBMODULE_NAME} to ${SUBMODULE_SHA} of ${SUBMODULE_BRANCH} branch...&quot;<br>    - cd ${SUBMODULE_PATH}<br>    - git pull origin ${SUBMODULE_BRANCH}<br>    - git checkout ${SUBMODULE_SHA}<br>    - cd ..<br>    - git add ${SUBMODULE_NAME}<br>    - git commit -m &quot;upd ${SUBMODULE_NAME}&quot;<br>    - git push https://root:$ACCESS_TOKEN@$CI_SERVER_HOST:$CI_SERVER_PORT/$CI_PROJECT_PATH.git HEAD:${CI_COMMIT_REF_NAME}</pre><p>It remains to remember to define the ACCESS_TOKEN from “Settings”&gt;”Access Tokens” to “Settings”&gt;”CI/CD”&gt;”Variables&quot;.</p><p><strong>Practice #3.</strong> <em>Use workflows to implement multiple pipelines in one project.</em></p><p>I assume that even without submodule updates, your CI has a lot to do. Well, then use workflows or mimic them.</p><p>A. Create a separate yaml for updates because you don’t need it in the main pipeline. <br>B. Define a variable only with the definition of which the update pipeline will be active.<br>C. Separate the rules. If you use self-hosted GitLab &gt; 14.2 add rules to <strong>include</strong> following this scheme:</p><pre>#.gitlab-ci.yml<br>include:<br>  - local: .gitlab-ci-main.yml<br>    rules:<br>      - if: $UPDATE_SUBMODULE != &quot;true&quot;<br>  - local: .gitlab-ci-subs.yml<br>    rules:<br>      - if: $UPDATE_SUBMODULE == &quot;true&quot;</pre><p>If not, add rules to every job definition.</p><p>If the pipeline assumes a specific commit name, use a regular expression. If it assumes a specific event, such as push, use $CI_PIPELINE_SOURCE.</p><p>Practice #4. <em>From git submodule to production in one click by modifying autocreated pipelines.</em></p><p>Using practices 1–3 described earlier, we’ve come very close to running a configurable pipeline: update the submodule, run the build, test it and send it to production. This is made possible by using the properties of the “<strong>git push -o”</strong> command and the rules together. As an example, I’ll take the last line of practice #2 and <a href="https://docs.gitlab.com/ee/user/project/push_options.html">add</a> this, where TRIGGER_SUBMODULE variable is the name of the build job:</p><p>-o ci.variable=&quot;TRIGGER_SUBMODULE=${SUBMODULE_NAME}:build&quot;</p><p>And add this to it to make the following:</p><pre>- git push https://root:$ACCESS_TOKEN@$CI_SERVER_HOST:$CI_SERVER_PORT/$CI_PROJECT_PATH.git HEAD:${CI_COMMIT_REF_NAME} -o ci.variable=&quot;TRIGGER_SUBMODULE=${SUBMODULE_NAME}:build&quot;</pre><p>For the job that we want to trigger, we will automatically write rules.</p><pre>rules:<br>    - if: $UPDATE_SUBMODULE == &quot;true&quot;<br>      when: never<br>    - if: $CI_JOB_NAME == $TRIGGER_SUBMODULE<br>      when: always<br>    - when: manual</pre><p>NB: The new pipeline will run on behalf of the Access token, so it must have the appropriate permissions: either it must be a group token (req. GitLab&gt;14.7) or create a user with the right permissions and replace root:$ACCESS_TOKEN with user credentials.</p><p>The voice of one crying in the wilderness: I will take this opportunity to say what I miss about Gitlab is the options for autocomplete variables at startup. So, for example, now you have to remember which variables and what values are important and what values they can take. Having the ability to choose from several options would make it possible to run pipelines more finely. But at least part of the choice of variables can be transferred to the cases described in this practice #4.</p><p><strong>Practice #5.</strong> <em>Meeting CI and Docker by tagging and environment substitution.</em></p><p>A hike in which images are tagged from CI has proven to be a good idea. This can be either the branch name or the SHA commit, or both at once.</p><p>docker build -t $GC_IMAGE_NAME:$CI_COMMIT_REF_NAME -t $GC_IMAGE_NAME:$CI_COMMIT_SHA .</p><p>It is convenient to manage them in the cloud repository and look for errors. If you need a multi-step build, these tags can be used with the envsubst tool. For example, if two dependent images are built in one pipeline, it is sufficient to refer to the SHA commit tag in the second one and use</p><p>envsubst &lt; Dockerfile | tee Dockerfile_step2</p><p>This handy tool helps not only with CI variables but also with modifying files from demo to production, just like the Helm charts for Kubernetes is often used.</p><p><strong>Concept.</strong> <em>Use GitLab as Documentation.</em></p><p>This also goes into the CI centric approach to documentation — better than writing a lot in the wiki, automate it in CI “click the button, you get the result,” and most importantly it is immediately clear how to repeat it by hand.</p><p>Practices are being refined and new ones mastered, so I hope the article will have a sequel.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e98d1df57dfa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/yet-another-5-practices-of-ci-for-web-apps-deployment-e98d1df57dfa">Yet another 5 practices of CI for Web-App’s deployment.</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AVS3 Block Partitioning]]></title>
            <link>https://medium.com/vicuesoft-techblog/avs3-block-partitioning-3648c09328e7?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/3648c09328e7</guid>
            <category><![CDATA[avs3]]></category>
            <category><![CDATA[streaming]]></category>
            <category><![CDATA[codec]]></category>
            <category><![CDATA[video-coding]]></category>
            <category><![CDATA[videostream]]></category>
            <dc:creator><![CDATA[Denis Fedorov]]></dc:creator>
            <pubDate>Mon, 29 Aug 2022 05:50:38 GMT</pubDate>
            <atom:updated>2022-08-29T05:50:38.774Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Example of block partitioning in AVS3 video codec" src="https://cdn-images-1.medium.com/max/868/1*WcI1gAk56cpjo1sHDbTfmA.png" /></figure><p>Welcome back! In this article, I will review block splitting methods and coding unit tree shape of the third generation of Audio Video Coding Standard (AVS3).</p><p><strong>Coding unit tree</strong></p><p>AVS3 is developed on top of its predecessor — AVS2, one of its objectives is to adapt to the growing demand for Ultra High Deﬁnition content (4K, 8K).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/0*ev0hVivphj_zTH9v.png" /><figcaption>Pic 1. A quadtree structure example of AVS2</figcaption></figure><p>Audio Video Standard is a block-based video coding framework. For block partitioning, AVS2 uses the quadtree scheme only (pic 1), which is not efficient for 8k videos. To fix this, AVS3 developed a more flexible partitioning scheme: quadtree (QT), binary tree (BT), and extended quadtree (EQT) (pic 2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OxfdAN6nV6XlWytWqpJ9dg.png" /><figcaption>Pic 2. Quadtree, Binary Tree, and Extended Quadtree splitting schemes example of AVS3</figcaption></figure><p>For each largest coding unit, one bit is ﬁrst used to indicate whether the large coding unit (LCU) is divided into QT or not. If not, one more bit is then used to represent whether the LCU is divided into EQT or BT (pic 3). Once the coding unit is divided into BT or EQT, QT cannot be used in the subsequent partition process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*axtxwvUhX8iKLMdS.png" /><figcaption>Pic 3. Signaling on LCU splits in AVS3</figcaption></figure><p><strong>Intra Derived Tree</strong></p><p>AVS3 has an additional mode for splitting Intra coding units into smaller prediction units. It splits the luma component only. CU could be split into 4 PUs vertically or horizontally. There are six split modes; you can see them in pic 4. In terms of symmetry, the derived modes can be subdivided into two classes:</p><ul><li>Symmetric modes: 2N×hN and hN×2N, which are designed to improve the signaling efficiency of side information</li><li>Asymmetric modes: 2N×nU, 2N×nD, nL×2N, and nR×2N, which are designed to improve the flexibility of the partition structure to the asymmetric texture</li></ul><p>And there is one thing that should be mentioned for asymmetric modes (2MxnD, 2MxnU, nLx2N, nRx2N) — they are initially split into 2 PUs: large and small. Then large PU is split into three sub-PUs with the same params; as a result, there are 3+1 PUs. This way, the prediction accuracy can be improved as per its stronger correlation between the prediction sample and the reference sample.</p><p>Also, when DT is used, the transform unit is forced to split according to the prediction units.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/0*4NWnSV6fKCx0HnN1.png" /><figcaption>Pic 4. Split schemes of Intra Derived Tree mode in AVS3. Sub-PUs with the same mode are marked with grey color.</figcaption></figure><p>The intra-DT ﬂag is signaled at CU level. The split direction and split type are subsequently transmitted when the intra-DT ﬂag is true.</p><p>Intra DT is applied only for the luma component, so chroma is not split, and chroma PU remains the same size as CU. Here is an example in pic 5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/0*ScT0W0S8fy7unNmW.png" /><figcaption>Pic 5. VQ Analyzer. Example of AVS3 prediction block structure. Blue solid lines — tree splits of luma and chroma, blue dashed lines — tree splits of luma only, green dashed lines — Intra DT luma splits.</figcaption></figure><p><strong>Chroma tree split restriction</strong></p><p>AVS3 has a particular restriction for splitting chroma components — both chroma sides should be at least 4 pixels in chroma down-sampled resolution (8 pixels in luma resolution). If partitioning could create smaller chroma blocks, then chroma splitting is not performed. So chroma coding block remains bigger than the partitioned luma blocks (pic 5, pic 6).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Rcax0Fjm3MF7DIW2.png" /><figcaption>Pic 6. VQ Analyzer. Example of AVS3 luma split PU with non-split chroma PU. The Luma region is highlighted on the chroma parts.</figcaption></figure><p><strong>Non-rectangular block splitting</strong></p><p>There are special modes of Angular Weighted Prediction (AWP) for Inter blocks and Spatial Angular Weighted Prediction (SAWP) for Intra blocks that act like splitting CU into two PUs by a slope line. Initially, each part is predicted with different prediction param; they are combined then using a weight mask that is generated by a slope line with a smooth transition (pic 7). A more detailed review of these modes is presented in <a href="https://medium.com/vicuesoft-techblog/non-rectangular-block-splitting-in-avs3-awp-and-sawp-modes-5479cc2bba98">the individual article</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9hqJddp6LmbcmyAH.png" /><figcaption>Pic 7. VQ Analyzer. Example of SAWP prediction block in AVS3.</figcaption></figure><p><strong>AVS3 tree examples</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MRrn8blOB1Fbjn4jYc1TZw.png" /><figcaption>Pic 8. Example of blocks splitting on B frame of AVS3. Screenshot of VQ Analyzer, prediction mode.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ys7vwmdjKUfD0RgEV2H9-Q.png" /><figcaption>Pic 9. Example of blocks splitting on I frame of AVS3. Screenshot of VQ Analyzer, prediction mode.</figcaption></figure><p>References:</p><ol><li>AVS3 — Part 2: Video. Draft text of AVS video group</li><li>L. Wang, X. Cao, B. Niu, Q. Yu, J. Zheng and Y. He, “Derived Tree Block Partition for AVS3 Intra Coding,” 2019 Picture Coding Symposium (PCS), 2019, pp. 1–5, doi: 10.1109/PCS48520.2019.8954542.</li><li>J. Zhang, C. Jia, M. Lei, S. Wang, S. Ma, and W. Gao, “Recent Development of AVS Video Coding Standard: AVS3,” 2019 Picture Coding Symposium (PCS), 2019, pp. 1–5, doi: 10.1109/PCS48520.2019.8954503.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3648c09328e7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/avs3-block-partitioning-3648c09328e7">AVS3 Block Partitioning</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Non-rectangular block splitting in AVS3: AWP and SAWP modes]]></title>
            <link>https://medium.com/vicuesoft-techblog/non-rectangular-block-splitting-in-avs3-awp-and-sawp-modes-5479cc2bba98?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/5479cc2bba98</guid>
            <category><![CDATA[codec]]></category>
            <category><![CDATA[encoder-decoder]]></category>
            <category><![CDATA[avs3]]></category>
            <category><![CDATA[video-coding]]></category>
            <dc:creator><![CDATA[Denis Fedorov]]></dc:creator>
            <pubDate>Wed, 20 Jul 2022 07:13:08 GMT</pubDate>
            <atom:updated>2022-08-09T12:00:40.897Z</atom:updated>
            <content:encoded><![CDATA[<p>In this paper, I will review the non-rectangular block splitting method for prediction in the currently developing codec Audio Video Standard (AVS3 phase 2): <em>Angular Weighted Prediction (AWP) and Spatial Angular Weighted Prediction Mode</em> <em>(SAWP)</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*4LeUXfRZ58-IDW7HrR7K-Q.png" /><figcaption>Pic 1. Prediction mode view with zoomed AWP prediction block and weight matrix (left). Screenshot of VQ Analyzer with AVS3 bitstream</figcaption></figure><p>One of the most fundamental techniques in video coding is partitioning a picture into smaller blocks for prediction and transform coding. Different tree-structured methods were developed to split variable block sizes during codec evolution. Although the partitioning structure becomes more flexible and efficient, the rectangular shape of blocks is not always the best way to describe an arbitrary object shape (pic 1).</p><p>Usually, when the object boundary is not horizontal or vertical, more blocks are required to split areas by a slope line. Therefore, non-rectangular block splitting techniques were developed to handle such cases [3].</p><p><strong>Overview of comparable tools</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/1*mUz_mV2Dz9F1menrYwcs3w.png" /><figcaption>Pic 2. Predefined masks for splitting blocks in AV1 (left) and VVC (right)</figcaption></figure><p>Compound wedge prediction (CWP) was adopted in AV1; it is based on using predefined weight masks; there are 16 variants for each block size [2]. Geometric partition mode (GPM) was adopted in VVC [4]; it supports 64 partition variants (pic 2). All these masks are being used in the same way to divide block into two parts by a slope line. For a camera-captured content, sample values near the object’s edges usually have a smooth transition; therefore, masks are designed with a non-binary transition area.</p><p><strong>AWP</strong></p><p>AVS3 developed <em>Angular Weighted Prediction mode</em> (AWP) to handle non-rectangular prediction; it supports 8 angle and 7 offset values (56 combinations) for one block size, and it also supports two variants of the transition area smoothness. It seems to be a compromise: more flexible than CWP in AV1 and less computational complexity than GPM in VVC.</p><p><strong>SAWP</strong></p><p>An interesting remark is that both AV1 and VVC use the described methods as Inter Bi-prediction technique, but AVS3 has also applied the splitting method for Intra prediction; it is called <em>SAWP — Spatial Angular Weighted Prediction</em>. So, for AVS3, we have AWP — for Inter prediction and SAWP — for Intra prediction.</p><figure><img alt="Pic 3. Intra frame, prediction mode view with zoomed SAWP prediction block (right). Screenshot of VQ Analyzer with AVS3 bitstream" src="https://cdn-images-1.medium.com/max/1024/1*u-Q1WbBBGn4XZ9eXSkWVbA.png" /><figcaption>Pic 3. Intra frame, prediction mode view with zoomed SAWP prediction block (right). Screenshot of VQ Analyzer with AVS3 bitstream</figcaption></figure><p><strong>AWP &amp; SAWP Algorithm stages</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*MsQ-gXYk-VBdn1cTGBfAog.png" /><figcaption>Pic 4. Angular Weighted Prediction stages in AVS3</figcaption></figure><p>The decoding algorithm is based on three parts:</p><ol><li>Derive independent predictions for part A and part B</li><li>Derive AWP weight mask</li><li>Derive prediction results by combining parts using the weight mask</li></ol><p><strong>Prediction parts derivation</strong></p><p>For SAWP mode, parts are predicted using intra angular prediction; for AWP mode, motion-compensated prediction is used.</p><p><strong>Weight mask derivation</strong></p><p>Weight mask is filled in a way similar to the angular intra prediction — a block is generated using the reference line and angle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*e7yaW47QSPnP-Vy7D4oVyw.png" /><figcaption>Pic 5. Weight determination for AWP &amp; SAWP. Reference line length is derived by AWP angle.</figcaption></figure><p>Calculation of mask is followed by three steps:</p><ol><li>Derive the length of reference weight line</li><li>Fill reference weight line</li><li>Fill reference weight block based on the line</li></ol><p><strong>Weight Mask signaling</strong></p><p>Mask form is derived by <em>AwpIndex </em>that is signaled in the bitstream. For B picture value, it is in the range of 0..55 for 56 combinations and used directly; for P picture value, it is derived by predefined tables for different CU sizes.</p><p>Mask boundary smoothness is derived from the binary value <em>PictureAwpRefineIndex </em>signaled in the bitstream.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*JhkAmkxmxpag9Sd12yqo5w.png" /><figcaption>Pic 6. AWP set of 8 angles (left) and set of 7 reference weight lines (right)</figcaption></figure><p>There are 8 possible angles and 7 steps for generating the initial reference line (pic 6). Angles are grouped in <em>AwpIndex </em>by sets for each step.</p><p><em>AwpIndex </em>is translated to the following parameters for further calculations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/1*wsGtj_841XtaCys6HQbXfw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*YQCMKLT2zU8_uOWZUVf70w.png" /><figcaption>Pic 7. Details view for AWP prediction block with derived parameters. Screenshot of VQ Analyzer with AVS3 bitstream.</figcaption></figure><p><strong>Calculations<br></strong>1.<strong> </strong>Determine vL, length of the valid initial reference weights line. <br>Half pixel precision is used for the initial line.<br>For the first 4 angles, the column on the left is taken as the reference line. For the last 4 angles, the row on the top is taken as the reference line (pic 6):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*jMUIao3kyiB4nc5Hznix8Q.png" /><figcaption>where H and W are the current block’s height and width.</figcaption></figure><p>2. Fill initial reference weight line <em>Lw</em> with the length <em>vL</em> (pic 5). <br>Pattern for selected angle and step could be derived from the next equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*ZTUgAYbnayHKAJuy1VqWJg.png" /><figcaption>where<em> PictureAwpRefineIndex</em> sets the transition area smoothness</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*Pzsfu3fJeKPvP2f4OoSGbA.png" /><figcaption>Pic 8. VQ Analyzer. Details view for two SAWP prediction blocks in AVS3 bitstream. Smooth transition area on the left block, rough transition area on the right block.</figcaption></figure><p>3. Fill the luma weight mask using the reference line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/242/1*pcDRBl0Z0lFsZMVP8uD5jA.png" /><figcaption>Where tP is derived from the following tables.</figcaption></figure><p>For AWP mode in B picture or SAWP mode:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*VzS2ixRs7lSDDM40WaTHSQ.png" /></figure><p>For AWP mode in P picture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*9A3zytuw3IWtaIQveoOlPg.png" /></figure><p>4. Fill the chroma weight mask with the luma weight mask.<br>For AWP mode in B picture or SAWP mode:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*2MgB-KDWau9X62sJ-9mbDA.png" /></figure><p>For AWP mode in P picture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/659/1*gLrAixrHvID0gKcPXeMFkg.png" /></figure><p>5. Combine the final prediction result</p><p>Use prediction parts A and B and weight maps for luma and chroma, respectively:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/1*t_YKVqxTWP5Ie_KtJEhFAg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/941/1*Uk3_wgiu5hMUOUtJUxd4dw.png" /><figcaption>Pic 9. Details view for AWP prediction block. Screenshot of VQ Analyzer with AVS3 bitstream.</figcaption></figure><p>References:</p><p>1. AVS3 — Part 2: Video. Draft text of AVS video group</p><p>2. Y. Sun, F. Chen, L. Wang and S. Pu, “Angular Weighted Prediction for Next-Generation Video Coding Standard,” 2021 IEEE International Conference on Multimedia and Expo (ICME), 2021, pp. 1–6, doi: 10.1109/ICME51207.2021.9428315</p><p>3. X. Xu and S. Liu, “Overview of Screen Content Coding in Recently Developed Video Coding Standards,” in IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 2, pp. 839–852, Feb. 2022, doi: 10.1109/TCSVT.2021.3064210.</p><p>4. Algorithm description for Versatile Video Coding and Test Model 12 (VTM 12)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5479cc2bba98" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/non-rectangular-block-splitting-in-avs3-awp-and-sawp-modes-5479cc2bba98">Non-rectangular block splitting in AVS3: AWP and SAWP modes</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is dev infrastructure for beginners (part 1): understanding]]></title>
            <link>https://medium.com/vicuesoft-techblog/what-is-dev-infrastructure-for-beginners-part-1-understanding-8745e2fb9535?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/8745e2fb9535</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[process-automation]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[information-technology]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <dc:creator><![CDATA[Alexander Dydychkin]]></dc:creator>
            <pubDate>Thu, 24 Mar 2022 00:12:06 GMT</pubDate>
            <atom:updated>2022-03-24T00:12:06.214Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VyIJbNmMLYGKVCpl" /><figcaption>Photo by <a href="https://unsplash.com/@marvelous?utm_source=medium&amp;utm_medium=referral">Marvin Meyer</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Synopsis</h3><p>The ideas of this article were born during my discussion with bachelor students. I understood that there are many people who do not understand how infrastructure for development works and looks. So I decided to make a try to explain as much as I can. I will start by defining the problem, principles and “forms” of dev infrastructure so that will be more about understanding rather than a step-by-step solution. But I will expand the topic with additional articles if I have feedback.</p><h3>Why is the author allowed to comment on this topic?</h3><p>My first three years in the IT-industry were in the positions of infrastructure guy. I had the experience of integrating CI processes in big companies and also implementing development infrastructure from scratch in small companies. Yeah, I am not an expert with 30+ years in the industry, but I have experience and opinion. If you have additions, let’s discuss them because it is a discussion wherein the truth is born.</p><h3>Article structure</h3><ul><li>Find and formulate the problem</li><li>Automation</li><li>Continuous Integration (CI)</li><li>What is needed for automation?</li><li>CI service responsibility</li><li>Thoughts on some bottlenecks</li><li>Bonus: DevOps</li><li>Conclusion</li></ul><h3>Find and formulate the problem</h3><p>First of all let’s find and highlight the problem because correct understanding helps to find a solution.</p><h4>Number 1 — teamwork</h4><p>The IT industry was all the time about working in the team. Yeah, we had some examples of dev-stars but their software needs the community to improve, productize, and support. The community will discuss priorities, report bugs, push the code and sometimes hate you :)</p><h4>Number 2 — in fact, projects are vast and complex</h4><p>See the projects like Linux kernel (<a href="https://kernel.org/">kernel.org</a>) or Chromium (<a href="https://www.chromium.org/Home/">www.chromium.org/Home/</a>). They are essential for all the internet and have hundreds of developers (not only developers but also how many managers, testers?), very old, vast and complex. Don’t believe me? Let’s see the stats from <a href="https://lwn.net/Articles/867540/">lwn.net</a> about Linux kernel version 5.14 development:</p><blockquote>To create 5.14, the kernel community applied 14,735 non-merge changesets from <strong>1,912 developers</strong>; 261 of those developers made their first kernel contribution during this cycle. There were <strong>861,000 lines of code added</strong> to the kernel and <strong>321,000 lines removed</strong>, for a net growth of 540,000 lines.</blockquote><h4>Number 3 — you are not a programmer; you are a business problem-solver</h4><p>…your code is not the code; it is solving a business problem. From this point of view managers and responsibility to customers “enter the stage”. You, as a developer, need instruments to prove that your code fixes and improves the current product status. It would help if you had the minimum build, test and validation systems.</p><h4>To sum up</h4><p>We have a lot of people who interact with each other, vast mass of source code and managers who want some proofs of your work. To solve this we can hire more people, do all the staff manually or… <strong>implement automation</strong>!</p><h3>Automation</h3><p>Honestly almost all the staff in the IT-industry can be called automation because of the nature of computers. Still, I focused this article on<strong> infrastructure</strong> <strong>for development</strong> and <strong>principles </strong>so the word automation means — <strong>Continuous Integration (CI) </strong>and some things around it.</p><h3>Continuous Integration (CI)</h3><p>First programmers (or very bad teams today) had a lot of “manual“ work because they <strong>didn’t have so many libraries, frameworks and tools </strong>such as we have today. God, thanks!</p><p>Thus, for example, a middle-size C++ project in the past. Developers have only some self-written bash scripts to build and test the project. They start them manually or with cron before the release or once per week. I can imagine how much <strong>pain and hell</strong> <strong>the build and tests</strong> can be after another commit.</p><p>Therefore, upon analyzing this what was the decision by these “first programmers”? Building and testing are like hell? Yeah? So let’s do it much often! Every day! For every commit!</p><p>They implemented the systems that we now call <strong>CI-systems </strong>to support this practice.</p><p>More importantly, they formulated <strong>CI principles</strong> — <strong>integrate your changes to the mainline product as soon as possible</strong>.</p><p>In addition, program managers formulated <strong>unique formula</strong> which tells us:</p><blockquote>The sooner you find a mistake, the cheaper it is to fix it.</blockquote><p>CI, strict definition by Wikipedia:</p><blockquote><em>In software engineering, continuous integration (CI) is the </em><strong><em>practice</em></strong><em> of </em><strong><em>merging</em></strong><em> all developers’ working copies to shared mainline </em><strong><em>several times</em></strong><em> a day.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*90eXrRq9BymnZpec3omARg.png" /><figcaption>Simplified CI/CD process</figcaption></figure><h4>Bonus:</h4><p>Continuous delivery (CD) is a continuation of CI principles in the field of delivery of product updates to the customers/users. I don’t want to delve deeper into this practice because we must first understand the basic principles. I will give only a strict definition from Atlassian for those who suffer:</p><blockquote>Continuous Deployment (CD) is a software release process that uses automated testing to validate if changes to a codebase are correct and stable for immediate autonomous deployment to a production environment.</blockquote><h3>What is needed for automation?</h3><p>Ok. I hope we are ready to define problems and principles for solving but what are the “bricks” of our automation? What are the “bricks” of infrastructure for software development?</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba7bdfcaa6605b6a977f9ab8ce03dfcb/href">https://medium.com/media/ba7bdfcaa6605b6a977f9ab8ce03dfcb/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/1*Kxt0dVSNcsC9VnPEPDge2g.png" /><figcaption>Model a very simple CI infrastructure with active re-usage of Gitlab services</figcaption></figure><p>Every infrastructure architect/developer should choose the “bricks” based on the domain, company rules, product programming language, experience, costs, flexibility level and team. There is not a “holy grail”. Yeah, there are systems like Gerrit and others which are a bit legacy and not trendy. So, they have modern analogs, but in general, the architect should know the actual needs of the development team to implement the right, easily supportable, and flexible infrastructure.</p><h3>CI service responsibility</h3><p>From the illustrations above we can see that in the center of infrastructure there should be “CI service”, but what should this service do? I tried to formulate it, my version:</p><ul><li>Check changes in Version control systems (VCS)</li><li>Take the changes from VCS</li><li>Allocate suitable machine (scheduler)</li><li>Execute predefined scenario on the machine</li><li>Log the scenario</li><li>Report build/test status</li><li>Upload build/test artifacts</li></ul><h3>Thoughts on some bottlenecks</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_pQsK2xm6BOp0gVAoWCX3Q.png" /><figcaption>Typical dev process with comments</figcaption></figure><p>I used the same illustration but added some comments to mark that it cannot be easy. There are only a few possible issues and complications that an infrastructure guy may encounter.</p><p>If we speak about CI only in this case the most popular example is how many build, tests and simple checks can be started for one regular commit? 2? 4? 10? Here is a small illustration from (not the biggest) project <a href="https://github.com/Intel-Media-SDK/MediaSDK">Intel MediaSDK</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*sB6n0q11buBm_lRS7pC3Wg.png" /><figcaption>Number of checks started for one commit in master</figcaption></figure><p>29 various checks (CI runs) started for only one commit and all of them should be predefined, the environment for execution should be ready and so on and so forth… I want to underline that such project is not as big as the Linux Kernel project, and we do not discuss the validation process (the process of complete testing).</p><h3>Bonus: DevOps</h3><p>Term DevOps deserves a separate article, but as a bonus, I will provide a strict definition and some personal comment; Wikipedia’s strict definition:</p><blockquote><em>DevOps is a </em><strong><em>set of practices</em></strong><em> that combines software development (Dev) and IT operations (Ops). It aims to shorten the systems development life cycle and provide continuous delivery at high software quality.</em></blockquote><p>As you see from this definition, it is all about the reduction of development time (cycles) -&gt; as a result, become more adaptive to clients (market) -&gt; as a result, more income.</p><blockquote>As DevOps is intended to be a cross-functional mode of working, those who practice this methodology use different sets of tools — referred to as “toolchains” — rather than a single one:</blockquote><blockquote><strong>Coding </strong>— code <strong>development </strong>and review, source code management tools, code merging.</blockquote><blockquote><strong>Building </strong>— <strong>continuous integration</strong> tools, build status.</blockquote><blockquote><strong>Testing </strong>— continuous <strong>testing</strong> tools that provide quick and timely feedback on business risks.</blockquote><blockquote><strong>Packaging </strong>— artifact repository, application pre-deployment staging.</blockquote><blockquote><strong>Releasing </strong>— change management, <strong>release approvals</strong>, release automation.</blockquote><blockquote><strong>Configuring </strong>— <strong>infrastructure configuration</strong> and management.</blockquote><blockquote><strong>Monitoring </strong>— applications performance <strong>monitoring</strong>, end-user experience.</blockquote><p>DevOps destroys strict responsibility limits and says that the personel should be cross-functional within reason. I specifically wrote “personel” because, I think, there is a big problem when you see job offers with trendy slogans “We are looking for a DevOps expert”. My opinion is that the team should be DevOps, not particular worker. Yeah, the sysadmin must know more about the product, but I think it is rather a question of integrity and motivation.</p><p>So I think, DevOps is an excellent methodology but be careful with the term because sometimes its use is wrong.</p><h3>Conclusion</h3><p>In this article we dissected problems, introduced terms with their definitions, decided on the system’s structure, and marked possible bottlenecks. So we have a theoretical basis to start building infrastructure for software development.</p><p>In the following article, I will try to create a model of the actual development process and try to implement some elementary infrastructure to support this dev process.</p><p>To sum up, all the <strong>right </strong>work of infrastructure guys comes down to simple as possible operations which a rank-and-file developer should execute to push the code and close the ticket, like this:</p><pre>git clone <a href="https://github.com/Intel-Media-SDK/MediaSDK.git">https://github.com/Intel-Media-SDK/MediaSDK.git</a> <br>git checkout -b “my_super_branch”<br># some changes in code<br>git commit -am “[cmake] Add right location for sha execution”<br>git push origin my_super_branch</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*n64s38WtbhooeEvy" /><figcaption>Merged commit after the push, after the review and discussions, with the so desired green CI status</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8745e2fb9535" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/what-is-dev-infrastructure-for-beginners-part-1-understanding-8745e2fb9535">What is dev infrastructure for beginners (part 1): understanding</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DVK: Reducing codec development costs (part 1)]]></title>
            <link>https://medium.com/vicuesoft-techblog/dvk-reducing-codec-development-costs-part-1-1e929a4bbcef?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/1e929a4bbcef</guid>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[av1]]></category>
            <category><![CDATA[vvc]]></category>
            <category><![CDATA[codec]]></category>
            <dc:creator><![CDATA[Alexander Dydychkin]]></dc:creator>
            <pubDate>Tue, 12 Oct 2021 04:02:41 GMT</pubDate>
            <atom:updated>2021-10-12T04:02:41.424Z</atom:updated>
            <content:encoded><![CDATA[<p>This article is a continuation of my previous articles about <a href="https://vicuesoft.com/blog/titles/%D0%A1odec_War_the_show_must_go_on/">the codec industry</a>’s situation and <a href="https://vicuesoft.com/blog/titles/dvk_advantages/">what the DVK</a> is, where I explain the usage of “helper tools”, especially DVK. Here I want to clarify how DVK works with the focus on the streams and the encoder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hKaN_sPptl4JeaJVV1iWvA.png" /><figcaption>Part of VVC DVK syntax element coverage by regular</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/364/1*R3HXZQS89p6Vx5kGi0Sziw.png" /><figcaption>Part of VVC DVK cross coverage report</figcaption></figure><h3>What is the DVK?</h3><blockquote><a href="https://vicuesoft.com/vq-dvk"><strong>DVK</strong></a> is a bitstream and software toolchain for the Decoder IP design and compliance validation. It is a must-have tool for chip RTL design, target appliance integration (such as a smartphone, smart TV, Set Top Box), software stack verification before shipment to the customer. It is carefully structured to become as compact as possible, address maximum coverage and minimize validation time during the design process.</blockquote><p>DVK contains:</p><ul><li>parfiles</li><li>random_encoder</li><li>streams</li><li>analytic tools <em>(will be explained in detail in a separate article)</em></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*nymBiPwJzgU8pN_oHbjRhw.png" /><figcaption>Entire DVK workflow: from original yuv to coverage report</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/1*aomMbd5SudnecFoMb3O9Eg.png" /><figcaption>Workflow: how to use streams (use case example)</figcaption></figure><h3>DVK package</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/116/1*HFc-xGn1boGCZtCm8HF0Zg.png" /><figcaption>Content of usual VVC DVK package</figcaption></figure><p>Content description:</p><ul><li><strong>analytic tools</strong> — tools to review and visualize streams coverage (Win/Lin);</li><li><strong>coverage</strong> — html reports;</li><li><strong>decoder </strong>— reference and instrumented decoder, instrumented decoder needed to grab syntax coverage;</li><li><strong>docs</strong> — detailed documents how to use tools and technical description of the streams with detailed description which coding tools and value ranges are covered by each bitstream;</li><li><strong>encoder </strong>— special encoder random_encoder based on reference, which generates streams based on parfiles;</li><li><strong>parfiles </strong>— special configuration files which fully specify generated streams;</li><li><strong>streams </strong>— streams grouped by profiles and color spaces (or by customer request);</li><li><strong>yuv </strong>—VicueSoft source yuvs. User can use any yuvs he wants. It needs only to use the same file naming convention;</li><li><strong>readme.txt</strong> — contains basic package info and the changelog;</li><li><strong>vvc_cmds.txt</strong> — source cmds to reproduce generation of the the delivered streams folder.</li></ul><h3>Parfiles</h3><p>“Parfiles” are special configuration files that fully specify generated streams. They give full description for the random_encoder as to which syntax elements should be used in a particular stream; in other words, they enable certain video technologies and blocks of certain syntax elements, which are defined by the official specification of the codec.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/174/1*1bLC-DuJGnD6Rt7o5Gnskw.png" /><figcaption>Some parfiles for VVC streams, each focused on particular elements</figcaption></figure><p>Any user can easily create or change the existing parfiles because they are standard json files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/397/0*X1z_UXPIxlMfSrq5.png" /><figcaption>The part of the config file for VVC</figcaption></figure><p>Each syntax element name has the same naming as in the standard specification (or almost the same because there are rare cases when the committee renamed an element in the specification). The number of values is the same as in the specification too. How to manage the parfile?</p><ul><li>Hard code element value:</li></ul><p>&quot;pred_mode_flag&quot;: 0, — enables inter flag</p><p>&quot;pred_mode_flag&quot;: 1, — enables intra flag</p><ul><li>Set up the distribution for the element :</li></ul><p>&quot;pps_init_qp_minus26&quot;: [-10,10], — enables <strong>uniform distribution</strong> for values from -10 to 10</p><ul><li>Arrays:</li></ul><p>&quot;inter_pred_idc&quot;: [40,40,20],— the probability value for each element of the array, the first — 40%, the second — 40% and the last — 20%.</p><h3>Random encoder</h3><p>A random encoder is based on the reference encoder with the capability to read parfiles mentioned above to override encoding decisions.</p><p>The random encoder has some parameters, too, for example:</p><ul><li>bit depth</li><li>chroma format</li></ul><p>The help of random encoder for VVC VTM-13:</p><pre>vvc_random_encoder.exe --help<br>ViCue Decoder Validation Kit 0.13.0 for VVC VTM-13.0, sha1 274e8fc7705e567b9c9adbc69469d95bb8a70745<br>Copyright (c) 2019-2021, ViCue Soft LLC. All rights reserved.<br>Usage: vvc_random_encoder.exe -i &lt;input.yuv&gt; -o &lt;output.vvc&gt; -w &lt;width&gt; -h &lt;height&gt;<br>                              [-p &lt;config.json&gt; -s &lt;seed&gt; -r &lt;reconstruct.yuv&gt;]<br>                              [-f &lt;# of frames to process&gt;]</pre><pre>Options:<br>  -i STRING, --input=STRING<br>                        input file (raw yuv)<br>  -o STRING, --output=STRING<br>                        output vvc bitstream<br>  -w INT, --width=INT   width of input yuv<br>  -h INT, --height=INT  height of input yuv<br>  -f INT, --frames=INT  number of frames to process<br>  -r STRING, --recon=STRING<br>                        output file for encoder&#39;s reconstruct (optional)<br>  --store-input=STRING  input file after possible bit-depth and chroma-format<br>resampling (optional)<br>  -p STRING, --parfile=STRING<br>                        json file with distribution parameters (optional)<br>  -s INT, --seed=INT    seed for random generator (optional)<br>  --input-bitdepth=CHOICE<br>                        input YUV file bitdepth<br>  --output-bitdepth=CHOICE<br>                        output bitstream bitdepth<br>  --input-chromaformat=CHOICE<br>                        input YUV file chroma format<br>  --output-chromaformat=CHOICE<br>                        output bitstream chroma format. By default is the same<br>                        as input chroma format<br>  --TraceFile=STRING    tracing file<br>  --TraceRule=STRING    tracing rule (format as in VVC software). Example:<br>                        --TraceFile=trace.txt  --TraceRule=&quot;D_CABAC,D_HEADER:poc&gt;=0&quot;<br>  --help                output this message<br>  --ext_trial=STRING    extend trial with specified serial number<br>  --activate=STRING     activate by using specified serial number for single<br>user license<br>  --deactivate          deactivate single user license</pre><h3>Streams</h3><p>DVK has pre-generated streams based on all parfiles. They are immediately ready to use. Usually, they are sorted by profile, bit depth, and color format. Inside the folder, user can see the listing as below:</p><ul><li><strong>*.vvc</strong> — stream;</li><li><strong>*.vvc.ccstats.json</strong> — raw data needed to grab cross-coverage by analytic tools <em>(it will be described in a separate article);</em></li><li><strong>*.vvc.rec.yuv.md5</strong> — md5 sum of reconstruct for the user’s check;</li><li><strong>*.vvc.stats.csv</strong> — used elements statistics also needed for the analytic tools.</li></ul><p>What does naming of the stream mean? Let’s explain on the basis of the file <em>VVC_Main10_8b400_24x72_transform_lfnst_cbf_00_v0.13.0.vvc:</em></p><ul><li><strong><em>VVC</em></strong><em>_Main10_8b400_24x72_transform_lfnst_cbf_00_v0.13.0.vvc </em>— codec</li><li><em>VVC_</em><strong><em>Main10</em></strong><em>_8b400_24x72_transform_lfnst_cbf_00_v0.13.0.vvc </em>—profile</li><li><em>VVC_Main10_</em><strong><em>8b</em></strong><em>400_24x72_transform_lfnst_cbf_00_v0.13.0.vvc </em>— bit depth</li><li><em>VVC_Main10_8b</em><strong><em>400</em></strong><em>_24x72_transform_lfnst_cbf_00_v0.13.0.vvc </em>— color space</li><li><em>VVC_Main10_8b400_</em><strong><em>24x72</em></strong><em>_transform_lfnst_cbf_00_v0.13.0.vvc </em>— resolution</li><li><em>VVC_Main10_8b400_24x72_</em><strong><em>transform_lfnst_cbf_00</em></strong><em>_v0.13.0.vvc </em>— parfile name on the basis of which this stream is generated</li><li><em>VVC_Main10_8b400_24x72_transform_lfnst_cbf_00_</em><strong><em>v0.13.0</em></strong><em>.vvc —</em> version of the codec standard</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*2IEEan29iizm2xs9PdVXhg.png" /><figcaption>Part of VVC main10–8bit-400 streams</figcaption></figure><p>Also, each DVK’s “Syntax” stream is focused on a particular syntax element or block defined by the official codec specification. They are very small. This approach speeds up the testing process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*Ve02CBuuEipr6S52p1D71Q.png" /><figcaption>Parfile for particular block</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*zVYZaVM62pzYWXuyvuarKg.png" /><figcaption>A “Syntax” stream generated from the parfile above. See? It is very focused, only 17kB!</figcaption></figure><p>Types of streams:</p><ul><li><strong>Syntax</strong> streams are low resolution bitstreams with a small number of frames which maximize coverage of each syntax element and cross-coverage of essential pairs. These streams are designed to test a certain subset of features, for example, all the intra-prediction modes or all loop-filter related parameters. One stream tests the smallest possible coding block in the decoder allowing an independent design of each block. This type of streams is good for the <strong>initial RTL design</strong>;</li><li><strong>Stress</strong> bitstreams are extended from the pool of Syntax streams with higher resolution and a larger number of frames. Stress streams include all the features covered by the bucket, so they are useful for smoke testing: if a decoder passes the Stress stream, it is likely to pass all the Syntax streams from the bucket. Stress streams are good for a <strong>mature stage of the decoder design</strong>;</li><li><strong>Performance</strong> bitstreams test speed capabilities of the decoder’s hardware. They are good for final validation to <strong>proof compliance to the codec Level requirements</strong>;</li><li><strong>Error resilience</strong> subset includes not fully compliant bitstreams which test the decoder’s robustness to errors and ability to recover. They are good for <strong>final validation to deliver a viable end user product</strong>.</li></ul><h3>DVK supported codecs (2021)</h3><ul><li>VVC</li><li>AV1</li><li>VP9</li><li>HEVC</li><li>AVS3</li></ul><h3>Conclusion</h3><p>The DVK’s overview is finished in this article. The following article will be dedicated to analytic tools.</p><p>So the user of the DVK package can:</p><ul><li>use streams for <strong>CI/validation</strong> testing purposes;</li><li>use streams for the compliance check</li><li>use them for debug of the decoder;</li><li>edit or <strong>create new streams</strong> with the help of <strong>the random encoder</strong>;</li><li>analyze and<strong> create syntax&amp;code coverage reports on any set of streams (from DVK or any other source) or optimize a set of streams </strong>with the help of <strong>analytic tools</strong>.</li></ul><p>DVK improves the efficiency of codec testing by doing it “smarter” and improving its execution speed. As a bonus, it offers instruments for reviewing and optimizing the syntax/code coverage.</p><p>PS: Of course, each stream can be reviewed and analyzed in VQ Analyzer!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e929a4bbcef" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/dvk-reducing-codec-development-costs-part-1-1e929a4bbcef">DVK: Reducing codec development costs (part 1)</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Analytic tools 6.0 — new design of the proven tool]]></title>
            <link>https://medium.com/vicuesoft-techblog/analytic-tools-6-0-new-design-of-the-proven-tool-3f09e6081f52?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/3f09e6081f52</guid>
            <category><![CDATA[report]]></category>
            <category><![CDATA[vvc]]></category>
            <category><![CDATA[av1]]></category>
            <category><![CDATA[tools]]></category>
            <category><![CDATA[codec]]></category>
            <dc:creator><![CDATA[Alexander Dydychkin]]></dc:creator>
            <pubDate>Tue, 17 Aug 2021 05:43:51 GMT</pubDate>
            <atom:updated>2021-10-12T04:00:22.196Z</atom:updated>
            <content:encoded><![CDATA[<h3>Analytic tools 6.0 — new design of the proven tool</h3><p>I am happy to announce the release of Analytic tools 6.0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kc4m5EIjIL2E2vAZyWyuNw.png" /><figcaption>New design of report on some VVC streams</figcaption></figure><h3>Quick overview</h3><p>This major version was focused on updating the html report design. Let’s mark the most significant changes:</p><ul><li>Cross coverage tables are on the list now. The main decision is to save the space and give more color to the report, but in any case, the tables can be opened by pressing the button “˅” or “Expand all”:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Gdh9y8qJZLitWr5nRVOe8g.png" /></figure><ul><li>There is added a long waited for “Disable all” button for the Syntax elements coverage to disable the view of all the elements to open only wanted:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/296/1*ZwPm87t9wTCh6pieJjdP9g.png" /></figure><ul><li>There is added the tool revision for easier tracking of used tools. It is unique sha sum from which commit all analytic tools were built:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/127/1*Vkh4cZ0jvEQTbXmyM4WqDg.png" /></figure><p>Except for the visual things, we made some optimizations and features:</p><ul><li>Optimized the base html report (down by 50%) by deleting the listing of the per-stream coverage in the Syntax Element Coverage. The listing is still available by clicking the syntax element.</li><li>Introduced criteria for cross coverage tables (same logic for the general criteria). The user can specify which elements cannot be reached by defining the vvc.crosscoverage.criteria.yamlwith block like that:</li></ul><pre>&lt;crosscoverage table name&gt;:<br>    mask:<br>        - [&lt;mask value&gt;, &lt;mask value&gt;, ...]<br>        - [&lt;mask value&gt;, &lt;mask value&gt;, ...]<br>        ...</pre><p>where -1 value means to mark element “unreachable” (grey color) and ≥0 value will use original value from decoder.</p><p>Usage example with two types of criteria for av1:</p><pre>av1.reporter.exe streams --coverage av1.criteria.yaml --crosscoverage av1.crosscoverage-criteria.yaml</pre><h3>Full changelog</h3><pre>Version 6.0</pre><pre>Updated design and buttons of html report:<br><strong>- </strong>Cross coverage section:<br>    <strong>- </strong>Legend was added at the top the report<br>    <strong>- </strong>Cross coverage tables are on the list now (tables can be opened by pressing &quot;˅&quot; button<br>    <strong>- </strong>Global [Expand/Collapse all] button was added to show/hide all tables</pre><pre><strong>- </strong>Syntax Element Coverage section:<br>    <strong>- </strong>[Disable all] button added to switch off all technologies/blocks<br>    <strong>- </strong>[Hide/Show unsorted] button added to represent elements without a particular relation to a technology or a block<br>    <strong>- </strong>Now all technologies/blocks can be disabled</pre><pre>Optimization:<br><strong>- </strong>Expanded buttons for elements with multiple related streams were deleted for memory-consumption optimization. On a large number of streams report size reduction can be down to 50%</pre><pre>New feature:<br><strong>- </strong>Criteria for cross-coverage section was implemented (read more in README file)<br>    <strong>- </strong>By using the `skip` option, not needed table can be excluded from a report<br>    <strong>- </strong>A `mask` can be provided to a section for omitting specific table element</pre><pre>Fixes/minor changes:<br><strong>- </strong>Header with tool revision added to the html reports<br><strong>- </strong>Cross coverages location was changed to `.coverages/{sha}/*.json`</pre><h3>What are analytic tools?</h3><p>The analytic tools is a proven tool to analyze syntax and code coverage of video streams. They give maximal flexibility and can be used with DVK streams and any customer streams too.</p><p>The tools give a graphical representation of syntax coverages, speed up test cycles by reducing the streams and their number, compare the stream coverage.</p><h3>How to get them?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/567/1*-n7ekQrgT0tOnoBtAULIOg.png" /></figure><p>You can use the latest version of analytic tools by purchasing <a href="https://vicuesoft.com/vq-dvk/">DVK</a> or contact us to get solely analytic tools!</p><p>Or check the available sets <a href="https://vicuesoft.com/file/data_sheet_vq_dvk.pdf">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f09e6081f52" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/analytic-tools-6-0-new-design-of-the-proven-tool-3f09e6081f52">Analytic tools 6.0 — new design of the proven tool</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Сodec War: the show must go on!]]></title>
            <link>https://medium.com/vicuesoft-techblog/%D1%81odec-war-the-show-must-go-on-2ee2400cb54b?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/2ee2400cb54b</guid>
            <category><![CDATA[videos]]></category>
            <category><![CDATA[vvc]]></category>
            <category><![CDATA[av1]]></category>
            <category><![CDATA[hevc]]></category>
            <category><![CDATA[codec]]></category>
            <dc:creator><![CDATA[Alexander Dydychkin]]></dc:creator>
            <pubDate>Tue, 03 Aug 2021 16:03:19 GMT</pubDate>
            <atom:updated>2021-08-03T16:03:17.519Z</atom:updated>
            <content:encoded><![CDATA[<h3>Preamble</h3><p>A minor opus about the current situation in the codec industry with a retrospective and a driver for growth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*S0NjjmOpSHOuTqqq" /><figcaption>Photo by <a href="https://unsplash.com/@floschmaezz?utm_source=medium&amp;utm_medium=referral">Florian Schmetz</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The Internet</h3><p>I am pretty young 25 years old but I still remember the <strong>“good old Internet”</strong>, which was the <strong>Internet of texts and pictures</strong> (earlier only text even🙂) but now we have <a href="https://www.cisco.com/c/en/us/solutions/executive-perspectives/annual-internet-report/index.html">the Internet of video</a>, <strong>where ~80% of traffic is video</strong>. Why? That is the topic of another article about the humans’ brain and how they picture the world, but I want to mention that since “the Internet of text and pictures”, the industry has acquired:</p><ul><li>much faster <a href="https://www.cisco.com/c/en/us/solutions/executive-perspectives/annual-internet-report/index.html">Internet connection</a>;</li><li>more powerful client’s and server’s hardware (Moore’s law);</li><li>improved displays.</li></ul><p>Even IT-industry has so much hardware progress “the Internet of video” cannot be authentic without the compression algorithms and hardware acceleration. Why? Please read my <a href="https://medium.com/vicuesoft-techblog/what-is-a-video-codec-and-how-does-it-work-6f90bd037fa0">previous post</a>. Now the world is ruled <a href="https://go.bitmovin.com/video-developer-report-2020">chiefly by AVC and VP9</a>.</p><p>That’s why I want to discuss how the codecs were developed, standardized and entered the market.</p><h3>Codec industry: how it works</h3><ol><li>So, first of all, the Committee collects technologies (referred to as patents) from various members of a codec group. They harmonize technologies within a single standard — it is called <strong>the standardization procedure</strong>.</li><li>How to use standard? The committee forms core technologies, which will be mandatory for the codec because the decoder should decode any stream relevant to the appropriate standard (codec). Such patents are called <strong>essential patents</strong>. Patents that are outside the essential ones may create another extension to the standard.</li><li>Develop and publish reference implementation of codec.</li><li>Publish the distribution and usage policy</li></ol><p>So when a chipmaker produces a chip with the support of codec X it should pay to the codec group Y which will share the money between the members of the codec group according to their number and importance of patents for the codec. It can be different according to various license policies.</p><h3>Codec industry: yesterday</h3><p><a href="https://en.wikipedia.org/wiki/MPEG_LA">MPEG LA</a>, the company which administrates <a href="https://en.wikipedia.org/wiki/Advanced_Video_Coding">AVC/H.264</a> published transparent pricing policy. Usually a chipmaker should pay some cents per chip if it supports AVC and starting from particular number of produced chips. For example:</p><blockquote>On August 26, 2010, MPEG LA announced that it wouldn’t charge royalties for H.264 encoded Internet video that is free to end users. All other royalties remain in force, such as royalties for products that decode and encode H.264 video and the operators of free television and subscription channels. The license terms are updated in the 5-year blocks.</blockquote><p>See some insights on <a href="https://www.mpegla.com/programs/avc-h-264/faq">MPEG LA</a>.</p><p>VP8/VP9 was published as <a href="https://en.wikipedia.org/wiki/Open_format">open</a> and <a href="https://en.wikipedia.org/wiki/Royalty-free">royalty-free</a> by Google, it is used by YouTube, Wikipedia and Netflix (VP9).</p><p>100% sure AVC was industrially successful. One of the proofs of that statement published in 2003 is that it is still widely used (end of 2020). VP8/VP9 is commonly used by Google for streaming and by “fans” of open formats.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/231/1*QsZNNuU63zXShpPDRzxIWw.png" /></figure><p>In 2013 MPEG LA published <a href="https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding">HEVC/H.265</a> but it was not so successful as AVC. Why? A common opinion is because of its pricing and distribution policy. The patent holders understood the popularity of the Internet especially video streaming services like Netflix, Twitch and YouTube, which take 19, 33 and 2 places respectively at “The top 500 sites on the web” by <a href="https://www.alexa.com/topsites">Alexa</a>. They wanted some bigger piece of the cake and published a new rule: chipmakers should pay for streaming services according to their traffic. That was not the last problem: some patent holders did not agree with the rems, so they created <a href="https://accessadvance.com/">HEVC Advance</a>. After that another segment of the patent holders formed the 3rd group. So, if earlier license buyer should pay to MPEG LA and use the codec, now he should go to 2? 3? groups who read all usage rights and payment…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/0*KBoMdT1ZobacGnkJ.jpg" /><figcaption>From MPEG Chairman Leonardo blog — “A crisis, the causes and a solution”. It represents variety of patent holders and pools.</figcaption></figure><p>To sum up:</p><ul><li>AVC has transparent pricing;</li><li>VP9 was OK and free but not so good as HEVC;</li><li>HEVC was technically sound and much better than AVC;</li><li>HEVC has a terrible pricing policy.</li></ul><p>As a result:</p><ul><li>AVC is still very popular;</li><li>VP9 is used predominantly by Google and Wikipedia;</li><li>HEVC is of very low popularity on the market.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*mJKAPxWMeXWDWNYv7747Ew.png" /><figcaption>Codec used by developers. Source data: <a href="https://go.bitmovin.com/video-developer-report-2020">ttps://go.bitmovin.com/video-developer-report-2020</a></figcaption></figure><h3>Codec industry: today</h3><p>Today people consume more and more traffic with video priority, as we can see in Cisco reports.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*XH_jO4XFYKKtppmQ.png" /><figcaption>One of the illustrations of traffic types by Cisco</figcaption></figure><p>Internet users acquire new habits and demand, QHD and UHD (4K) display moving to the middle segment of the market so they become more popular. All that increases the requirements for video bitrates and the Internet speed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*qc13zrlrXPqKbCnZ.png" /><figcaption>Illustration represents how many Mbps “eats” each type of application. Source: <a href="https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html">https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html</a></figcaption></figure><p>Even traditional “good-old” TV companies choose Internet platforms (such as YouTube, Twitch and so on) or develop their own platforms to distribute their content.</p><p>They all use a media stack that should be encoded, decoded, packaged, delivered, optimized, recovered and so on.</p><p>New applications such as VR, Cloud Gaming and already super popular Game Streaming consume more and more traffic. It should be efficiently coded to save the bitrate but as illustrated earlier the industry still uses codecs from another decade such as AVC, VP9 and even MPEG2.</p><p>HEVC should have been “the codec of today” but it did not become popular (How many popular TV models have HEVC support in their chips? How many content providers or streamers use HEVC?). It fell under the onslaught of robber license policy.</p><h3>Codec industry: tomorrow</h3><p>If you are still reading this article and thinking about what a naive author he is: yes, that was a naive approach. Except for the technologies and optimizations (Did you read about Jan Ozer’s LCEV x264 Report? :) ), there are also:</p><ol><li>Market inertia and conservatism</li><li>Local government regulations and push</li><li>Organization’s lobby their own technologies or standards</li><li>Company’s business models</li></ol><h4><strong>Market inertia and conservatism</strong></h4><p>IP TV providers were already spending millions of dollars and pushed to the market millions of TV boxes with AVC support, so why should they invest more money to push new TV boxes with for example AV1 support? What is their profit if AVC is “OK” for HD TV? Maybe it is better to tune the existing encoder/decoder by software update? Update cycles for such companies could be <strong>decades</strong>.</p><h4>Local government regulations and push</h4><p>Some countries like China push the local market to use specific standards like the AVS-codec family to prevent political risks. It is more related to technological independence and sanctions (which is not the article’s subject).</p><h4>Organizations lobby their own technologies or standards</h4><p>Google, Netflix, Amazon invested a lot in the development of the AV-family and already implemented support into their products, so they will continue to promote it on the market. For example Intel has already pushed the hardware and drivers with the AV1 backing:</p><p><a href="https://www.phoronix.com/scan.php?page=news_item&amp;px=Intel-Media-Driver-2020.3">Intel Media Driver 2020.3 Released With Gen12 AV1 Decode, Other Improvements - Phoronix</a></p><p>MPEG organization promotes VVC.</p><p>China’s government and companies work with AVS-family.</p><h4>Company’s business models</h4><p>There is a big difference between the “hardware companies” like IP TV providers and for example streaming companies (OTT). Streaming companies are far more dynamic. It is easier for them to implement support of the new codec, but they are susceptible to the licensing policy. They cannot pay for each stream.</p><p>As mentioned there are many codecs implemented or in development. Let’s sum up most of them in one list:</p><ul><li>AV1</li><li>AV2 (next gen of AV1, already in development)</li><li>VVC</li><li>EVC</li><li>LVC</li><li>AVS family</li><li>MPAI (“motion picture AI” — it is a try of using AI instead of classic compression methods in the codec)</li></ul><p>And so on!</p><p><strong>It seems like tomorrow’s codec market will be very fragmented by codecs and there is no vision that 1 or 2 codecs will “rule the world”. Companies should support some codecs in their products. Which specifically? It depends on the terms.</strong></p><h3>Conclusion</h3><p>As we can see video traffic had made the most significant segment of the Internet a long time ago. However, with the popularization of new applications such as Cloud Gaming, VR, streaming and also reduction of costs for 4K devices the task of high effective bandwidth reduction becomes critical, though at the same time the outdated technologies, distribution policy and legal wars slow down this process.</p><p>In addition it seems like “in the new world of tomorrow” each codec will take its place on the market according to the business strategy, model, policy, technologies, environment requirements and government restrictions.</p><h3>Hard days for the technology enablers? No! If…</h3><p>No, the lovely days for the industry and technology enablers are ahead if the companies choose optimum investments. Yeah, the <strong>challenges are big outstanding </strong>such as the fragmentation of the codec industry mentioned above, the problematic 2020 with increased bandwidth and servers loads, increased prices for some of the hardware and so on. Still, there were<strong> also new opportunities</strong> like new users, new practices and new technologies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/1*LsNvhAsfQTQ3LjLGK2SvAQ.png" /><figcaption>Simple example from “Bitmovin developer report 2020” about which metrics used to optimize encoding settings</figcaption></figure><p>So to be successful tomorrow, we need to choose the right business, scalability models and suitable technologies. As an example: <a href="https://bitmovin.com/press-room/is-the-party-over-video-streaming-services">Bitmovin`s report</a> highlights that <em>“51% of respondents indicated that cost control for such things as bandwidth and storage is their biggest challenge” and “</em>taking services and assets faster to market.<em>” </em>How can it be solved?</p><p>Companies should choose their right partners to support the variety of codecs or tune the existing one to improve bandwidth efficiency by up to 20%. In such a case, <a href="https://vicuesoft.com/">VicueSoft</a> can help with its services and products.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/273/1*KB3rMIHglTeHn2HL3ZSmPA.png" /></figure><h3>About VicueSoft</h3><p>VicueSoft is a company of high experienced experts in the domain of video. It offers services like contract tuning of codecs or enabling of specified media stack on the custom hardware and also own full stack products like <strong>VQ Analyzer</strong>, <strong>VQ Probe</strong> and <strong>DVK</strong> which mentioned below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UqHgNUHq1NRhO6ft.jpeg" /><figcaption><em>VicueSoft`s map of presence</em></figcaption></figure><h3>Vicuesoft`s products as driver of cost reduction</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/117/1*sWUthdJaQYzWjNRPHP3ITg.png" /></figure><h4>VQ Analyzer</h4><p>VQ Analyzer must used tool for any media expert it is a video bitstream analysis tool designed to inspect every essential step of the decoding process graphically and numerically. Inspect, debug and improve.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BMMrUSswId9lwCL6.jpg" /><figcaption>VQ Analyzer</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/95/1*IWAiKRlg0RwvHDrTsG7dUA.png" /></figure><h4>VQ Probe</h4><p>Lightweight tool for quick compare of streams, take some metrics and so on. In one sentence it is the instrument for video quality probation.</p><p>The key usage model is to compare two implementations of the same codec standard or two different revisions of a codec.</p><p><a href="https://vicuesoft.com/vq-probe">It can be downloaded free!</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F_fp7xSe7aW8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D_fp7xSe7aW8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F_fp7xSe7aW8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e3cf820d9ea59470f3ba1b19f97bf09d/href">https://medium.com/media/e3cf820d9ea59470f3ba1b19f97bf09d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_EFhcSs5rDWO-2_TaMabKw.png" /><figcaption>Compare two streams original YUV (left) and HEVC streams (right) in VQ Probe</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/127/1*m1VeB0tW4JoM_BEi9E8b8w.png" /></figure><h4>DVK</h4><p>Decoder Validation Kit (DVK) is is set of instruments which enable process of validation of Decoder IP for any codec (VVC, AV1, HEVC, AVC, VP9). It contains:</p><ul><li>streams</li><li>encoder with ability to generate stream with any predefined syntax coverage</li><li>instrumented decoder</li><li>analytic tools for report generation</li></ul><p>As an additional can be used special <strong>broken streams </strong>which will help to test stability of decoder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/0*Ezwg9mEY_o6N_nwc.jpg" /><figcaption>Syntax coverage report by DVK`s set of streams</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/571/0*-uzRbgz6PqV2jZIa.jpg" /><figcaption>A HEVC bitstream view in VQ Analyzer. Performance bitstream maximizes memory access bandwidth</figcaption></figure><p>In next articles will be showed <strong>work pipelines</strong> where these products can speed up and facilitate the work of codec engineers and validators.</p><h3>Used materials:</h3><ul><li><a href="https://vicuesoft.com/">ViCueSoft: Software for Video Professionals</a></li><li><a href="https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html">Cisco Annual Internet Report - Cisco Annual Internet Report (2018-2023) White Paper</a></li><li><a href="https://www.cisco.com/c/m/en_us/solutions/enterprise-networks/networking-report.html">Read the 2020 Global Networking Trends Report</a></li><li><a href="https://go.bitmovin.com/video-developer-report-2020">Bitmovin&#39;s 4th Annual Video Developer Report 2020</a></li><li><a href="https://bitmovin.com/press-room/is-the-party-over-video-streaming-services">Bitmovin&#39;s 2020/2021 Video Developer Report Revealing Insights into Streaming Industry</a></li><li><a href="https://blog.chiariglione.org/a-crisis-the-causes-and-a-solution/">A crisis, the causes and a solution - Leonardo&#39;s Blog</a></li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FJx2VxHnT7Fk%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DJx2VxHnT7Fk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FJx2VxHnT7Fk%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/12c2eabf652f4e87d4c6a32aea056471/href">https://medium.com/media/12c2eabf652f4e87d4c6a32aea056471/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ee2400cb54b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/%D1%81odec-war-the-show-must-go-on-2ee2400cb54b">Сodec War: the show must go on!</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decoder-Side Motion Vector Refinement (DMVR) in VVC]]></title>
            <link>https://medium.com/vicuesoft-techblog/decoder-side-motion-vector-refinement-dmvr-in-vvc-e13cd5edecaf?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/e13cd5edecaf</guid>
            <category><![CDATA[vvc]]></category>
            <category><![CDATA[video-coding]]></category>
            <category><![CDATA[h266]]></category>
            <category><![CDATA[decoder]]></category>
            <category><![CDATA[versatile-video-coding]]></category>
            <dc:creator><![CDATA[Denis Fedorov]]></dc:creator>
            <pubDate>Tue, 20 Jul 2021 15:58:04 GMT</pubDate>
            <atom:updated>2021-07-20T15:58:03.228Z</atom:updated>
            <content:encoded><![CDATA[<p>Welcome to a new article about Versatile Video Coding! Today we are delighted to tell you about a new VVC feature that uses motion estimation on the decoder side! For demonstration we will use <a href="https://vicuesoft.com/vq-analyzer">VQ Analyzer</a> — a bitstream analysis tool developed by <a href="https://vicuesoft.com/">ViCueSoft</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/1*S26uUw4ee6R9r80t11q0mA.gif" /></figure><p>Although the main idea of decoder side motion derivation (DMVD) is not new — it was well studied during HEVC development — it was applied for the first time in VVC. The goal of DMVD is to avoid transmitting motion information in a bitstream by allowing the decoder to search for motion by itself.</p><p>In VVC, DMVD evolves into decoder-side motion vector refinement (DMVR). DMVR is used to increase the precision of Merge Mode prediction without additional signaling.</p><p>There are two translational motion vector prediction modes in VVC: advanced motion vector prediction (AMVP) and Merge mode. They both use motion prediction candidates from spatial or temporal neighboring blocks. In AMVP mode, this information is signaled in the bitstream: candidate index, motion vector difference and reference picture index. In Merge mode, only the best candidate index is signaled (Figure 1). That’s why Merge mode uses less bits for motion coding, but also provides less accuracy. Here, DMVR takes place to increase the precision.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/941/1*tKORx4BB_9Q67sfjrp8z3w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OSaxzi9t72sS9vr0XXTAbg.png" /><figcaption>Pic 1. VQ Analyzer. Example of prediction units details with modes: AMVP (left) and Merge (right)</figcaption></figure><p>The result of DMVR is a refined motion vector. It is searched around the original MV prediction on L0 and L1 reference pictures. Search is based on calculating distortion between the L0 and L1 candidate samples (Figure 2). For simplification, the sum of absolute difference (SAD) is used as a distortion metric. Offset with lowest SAD is used for derivation of refined motion vectors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/0*mY071jrXkcG2mZo6" /><figcaption>Figure 2. Derivation of Δmv for refinement by offset with lowest SAD</figcaption></figure><h4>DMVR Algorithm</h4><p>VVC applies DMVR with bilateral-matching method, so it works only for bi-prediction Merge blocks. To reduce memory requirement and gain precision, the block is divided into subblocks for DMVR processing. Subblock size is up to 16x16, each subblock gets its own refinement independently.</p><p>In order to reduce computational complexity, search is divided into two stages: integer and fractional. Resulting refinement is the sum of both stages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/0*zT_SF_2HPYCPW58L" /></figure><p>Integer search is performed by SAD comparisons, and then fractional refinement is applied by using 2d parabolic error surface.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/911/0*P-0tLhi59VNorB-Z" /><figcaption>Figure 3. Multi stage DMVR search process in VVC</figcaption></figure><h4><strong>Algorithm stages applied for each subblock:</strong></h4><ol><li><strong>Initial interpolation</strong></li></ol><p>Prefetch predictions for reference pictures L0 and L1 according to original Merge candidate MV. DMVR search area is larger than block area, it is <em>(W+2R) x (H+2R)</em>, where <em>WxH</em> is block size and <em>R=2</em> is max searched offset. In order to avoid extra memory consumption and lower computation complexity 2-tap interpolation filter is used, instead of standard motion prediction 8-tap filter.</p><p><strong>2. Initial check</strong></p><p>Estimating initial prediction offset (0, 0) for SAD cost. Initial SAD value is lowered by weighting factor 0.75 to reduce the penalty of the uncertainty of DMVR refinement. If it is lower than threshold <em>(W x H)</em>, DMVR search is skipped, selected offset is <em>(0, 0)</em>.</p><p><strong>3. Search for integer offset part</strong></p><p>SAD is calculated for other integer offsets (-2..+2, -2..+2). Offset with minimum SAD is selected.</p><p>In order to reduce the amount of candidate block pairs, DMVR applies a mirroring property. The DMVR algorithm assumes translational motion. In Bi-prediction with one reference from past and one from future picture it’s likely that MVs require refinement with opposite directions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/278/0*CFWwtCoXD1K4IuUw" /></figure><p>Max searched offset <em>R=2, </em>block could be moved up to (-2..+2, -2..+2), with integer offset there are 5 x 5 = 25 block candidates in each picture. That would require 25 x 25 = 625 checks for a full search. With mirroring property the amount of candidate pairs is reduced to 25.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_UQ0EHSWN58YADaa" /><figcaption>Figure 4. Example of integer offset for DMVR refinement</figcaption></figure><p><strong>4. Search for fractional offset part</strong></p><p>Fractional part would require a lot more additional interpolations, buffering operations and SAD calculations. That’s why instead of a straight fractional offset search, VVC applies leveraging the integer SAD cost array in a 2D parametric error surface. Fractional search is used only when the integer offset is less than max R=2 for both sides.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*noT7wMCj-ekCR4g1" /><figcaption>Figure 5. Quadratic-function-based SAD cost surface model for fractional DMVR search stage</figcaption></figure><p>The center position cost and the costs at four neighboring positions from the center are used to fit a 2D parabolic error surface equation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/503/0*Au--j-tc-m2y8Ksv" /></figure><p>Solving the above equations by using the cost value of the five search points (0,0), (-1,0), (1,0), (0,-1), (0,1), the point (Xmin, Ymin) is computed as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/0*RJMYy1qBVommjhFt" /></figure><p><strong>5.</strong> <strong>Final motion compensation</strong></p><p>Motion compensation is done by refined MV (Figure 2) as a sum of integer and fractional parts. Standard 8-tap interpolation filter is used.</p><p>Let’s consider how block refinement works in VQ Analyzer. You can access a calculated SAD array for integer offsets, selected offset, refined mv and Δmv in the Details view of the Prediction block, using the Motion Buffer tab. On the screenshot below (Figure 6) we see a single prediction unit with homogeneous motion vectors L0 (-14,20) and L1 (16,-18), but each subblock has independent DMVR refinement. For the currently selected subblock, the best offset found is (0, -2) in pixel density.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q1DSIYDTVBczpo5kzw760A.png" /><figcaption>Figure 6. VQ Analyzer. Refined subblocks of PU coded with DMVR mode</figcaption></figure><h4><strong>Usage of refined vector</strong></h4><p>Reconstructed MV of the current block is used not only for its prediction, but also for prediction of future blocks and the deblocking loop filter process. In the case of block with DMVR, we have refined and original vectors.</p><p>Refined MV is used for: current block MCP and temporal MV prediction for future pictures.</p><p>Original MV is used for: deblocking process and spatial motion vector prediction for future CUs in the current picture.</p><h4><strong>DMVR limitations</strong></h4><p>To apply DMVR block, it should satisfy the following limitations:</p><ul><li>Prediction mode is Merge and block is bi-predicted (because of bilateral matching)</li><li>CU has more than 128 luma samples</li><li>CU height &gt;= 8, CU width &gt;= 8</li><li>One reference in the past and one reference in the future, because of assumption of translational motion</li><li>The POC differences are equal, to avoid MV scaling</li><li>Both reference pictures are short-term reference pictures</li><li>The following coding tools are not used: Affine, MMVD, BCW, Geo, CIIP, SbTMVP</li></ul><p>This is all for today. In this article, we demonstrated how <a href="https://vicuesoft.com/vq-analyzer">VQ Analyzer</a> from <a href="https://vicuesoft.com/">ViCueSoft</a> can visualize and display details for Merge mode with Decoder-Side Refinement tool in Versatile Video Codec. If you have any questions or comments, please feel free to add them to this article or write to us directly at <a href="/service/mailto:info@vicuesoft.com">info@vicuesoft.com</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e13cd5edecaf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/decoder-side-motion-vector-refinement-dmvr-in-vvc-e13cd5edecaf">Decoder-Side Motion Vector Refinement (DMVR) in VVC</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Low-Frequency Non-Separable Transform (LFNST) in VVC]]></title>
            <link>https://medium.com/vicuesoft-techblog/low-frequency-non-separable-transform-lfnst-in-vvc-6facc822be8f?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/6facc822be8f</guid>
            <category><![CDATA[lfnst]]></category>
            <category><![CDATA[transform]]></category>
            <category><![CDATA[h266]]></category>
            <category><![CDATA[video-coding]]></category>
            <category><![CDATA[vvc]]></category>
            <dc:creator><![CDATA[Kirill Suverov]]></dc:creator>
            <pubDate>Tue, 06 Jul 2021 14:41:07 GMT</pubDate>
            <atom:updated>2021-07-06T14:41:06.194Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Bmyy9vITP5jW1a8ip6qcg.jpeg" /></figure><p>In VVC, in comparison with HEVC, a new stage of transform called low-frequency non-separable transform(LFNST) was added. The main condition is that LFNST applies only for intra blocks for both luma/chroma components (separate chroma is enabled).</p><p>The main goal of LFNST is to further compress the redundancy between low-frequency primary transform coefficients, which are transform coefficients from the conventional direction intra prediction. We can also add that for elongated rectangles LFNST helps to concentrate coefficients exactly in the top right corner.</p><p>So how does it work? LFNST is applied between forward primary transform and quantization (on the encoder side) and between de-quantization and inverse primary transform (on the decoder side). LFNST consists of 2 modes: 4x4 LFNST (for blocks with width/height &lt; 8) and 8x8 LFNST (for blocks with width/height &gt;= 8).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*1SU8tqgs0X4Km8nwk38BXQ.png" /></figure><p>As a way of example, let’s consider 4x4 LFNST for block <strong>X</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/356/1*oAKWURdDOji8hcsXLkssNg.png" /></figure><p>First of all, we need to represent <strong>X</strong> as a one-dimensional vector:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/889/1*yicMqoWB-P27MnFDw45D4A.png" /></figure><p>After that, the non-separable transform can be calculated as: <strong>F = <em>T</em>⋅ X</strong> (<strong>F</strong> and <strong>X</strong> are vectors), where <strong><em>T</em></strong><em> </em>is 16x16 transform matrix and <strong>F</strong> is the 16x1 vector with transform coefficients which can be reorganized into a 4x4 block using a raster scan order.</p><p>In LFNST, amatrix multiplication approach is used, so to save memory space to store matrix coefficients and to decrease computational complexity it will be good if the matrix dimension can be minimized. In this way. the main idea is to map N dimensional vector to R dimensional vector, where N &gt; R and N/R is the reduction factor. For 8x8 LFNST the reduction factor is 4, so the transform matrix is 16x64. But in later stages of VVC it was further reduced to 16x48. So due to matrix dimensional reduction the memory usage for storing them decreased from 10Kb to 8Kb without much performance degradation.</p><p>For inverse process on the decoder side the inverse LFNST matrix is the transposed matrix of the forward transform matrix.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*cDk76UMIUEF50mjsOoeTNA.png" /></figure><p>LFNST has some restrictions and it cannot be used in the following cases:</p><p>1) block size is more than 64x64 or width/height CU are more than max transform_block_size;</p><p>2) if ISP is applied TU width or height is less than 4;</p><p>3) block has INTER mode;</p><p>4) transform skip is used;</p><p>5) blocks 4x4 and 8x8 have more than 8 first significant coefficients;</p><p>6) there are some coefficients out of the 1st subblock;</p><p>7) if there is only 1 non-zeroed coefficient (DC only);</p><p>It is important to note that LFNST and MTS are connected: if LFNST is enabled, only DCT-II mode (MTS index = 0) can be applied.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*pDZZpxvohQgGS9t7AHpXZw.png" /></figure><p>LFNST transform matrices <em>T</em> can be represented as follows:</p><p>- 8 different 16x16 matrices for 4x4 LFNST (Cartesian product of 4 lfnstTrSetIdx &amp; 2 lfnst_idx);</p><p>- 8 different 16x48 matrices for 8x8 LFNST;</p><p>All these matrices are precalculated and stored on the encoder/decoder side.</p><p>The choice of the best suitable LFNST matrix depends on lfnstTrSetIdx, which depends on IntraPredMode and lfnst_idxwhich is transmitted in bitstream. Below you can see the table of dependence of lfnstTrSetIdx on IntraPredMode:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*PZMgZJi_6LkEum8q9VD2Rw.png" /></figure><p>And here you can see, how LFNST works on decoder side:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/443/1*sRAS1JTdFMiNs7d3GvacUw.png" /></figure><p>This is all for today. In our next article we will continue telling you about VVC transform innovations. You can find more about our products in <a href="https://vicuesoft.com/">ViCueSoft website</a>. Also you can check our previous article about MTS <a href="https://vicuesoft.com/blog/titles/VVC_MTS">here</a>.</p><h3>References:</h3><p>- Algorithm description for Versatile Video Coding and Test Model 12 (VTM 12)</p><p>- Low Frequency Non-Separable Transform (LFNST) (<a href="https://ieeexplore.ieee.org/document/8954507">https://ieeexplore.ieee.org/document/8954507</a> )</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6facc822be8f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/low-frequency-non-separable-transform-lfnst-in-vvc-6facc822be8f">Low-Frequency Non-Separable Transform (LFNST) in VVC</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Subblock-based Temporal Motion Vector Prediction (SbTMVP) in VVC]]></title>
            <link>https://medium.com/vicuesoft-techblog/subblock-based-temporal-motion-vector-prediction-sbtmvp-in-vvc-2d9203e4a7b3?source=rss----a7648e30fa7f---4</link>
            <guid isPermaLink="false">https://medium.com/p/2d9203e4a7b3</guid>
            <category><![CDATA[h266]]></category>
            <category><![CDATA[codec]]></category>
            <category><![CDATA[vvc]]></category>
            <category><![CDATA[vidéo]]></category>
            <category><![CDATA[video-compression]]></category>
            <dc:creator><![CDATA[Denis Fedorov]]></dc:creator>
            <pubDate>Mon, 07 Jun 2021 09:56:31 GMT</pubDate>
            <atom:updated>2021-12-08T05:03:00.782Z</atom:updated>
            <content:encoded><![CDATA[<p>Welcome to a new article about Versatile Video Coding! Previously, we explained how <a href="https://medium.com/vicuesoft-techblog/affine-motion-compensated-prediction-in-vvc-f47979cfb626">Affine Merge Mode</a> works and now we’ll look at Subblock-based Temporal Motion Vector Prediction (SbTMVP). SbTMVP is a type of a merge mode too. Both these methods are connected in VVC by using the same merge context. For demonstration we will use <a href="https://vicuesoft.com/vq-analyzer">VQ Analyzer</a> — a bitstream analysis tool developed by <a href="https://vicuesoft.com/">ViCueSoft</a>.</p><figure><img alt="Figure 1. VQ Analyzer. Prediction mode with SbTMVP prediction blocks" src="https://cdn-images-1.medium.com/max/1022/1*cF3TbC_Gn6zb_gTCpAiuSA.png" /><figcaption>Figure 1. VQ Analyzer. Prediction mode with SbTMVP prediction blocks</figcaption></figure><p>The main idea of SbTMVP algorithm is to use motion information from the collocated picture. It is similar to TMVP in HEVC, but it works for CU-subblocks instead of whole CU. Motion vectors from the collocated picture are scaled.</p><figure><img alt="Figure 2. Deriving subblocks motion vectors in SbTMVP by applying shift from A1 neighbor and scaling subblocks motion vectors" src="https://cdn-images-1.medium.com/max/1024/1*bXE8G30mn2xsxePSqZ55AQ.png" /><figcaption>Figure 2. Deriving subblocks motion vectors in SbTMVP by applying shift from A1 neighbor and scaling subblocks motion vectors</figcaption></figure><p>SbTMVP algorithm is demonstrated in Figure 2. Its main steps are:</p><p>1. Divide CU onto small subblocks, subblock size is fixed to 8x8 pixels. As in Affine Merge Mode, SbTMVP is applicable for coding units which are larger then 8x8 pixels.</p><p>2. Derive motion shift on the collocated picture:</p><ul><li>If A1 neighbor has a motion vector that uses the collocated picture as its reference picture — use it</li><li>Otherwise, motion shift is set to zero MV (0,0)</li></ul><p>3. Apply derived motion shift in the collocated picture. Shift is added to the current block’s coordinates.</p><p>4. Get initial vector from the center of CU. it will be used as default for subblocks with unavailable motion information.</p><p>5. Identify subblocks motion information (from the center of subblocks). If any subblock is unavailable, use default MV.</p><p>6. Derive subblocks motion vectors and reference indices by applying scaling for corresponding subblocks. Scaling is done by using POC differences tb and td, as shown in Figure 3. Td is difference between collocated picture and reference, tb is difference between current picture and reference.</p><figure><img alt="Figure 3. Illustration of motion vector scaling for TMVP" src="https://cdn-images-1.medium.com/max/858/1*D80G4gP5sdscoQz4FMODjw.png" /><figcaption>Figure 3. Illustration of motion vector scaling for TMVP</figcaption></figure><p>When SbTMVP is enabled, the list of merge candidates consists of SbTMVP predictor and <a href="https://medium.com/vicuesoft-techblog/affine-motion-compensated-prediction-in-vvc-f47979cfb626">Affine Merge</a> candidates, as shown in Figure 4. SbTMVP candidate is added to the first position of the list. The SbTVMP mode is enabled by a sequence parameter set (SPS) flag.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9u5Q3LJGr1btLvifbxeh0A.png" /><figcaption>Figure 4. VQ Analyzer. Detail view of SbTMVP prediction unit</figcaption></figure><p>This is all for today. In this article, we demonstrated how <a href="https://vicuesoft.com/vq-analyzer">VQ Analyzer</a> from <a href="https://vicuesoft.com/">ViCueSoft</a> can visualize and display details for Subblock-based Temporal Motion Vector Prediction in Versatile Video Codec. If you have any questions or comments, please feel free to add them to this article or write to us directly at <a href="/service/mailto:info@vicuesoft.com">info@vicuesoft.com</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d9203e4a7b3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vicuesoft-techblog/subblock-based-temporal-motion-vector-prediction-sbtmvp-in-vvc-2d9203e4a7b3">Subblock-based Temporal Motion Vector Prediction (SbTMVP) in VVC</a> was originally published in <a href="https://medium.com/vicuesoft-techblog">vicuesoft-techblog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>