<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Luc Trudeau on Medium]]></title>
        <description><![CDATA[Stories by Luc Trudeau on Medium]]></description>
        <link>https://medium.com/@luc.trudeau?source=rss-3ab9a74ea2fc------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*upgDHimsqeka8Ot-2HakRw.jpeg</url>
            <title>Stories by Luc Trudeau on Medium</title>
            <link>https://medium.com/@luc.trudeau?source=rss-3ab9a74ea2fc------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:34:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@luc.trudeau/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Stop Guessing Your Stack Usage: Here’s an Easy Way to Measure It!]]></title>
            <link>https://medium.com/@luc.trudeau/stop-guessing-your-stack-usage-heres-an-easy-way-to-measure-it-70b205a4c945?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/70b205a4c945</guid>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Thu, 14 Nov 2024 20:48:13 GMT</pubDate>
            <atom:updated>2024-11-14T20:48:13.648Z</atom:updated>
            <content:encoded><![CDATA[<p>Did you know that Dav1d isn’t just about raw performance? Many other aspects of the decoder are optimized to ensure it remains lightweight and scalable across diverse platforms, including:</p><ul><li>Binary Size</li><li>Memory Consumption</li><li>Stack Size</li></ul><p>While stack size optimization might not seem like a high priority for most projects, it becomes critically important in a mature, widely distributed project like Dav1d.</p><p>At <a href="https://www.linkedin.com/company/videolan/">VideoLAN</a> Dev Days 2024, I had the privilege of learning about a simple yet powerful tool from <a href="https://www.linkedin.com/in/martinstorsjo/">Martin Storsjö</a>: the -fstack-usage flag in Clang. This tool has completely transformed how I approach stack optimization, enabling me to profile stack usage accurately and focus on the functions that matter most.</p><h3>Profiling Beats Guesswork</h3><p>This aligns perfectly with my broader philosophy of optimization: profiling beats guesswork every time. Without proper profiling, developers often fall into the trap of optimizing stack usage in an ad hoc manner, wasting time on functions that don’t actually make a difference.</p><p>With tools like Clang’s -fstack-usage, you can easily uncover the true stack-hungry culprits and direct your efforts where they’ll have the biggest impact.</p><h3>A Concrete Example from Dav1d 1.5</h3><p>Here’s how I used -fstack-usage to profile stack usage in the Dav1d 1.5 codebase:</p><ol><li>Adding the -fstack-usage Flag: I modified the Meson build file to include the flag and compiled a release build.</li><li>Generating .su Files: Each file contained the function name and its corresponding stack usage size.</li><li>Aggregating the Data: Using the cat command, I concatenated all the .su files into one large file.</li><li>Analyzing the Data: I imported the combined data into a spreadsheet editor, sorted the entries by descending stack size, and created the following graph:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/0*dGVXsif8_tVuH1Jj" /></figure><h3>Takeaway</h3><p>This graph highlights how profiling provides actionable insights. For instance, in Dav1d, the Self-Guided Filter (SGR) functions are by far the largest consumers of stack space. By focusing optimization efforts on these areas, we can achieve far greater efficiency than by tweaking smaller, less impactful functions.</p><p>This is why I believe tools like -fstack-usage are indispensable for anyone looking to optimize stack usage effectively.</p><h3>What About You?</h3><p>Have you tried using -fstack-usage or similar tools in your projects? What’s your approach to profiling and optimizing stack usage? Let me know your thoughts or experiences in the comments below — I’d love to hear from you!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70b205a4c945" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Did you know that SVT-AV1 uses linear models to determine deblocking levels?]]></title>
            <link>https://medium.com/@luc.trudeau/did-you-know-that-svt-av1-uses-linear-models-to-determine-deblocking-levels-5294beed12d0?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/5294beed12d0</guid>
            <category><![CDATA[video-compression]]></category>
            <category><![CDATA[image-deblocking]]></category>
            <category><![CDATA[av1]]></category>
            <category><![CDATA[video-coding]]></category>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Thu, 10 Oct 2024 12:01:51 GMT</pubDate>
            <atom:updated>2024-10-10T12:01:51.337Z</atom:updated>
            <content:encoded><![CDATA[<p>Have you ever wondered how recent video encoders manage deblocking strengths while maintaining high encoding speeds? In SVT-AV1, the secret lies in multiple linear models.</p><p>In this post, I’ll explain why linear models are used, what deblocking is, and what makes SVT-AV1’s approach unique.</p><h3>Why use a model?</h3><p>Before diving into SVT-AV1’s specific models, you might wonder: ‘Why use a linear model at all?’ The truth is that encoders don’t always rely on models — they can search for deblocking values in many ways, often with better results. However, linear models are much faster, making them ideal when speed is critical. In high-speed encoding, encoders may either use a linear model or disable deblocking altogether to save time.</p><h3>What’s deblocking again?</h3><p>Now that we understand why a linear model is used, let’s take a closer look at deblocking. Blocking artifacts arise from discontinuities at block boundaries due to the differences in encoding decisions between adjacent blocks. When the visual quality is high, these artifacts are subtle, but at lower bitrates, they can become more noticeable. Deblocking helps smooth these artifacts out, improving the overall quality of the video.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/0*yTipv4oQkRvgeal-.png" /><figcaption>Would it even be a tech blog post without a reference to XKCD? In this case, <a href="https://xkcd.com/598/">xkcd #598</a></figcaption></figure><p>The linear model leverages the relationship between quantization and deblocking: more quantization leads to lower visual quality, resulting in more blocking artifacts, which calls for stronger deblocking.”</p><h3>Resolution based-models</h3><p>The following chart illustrates the 3 models used by SVT-AV1 for 8-bit deblocking. The x-axis is the quantizer used internally by the encoder and the y-axis is the predicted deblocking level from 0 to 63, where 0 disabled deblocking and 63 is maximum deblocking strength.</p><p>Key frames, which contain more information because they are encoded independently from other frames, tend to have more blocking artifacts. This is why SVT-AV1 applies more aggressive deblocking to key frames compared to inter frames.</p><p>For inter frames, SVT-AV1 uses different models based on whether the frame is smaller than 720p or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*zpHcRMWCRMoZ1LNTvqBPYw.png" /></figure><h3>Don’t blur the image</h3><p>Highly predictable frames, such as those with minimal motion, are less likely to exhibit blocking artifacts. Therefore, SVT-AV1 disables deblocking to avoid unnecessary blurring.</p><h3>Conclusion</h3><p>So there you have it — that’s how SVT-AV1 selects deblocking values at high speeds. Was this article easy to follow? Are there any areas that could be clearer? Let me know your thoughts in the comments. And if you’re curious to learn more about SVT-AV1, their <a href="https://gitlab.com/AOMediaCodec/SVT-AV1/-/tree/master/Docs">documentation folder</a> is full of great insights into the inner workings of the AV1 encoder.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5294beed12d0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A pre-processing filter for SSIM]]></title>
            <link>https://medium.com/@luc.trudeau/a-pre-processing-filter-for-ssim-a60ce91f3374?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/a60ce91f3374</guid>
            <category><![CDATA[h264]]></category>
            <category><![CDATA[ssim]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[video-compression]]></category>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Mon, 12 Dec 2022 22:37:24 GMT</pubDate>
            <atom:updated>2022-12-12T22:37:24.767Z</atom:updated>
            <content:encoded><![CDATA[<p>In his paper <a href="http://www.reznik.org/papers/PCS2022_SSIMv2b.pdf">Another Look at SSIM Image Quality Metric</a>, Yuriy Reznik talks about a pre-processing filter that can improve <a href="https://en.wikipedia.org/wiki/Structural_similarity">SSIM</a> score achieved by a video encoder.</p><blockquote><em>[…] the design of a pre-processing filter that removes some of the low-frequency features and thus allows the encoder to encode the remaining high-frequency content with fewer errors. With proper tuning, such a filter may allow existing encoders to achieve improved SSIM scores.</em></blockquote><p>Well that sounds interesting, let’s give it a try and see if we can improve some SSIM score. For this experiment, let’s use <a href="http://r0k.us/graphics/kodak/kodim04.html">kodim04.png</a> from the <a href="http://r0k.us/graphics/kodak/">Kodak Lossless True Color Image Suite</a>, the same image used by Reznik in his paper.</p><p>Similar to <a href="https://github.com/Netflix/vmaf/issues/489">VMAF hacking</a>, let’s use the good old <a href="https://en.wikipedia.org/wiki/Unsharp_masking">unsharp mask</a> to remove low frequencies. I say old because it actually comes from the era of darkroom photography. To apply an unsharp mask on our image, let’s use <a href="https://www.gimp.org/">gimp</a> on it’s default settings (radius=3, amount=0.5 threshold=0.0). Gimp turns out to be a good choice for this experiment since it implements the unsharp mask using a Gaussian filter which is also what SSIM uses internally.</p><p>We end up with the following images. One could argue that after the unsharp mask, the image actually looks better. It might be that the human visual systems favours higher frequencies over lower ones.pre-processing filter</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*ej2hRdRHPuNlAZSQ9d2s9g.png" /></figure><p>Next, we encode both images. Similar to Reznik’s paper we will use H.264 with QP=42. This will produce noticeable compression artefacts</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*ZhGnA6HvxXaAzgkLord0gg.png" /></figure><p>Success, Gimp’s unsharp mask filter using default values allowed us to improve SSIM from 0.9275 to 0.9390, not bad. Looking at the textured areas before and after the unsharp mask it looks like we over compensated for the loss of high frequencies in the image after compression.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a60ce91f3374" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Experts are Wrong About AV1 Being Slow]]></title>
            <link>https://medium.com/@luc.trudeau/why-experts-are-wrong-about-av1-being-slow-1e9bda9f09d4?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e9bda9f09d4</guid>
            <category><![CDATA[videos]]></category>
            <category><![CDATA[video-compression]]></category>
            <category><![CDATA[codec]]></category>
            <category><![CDATA[av1]]></category>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Mon, 01 Jul 2019 02:32:23 GMT</pubDate>
            <atom:updated>2019-07-01T14:04:13.290Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gT1onR1rWtwagGGJFb2aWg.jpeg" /></figure><p>Imagine someone trying to convince you that English is too slow:</p><blockquote>“A French journalist can write an article in less than a day, while some English authors have taken years to write a single book, clearly English is too slow.”</blockquote><h3>A Video Format is a Language</h3><p>Video formats, like <a href="https://en.wikipedia.org/wiki/AV1">AV1</a>, <a href="https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding">HEVC</a> or <a href="https://en.wikipedia.org/wiki/Versatile_Video_Coding">VVC</a>, are languages to express video. Similar to how books are made up of chapters, pages, paragraphs and sentences, compressed video is made up of groups of pictures (GOPs), frames, tiles and macroblocks.</p><p>If I take 5 different individuals, show them the same scene and ask them to write a description of it; I will most likely get 5 descriptions of different lengths, that won’t be written in the same amount of time and that contain different words, yet all 5 will be valid English. The same is true for video encoders.</p><p>When you write a book or a post like this one, the time consuming task is not language-related, it is thinking of your content, breaking up your ideas into paragraphs and choosing the words in your sentences. The same applies for a video encoder, writing the bit stream is only a small fraction of the total encoding time, most of the time is spent choosing how to break the frame into blocks, choosing motion vectors, etc. We often refer to this as searching.</p><p>To recap, whether you write a book in French, English or Spanish, if you are fluent in that language, the language itself is probably not going to have a major impact on the time it will take. The same applies to video formats. Whether the bit stream is AV1, HEVC or VVC, most of the encoding time is spent searching.</p><h3>New English Words Improve the Language but Don’t Slow it Down</h3><p>Here’s where some experts go astray, commercial video encoders don’t need to search every coding tool. Just like I don’t need to consider the word <a href="https://en.wikipedia.org/wiki/Pneumonoultramicroscopicsilicovolcanoconiosis">pneumonoultramicroscopicsilicovolcanoconiosis</a> when I usually write a post. Adding more words to the English language does not make it slower, the same is true for video formats.</p><p>Commercial video encoders rely on search algorithms, that only consider a small relevant subset of the search space. These algorithms are what take up most of the encoding time and are not even part of the video format. In other words, you can use the same search algorithm in an H.264 encoder, an HEVC encoder or an AV1 encoder.</p><p>That’s why encoders like <a href="https://medium.com/netflix-techblog/introducing-svt-av1-a-scalable-open-source-av1-framework-c726cce3103a">SVT-AV1 from Intel and Netflix</a>, <a href="https://vimeo.com/blog/post/av1-new-standard-codecs/">Rav1e from Mozilla and Vimeo</a> and Cisco’s real time AV1 implementation are so important. These encoders implement different search algorithms that tailor to different needs. They also drive competition which will result in better search algorithms. <a href="https://medium.com/@luc.trudeau/3-important-takeaways-from-the-vimeo-av1-announcement-79fa2847eb61">In order for the AV1 video format to be a solution for everyone, everyone needs to get involved in making it right for them</a>.</p><h3>Can I encode AV1 at H.264 speeds? YES!</h3><p>Odds are, if you consider the same search space in two different video formats you will get a similar trade-off between visual quality and bit rate. That’s not because one codec is ripping off the other, it’s because of <a href="https://en.wikipedia.org/wiki/Entropy_(information_theory)">information entropy</a>.</p><p>While writing this post, I attended <a href="https://bigapple.video/">Big Apple Video 2019</a>, it was awesome to see <a href="https://vimeo.com/344366650">Thomas Davies from Cisco making a very similar point</a>. He refers to this as the golden triangle between codecs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GtqmyVR-wuGq0b3rePoivA.png" /><figcaption><a href="https://vimeo.com/344366650">The golden triangle between codecs presented by Thomas Davies at Big Apple Video</a>.</figcaption></figure><p>The golden triangle shows what happens to the compression efficiency (horizontal axis) of a video format as the search space decreases (vertical axis). Ideally, for a video format to be better than another video format, the better format must be above and to the left of the other format.</p><blockquote>“We observe significant real-world gains over H.264 and better complexity performance trade-offs than <a href="https://twitter.com/hashtag/HEVC?src=hash">HEVC</a>” — Thomas Davies at Big Apple Video 2019 talking about AV1</blockquote><p>This quote is important to understand; when compared to HEVC, AV1 offers a better complexity to encoding performance trade-off. What that means is that given the same processing time, you get more compression out of AV1 than HEVC (i.e. AV1 is above and to the left of HEVC). The reason for this is that you have more coding tools to chose from, so you can make better use of your time.</p><h3>But the AOM encoder is slow</h3><p>What confuses experts is that even with default settings, the<a href="https://en.wikipedia.org/wiki/Alliance_for_Open_Media"> Alliance for Open Media</a> (AOM) encoder is slow, but as we have seen, that does not mean that the AV1 format is too slow. The AOM encoder is just a small part of the golden triangle, as more AV1 encoders emerge, we will discover the true shape of the golden triangle.</p><p>Let’s go back to the example of an author taking years to write a book, in some situations this will make financial sense because the book will sell over 1 million copies. That’s exactly why the AOM encoder is slow in many settings, because for some companies, like Facebook, Amazon, Apple, Netflix and Google this makes financial sense.</p><p>In conclusion, those claiming that the AV1 format is slow aren’t considering all the facts. More coding tools don’t make formats slower, they have the potential to make them faster. Although we don’t know the exact shape of the golden triangle, current indicators show that AV1 is fast. Spoiler alert: when we have the exact shape of the golden triangle we won’t discover that AV1 is slow, but faster than what we thought possible (this already happened with x264).</p><p><em>Looking for more video codec talk? I strongly recommend checking out </em><a href="https://vimeo.com/344366650"><em>Thomas Davies talk</em></a><em> and the </em><a href="http://watch.bigapple.video/"><em>other awesome talks from the Big Apple Video</em></a><em> conference.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e9bda9f09d4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[3 Important Takeaways from the Vimeo AV1 Announcement]]></title>
            <link>https://medium.com/@luc.trudeau/3-important-takeaways-from-the-vimeo-av1-announcement-79fa2847eb61?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/79fa2847eb61</guid>
            <category><![CDATA[streaming]]></category>
            <category><![CDATA[av1]]></category>
            <category><![CDATA[videos]]></category>
            <category><![CDATA[vimeo]]></category>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Sat, 15 Jun 2019 02:07:42 GMT</pubDate>
            <atom:updated>2019-06-15T21:31:00.870Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*mqGocGeG_UViBIaOfUAUPg.jpeg" /></figure><p>Yesterday, <a href="https://vimeo.com/blog/post/av1-new-standard-codecs/">Vimeo announced support for the AV1 video format</a>. To my surprise, the announcement received some negative coverage as some think that it is not newsworthy.</p><p>Those dismissive of this announcement don’t realise that Vimeo is walking the walk and setting a great example for other “smaller” players, like other online streaming platforms and broadcasters (big and small) that would considerably increase their ROI in the AV1 format, by getting involved and making AV1 right for them.</p><p>Here are 3 important takeaways from the Vimeo announcement that some video experts overlooked:</p><h3>Not another FAANG AV1 deployment</h3><p>The wrong takeaway from the Vimeo announcement is: “Yet another member of the alliance deploying AV1, nothing to see here”. To my knowledge, Vimeo is the first streaming platform outside of Facebook, Amazon, Netflix and Google to announce support for the AV1 format.</p><p>This is huge, because AV1 is often wrongfully accused of being a FAANG video format. By deploying AV1, Vimeo is busting this myth and showing that AV1 is the video format for everyone.</p><h3>Not Google’s encoder</h3><p>Another first in this announcement is that Vimeo has chosen the <a href="https://github.com/xiph/rav1e">rav1e encoder</a> from Mozilla over Google’s AV1 encoder. This is equally important because, as far as I know, no one else has put this encoder into production.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eJ7ft2qMDZ9ungdz.jpg" /><figcaption>Image taken from <a href="https://blog.rom1v.com/2019/04/implementing-tile-encoding-in-rav1e/">Implementing tile encoding in rav1e</a></figcaption></figure><p>Similar to <a href="https://medium.com/netflix-techblog/introducing-svt-av1-a-scalable-open-source-av1-framework-c726cce3103a">Intel and Netflix with SVT-AV1</a>, Vimeo is not walking the beaten path with rav1e, which will greatly help diversify the AV1 ecosystem. This diversification not only drives competition between encoders, but also caters to the needs of different types of organisations.</p><h3>Skin in the game</h3><p>Last but certainly not least, Vimeo is actively involved in rav1e, they are “<em>contributing to its development and participating in the community</em>”. This is a crucial element, to the success of AV1 outside the FAANG. In order for the AV1 video format to be a solution for everyone, everyone needs to get involved in making it right for them.</p><p>That’s exactly what Vimeo did when <a href="https://medium.com/vimeo-engineering-blog/vimeo-joins-the-alliance-for-open-media-51065720e030">it got involved in the alliance for open media</a> and continues to do with it’s involvement in rav1e. Kudos Vimeo team, and keep us updated on your progress in your journey towards AV1; regardless of what others say about it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79fa2847eb61" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AV1, Opportunity or Threat for POWER and ARM Servers?]]></title>
            <link>https://medium.com/@luc.trudeau/av1-opportunity-or-threat-for-power-and-arm-servers-6871007ad99e?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/6871007ad99e</guid>
            <category><![CDATA[power9]]></category>
            <category><![CDATA[arm]]></category>
            <category><![CDATA[ibm]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[av1]]></category>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Mon, 09 Jul 2018 13:43:11 GMT</pubDate>
            <atom:updated>2018-07-09T17:12:27.322Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*38a0qRlRKcT3luKr" /></figure><p>While I haven’t seen an official announcement, Phoronix reported that the <a href="https://aomedia.googlesource.com/aom/">AV1 git repository</a> was <a href="https://www.phoronix.com/scan.php?page=news_item&amp;px=AOMedia-AV1-1.0.0-Tagged">tagged 1.0</a>, so the launch announcement is imminent. If you haven’t heard about it already, <a href="https://en.wikipedia.org/wiki/AV1">AOMedia Video 1 (AV1)</a> is an open, royalty-free video coding format by the <a href="https://aomedia.org/">Alliance for Open Media</a>.</p><h3>Not a codec war</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*v6BaIeaZPXshwim8" /></figure><p>In what some are hyping as a <a href="https://www.ibc.org/delivery/codec-wars-the-battle-between-hevc-and-av1/2710.article">codec war</a>, AV1 is disrupting the traditional video landscape. It’s the first next-gen video format that, on release, is planned to be supported in all browsers and never has a video format launched with such a <a href="https://aomedia.org/membership/members/">strong industry backing.</a></p><p>My personal opinion is that AV1 will disrupt cloud video services. Encoding AV1 is more than an order of magnitude more complex than encoding current generation video formats, and considering the bandwidth savings promised by AV1, the encoding cost will be a much bigger part of the <a href="https://en.wikipedia.org/wiki/Total_cost_of_ownership">total cost of ownership</a> of video content. This order of magnitude increase in complexity can clearly be seen in this figure from <a href="https://code.fb.com/video-engineering/av1-beats-x264-and-libvpx-vp9-in-practical-use-case/">the analysis done by Facebook of AV1</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qkwCsmT5sQvNsZ6z" /></figure><p>Since the release of the Facebook report, many engineers have worked hard to reduce AV1 encode time, but the reality of the matter is that, as AV1 becomes the de facto video format for the web, content providers will apply much more scrutiny on the efficiency of video coding infrastructure they buy or rent.</p><p>As an active member of the Alliance, <a href="https://en.wikipedia.org/wiki/Intel">Intel</a> has contributed significant resources to help speed up AV1. It follows that a part of the aforementioned AV1 speed ups come from <a href="https://aomedia-review.googlesource.com/q/AVX2">Intel specific software acceleration</a>. Concretely, non-Intel architectures are currently at a disadvantage when encoding AV1.</p><h3>POWER and ARM-based servers currently at a disadvantage</h3><p>Increased scrutiny of AV1 coding efficiency combined with less efficient encodes are important considerations for companies challenging <a href="https://www.nextplatform.com/2018/03/01/server-market-booms-last/">Intel’s dominance in the server market</a>.</p><p>For a concrete example, let’s consider two alternatives to <a href="https://www.intel.com/content/www/us/en/products/servers.html">Intel Xeon Servers</a>: the <a href="https://www.cavium.com/product-thunderx2-arm-processors.html">ThunderX2</a>, an ARM-based server by <a href="https://en.wikipedia.org/wiki/Cavium">Cavium</a> and the POWER9 powered <a href="https://www.raptorcs.com/TALOSII/prerelease.php">Talos II</a>, by <a href="https://www.raptorcs.com/">Raptor Computing Systems</a>. Both of these second generation offerings have considerably closed the gap with the Intel Xeon.</p><p>Even though both ThunderX2 and the Talos II offer a higher thread count than their Xeon counterparts, without software acceleration, both processors won’t outperform their Xeon counterparts at AV1 encoding.</p><p>At least that’s my opinion, we will have to wait for benchmarks to confirm this. While we wait for those, we can look at the latest <a href="https://www.phoronix.com/scan.php?page=article&amp;item=power9-talos-2&amp;num=1">13-Way IBM POWER9 Talos II vs. Intel Xeon vs. AMD x264 Benchmarks from Phoronix</a> and see that, as Phoronix states, POWER9 “could use improvement around multimedia/encoding”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/0*4lTii2QbwU3eV-OH" /></figure><p>In the Phoronix article, the Talos II systems are top contenders in many of the benchmarks, but not when it comes video encoding. While the article does not run benchmarks on AV1, it’s reasonable to assume that the complexity increase of encoding AV1 will only make things worse.</p><h3>Higher Thread Count, a Killer Feature for Video Encoding?</h3><p>One thing to note about the higher thread count on the Talos II is that it is the result of a higher thread count per core. Could this higher thread count per core be a killer feature for video encoding?</p><p>Let’s find out, thanks to Raptor Computing System and <a href="https://www.integricloud.com/">IntegriCloud</a>, I have access to a dual 8 core Talos II. That’s a grand total of 64 threads. Time to run some libvpx encodes.</p><p>To test this assumption, I used libvpx v1.7.0 “Mandarin Duck” and set the threads parameter to either 1, 2 and 4. I measured the time required to run batches of 1, 2, 4, 8, 16, 32 and 64 parallel instances of libvpx. Finally, I divided the measured time by the total number of frames encoded by that batch of parallel instances. For those following at home, you can reproduce this experiment using <a href="https://gist.github.com/luctrudeau/1e5773492e534ce0a61a464eba78bbf9">this script</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WYdvkC83Id_ruLsp" /></figure><p>As expected, increasing the number of encoding threads increases the number of frames per minute. However, for a given number of threads, single threaded encodes almost always offer the highest number of frames per minute. For example, if you have 4 threads, 4 parallel single threaded encodes will results in 43 frames per minute (fpm) whereas, 2 parallel encodes using 2 threads each will output 36.5 fpm and a single 4 threaded encode will produce 30.8.</p><p>As we can see in the following figure, the POWER9 scales almost linearly when the number of parallel encodes is below the number of cores (the Talos II has 16 cores). As we approach 16 parallel encodes, performance are slightly below the linear approximation. This could be caused by contention on core resources shared by threads.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sT3H6FFYwMEuw4Dk" /></figure><p>Sadly, but not surprisingly, when the number of parallel encodes exceed the number of cores the POWER9 does not scale linearly anymore. It would appear to be closer to a log function.</p><p>There’s a slight unexplained regression at 32 parallel encodes. Regressions appeared for every configuration I tested that used a total of 32 threads (1x32, 2x16, 4x8). This could be caused by more resource contention or IO. Let me know if you have ideas or assumptions on what could be causing this.</p><h3>Opportunity</h3><p>Now here’s the interesting thing, a higher thread count multiplies the impact of the platform specific software acceleration. This can be seen in the following figure, For example if we compare Mandarin Duck to the latest Master (show in green), which contains our <a href="https://medium.com/@luc.trudeau/video-compression-bounty-hunters-c8edf43d440">latest acceleration bounties</a>, we see the gains increasing along with the number of parallel encodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZcrVGFMEZcuk5cMx" /></figure><p>While our 64 thread Talos II encodes about 200 frames per minute on libVPX 1.7 “Mandarin Duck”, our latest POWER specific software accelerations bounties have increased the throughput of the Talos II to 246 frames per minute.</p><p>Based on our recent <a href="https://gist.github.com/luctrudeau/9d654afe9f9df0792b5a621aaa404cfa">profilings of libVPX on POWER</a>, we have updated our <a href="https://www.bountysource.com/trackers/46650841-lu-zero-libvpx">bountysource bounties</a>, and we are confident that if our bounties are funded, we can reach and exceed 300 frames per minute in time for the libVPX 1.8 release.</p><p>As for AV1, we have <a href="https://gist.github.com/luctrudeau/244a87c31bbf8060a6c1d3a198c2d2c4">profiled libaom on POWER</a>, and our <a href="https://www.bountysource.com/trackers/96183110-lu-zero-aom">bountysource bounties</a> are ready to be funded. So stay tuned for updates on AV1 speed ups for POWER as we achieve them.</p><h3>Conclusion</h3><p>The multiplicative effect of platform specific software acceleration on a high thread count machines could be a competitive advantage when encoding AV1. We have shown that our work has greatly increased the throughput of libVPX on the Talos II and could potentially have an even greater impact on AV1. Will this turn into an opportunity for POWER and ARM servers? Only time will tell.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6871007ad99e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Video Compression Bounty Hunters]]></title>
            <link>https://medium.com/@luc.trudeau/video-compression-bounty-hunters-c8edf43d440?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8edf43d440</guid>
            <category><![CDATA[vp9]]></category>
            <category><![CDATA[bounty-program]]></category>
            <category><![CDATA[power9]]></category>
            <category><![CDATA[videos]]></category>
            <category><![CDATA[streaming-video]]></category>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Thu, 14 Jun 2018 20:11:41 GMT</pubDate>
            <atom:updated>2018-06-14T20:11:41.054Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h264vvyaNhcCWP3vnNS16Q.jpeg" /></figure><p>In this post, we (<a href="https://twitter.com/lu_zero_">Luca Barbato</a> and Luc Trudeau) joined forces to talk about the awesome work we’ve been doing on <a href="https://en.wikipedia.org/wiki/AltiVec">Altivec/VSX</a> optimizations for the <a href="https://www.webmproject.org/code/">libvpx</a> library, you can read it here on on <a href="https://blogs.gentoo.org/lu_zero/">Luca’s blog</a>.</p><p>Both of us where in Brussels for <a href="https://fosdem.org/2018/">FOSDEM 2018</a>, Luca <a href="https://blogs.gentoo.org/lu_zero/2018/02/14/rust-av-rust-and-multimedia/">presented his work on </a><a href="https://github.com/rust-av/rust-av">rust-av</a> and Luc was there to <a href="https://twitter.com/trudluc/status/960531177479254016">hack on rav1e</a> — a<a href="https://github.com/xiph/rav1e">n experimental AV1 video encoder in Rust</a>.</p><p>Luca joined the rav1e team and helped give hints about how to effectively leverage rust. Together, we worked on <a href="https://github.com/xiph/rav1e/blob/master/src/predict.rs">AV1 intra prediction code</a>, among the other things.</p><blockquote><strong>Luc Trudeau</strong>: I was finishing up my work on <a href="https://vimeo.com/269083327">Chroma from Luma in AV1</a>, and wanted to stay involved in royalty free open source video codecs. When Luca talked to me about <a href="https://www.bountysource.com/trackers/46650841-lu-zero-libvpx">libvpx bounties on Bountysource</a>, I was immediately intrigued.</blockquote><blockquote><strong>Luca Barbato</strong>: Luc just <a href="https://twitter.com/trudluc/status/979349356544278530">finished implementing the Neon version of his CfL</a> work and I wondered how that code could work using VSX. I prepared some of the machinery that was missing in libaom and <a href="https://twitter.com/trudluc/status/981736758630109184">Luc tried his hand on Altivec</a>. We still had some pending libvpx work sponsored by IBM and I asked him if he wanted to join in.</blockquote><h3>What’s libvpx?</h3><p>For those less familiar, libvpx is the official Google implementation of the <a href="https://en.wikipedia.org/wiki/VP9">VP9</a> video format. VP9 is most notably used in <a href="https://youtube-eng.googleblog.com/2015/04/vp9-faster-better-buffer-free-youtube.html">YouTube</a> and <a href="https://medium.com/netflix-techblog/more-efficient-mobile-encodes-for-netflix-downloads-625d7b082909">Netflix</a>. VP9 playback is available on some browsers including <a href="https://caniuse.com/#search=vp9">Chrome, Edge and Firefox</a> and also on Android devices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GVcL94L_yNWzKExa" /><figcaption>Ref: <a href="https://caniuse.com/#search=vp9">caniuse.com VP9 support in browsers.</a></figcaption></figure><h3>Why use VP9, when the de facto video format is H.264/AVC?</h3><p>Because VP9 is <a href="https://www.cnet.com/news/google-reaches-deal-with-mpeg-la-over-its-vp8-video-codec/">royalty free</a> and the <a href="https://medium.com/netflix-techblog/more-efficient-mobile-encodes-for-netflix-downloads-625d7b082909">bandwidth savings are substantial when compared to H.264</a> when playback is available (<a href="https://ngcodec.com/news/2017/10/21/why-we-are-supporting-vp9-and-av1">an estimated 3.3B devices support VP9</a>). In other words, having VP9 as a secondary codec can pay for itself in bandwidth savings by not having to send H.264 to most users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*GYPpuYfb_s5Hk_to" /><figcaption>Ref: <a href="https://medium.com/netflix-techblog/more-efficient-mobile-encodes-for-netflix-downloads-625d7b082909">Netflix VP9 compression analysis.</a></figcaption></figure><h3>Why care about libvpx on Power?</h3><p>Dynamic adaptive streaming formats like <a href="https://en.wikipedia.org/wiki/HTTP_Live_Streaming">HLS</a> and <a href="https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP">MPEG DASH</a> have completely changed the game of streaming video over the internet. Streaming hardware and custom multimedia servers are being replaced by web servers.</p><p>From the servers’ perspective streaming video is akin to serving small videos files; lots of small video files! To cover all clients and most network conditions a considerable amount of video files must be encoded, stored and distributed.</p><p>Things are changing fast and while the total cost of ownership of video content for previous generation video formats, like H.264, was mostly made up of bandwidth and hosting, encoding costs are growing with more complex video formats like <a href="https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding">HEVC</a> and VP9.</p><p>This complexity is <a href="https://code.facebook.com/posts/253852078523394/av1-beats-x264-and-libvpx-vp9-in-practical-use-case/">reported to have grown exponentially with the upcoming AV1 video format</a>. A video format, built on the libVPX code base, by the <a href="https://en.wikipedia.org/wiki/Alliance_for_Open_Media">Alliance for Open Media</a>, of which <a href="https://aomedia.org/membership/members/">IBM is a founding member</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bMNRE0yjoJVf9l89" /><figcaption><em>Ref: Facebook’s AV1 complexity analysis</em></figcaption></figure><p>At the same time, IBM and its partners in the<a href="https://en.wikipedia.org/wiki/OpenPOWER_Foundation"> OpenPower Foundation</a> are releasing some very impressive hardware with the new <a href="https://en.wikipedia.org/wiki/POWER9">Power9</a> processor line up. Big Iron Power9 systems, like the <a href="https://www.phoronix.com/scan.php?page=news_item&amp;px=Raptor-Talos-2-Lite">Talos II</a> from <a href="https://www.raptorcs.com/TALOSII/prerelease.php">Raptor Computing Systems</a> and the collaboration between <a href="https://cloudplatform.googleblog.com/2016/10/introducing-Zaius-Google-and-Rackspaces-open-server-running-IBM-POWER9.html">Google</a> and <a href="https://blog.rackspace.com/barreleye-g2-zaius-pvt-entry-and-motherboard-at-openpower-opencompute-and-ibm-think">Rackspace</a> on Zaius/Barreleye servers, are ideal solutions to the tackle the growing complexity of video format encoding.</p><p>However, these awesome machines are currently at a disadvantage when encoding video. Without the platform specific optimizations, that their competitors enjoy, the Power9 architecture can’t be fully utilized. This is clearly illustrated in the x264 benchmark released in a <a href="https://www.phoronix.com/scan.php?page=article&amp;item=power9-epyc-xeon&amp;num=2">recent Phoronix article</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/0*4Hx-R2w-JZ4fL5wn" /><figcaption>Ref: <a href="https://www.phoronix.com/scan.php?page=article&amp;item=power9-epyc-xeon&amp;num=2">Phoronix x264 server benchmark.</a></figcaption></figure><p>Thanks to the optimization bounties sponsored by IBM, we are hard at work bridging the gap in libvpx.</p><h3>Optimization bounties?</h3><p>Just like <a href="https://en.wikipedia.org/wiki/Bug_bounty_program">bug bounty programs</a>, optimization make for great bounties. Companies that see benefit in platform specific optimizations for video codecs can sponsor our bounties on the <a href="https://www.bountysource.com/trackers/46650841-lu-zero-libvpx">Bountysource platform</a>.</p><p>Multiple companies can sponsor the same bounty, thus sharing cost of more important bounties. Furthermore, bounties are a minimal risk investment for sponsors, as they are only paid out when the work is completed (and peer reviewed by libvpx maintainers)</p><p>Not only is the Bountysource platform a win for companies that directly benefit from the bounties they are sponsoring, it’s also a win for developers (like us) who can get paid to work on free and open source projects that we are passionate about. Optimization bounties are a source of sustainability in the free and open source software ecosystem.</p><h3>How do you choose bounties?</h3><p>Since we’re a small team of bounty hunters (<a href="https://github.com/lu-zero">Luca Barbato</a>, <a href="https://github.com/sasshka">Alexandra Hájková</a>, <a href="https://github.com/rafaeldelucena">Rafael de Lucena Valle</a> and <a href="https://github.com/luctrudeau">Luc Trudeau</a>), we need to play it smart and maximize the impact of our work. We’ve identified two common use cases related to streaming on the Power architecture: YouTube-like encodes and real time (a.k.a. low latency) encodes.</p><p>By profiling libvpx under these conditions, we can determine the key functions to optimize. The following charts show the percentage of time spent the in top 20 functions of the libvpx encoder (without Altivec/VSX optimisations) on a Power8 system, for both YouTube-like and real time settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xwHiW7VORB2Mgiq6" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9f_894xFxMsANnLF" /></figure><p>It’s interesting to see that the top 20 functions make up about 80% of the encoding time. That’s similar in spirit to the <a href="https://en.wikipedia.org/wiki/Pareto_principle">Pareto principle</a>, in that we don’t have to optimize the whole encoder to make the Power architecture competitive for video encoding.</p><p>We see a similar distribution between YouTube-like encoding settings and real time video encoding. In other words, optimization bounties for libvpx benefit both Video on Demand (VOD) and live broadcast services.</p><p>We add <a href="https://www.bountysource.com/trackers/46650841-lu-zero-libvpx">bounties on the Bountysource platform</a> around common themed functions like: convolution, sum of absolute differences (SAD), variance, etc. Companies interested in libvpx optimization can go and fund these bounties.</p><h3>What’s the impact of this project so far?</h3><p>So far, we delivered multiple libvpx bounties including:</p><ul><li>Convolution</li><li>Sum of absolute differences (SAD)</li><li>Quantization</li><li>Inverse transforms</li><li>Intra prediction</li><li>etc.</li></ul><p>To see the benefit of our work, we compiled the latest version of libVPX with and without VSX optimizations and ran it on a Power8 machine. Note that the C compiled versions can produce Altivec/VSX code via <a href="https://en.wikipedia.org/wiki/Automatic_vectorization">auto vectorization</a>. The results, in frames per minutes, are shown below for both YouTube-like encoding and Real time encoding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/0*3kNEed5cAzq4IYFs" /></figure><p>Our current VSX optimizations give approximately a 40% and 30% boost in encoding speed for YouTube-like and real time encoding respectively. Encoding speed increases in the range of 10 to 14 frames per minute can considerably reduce cloud encoding costs for Power architecture users.</p><p>In the context of real time encoding, the time saved by the platform optimization can be put to good use to improve compression efficiency. Concretely, a real time encoder will encode in real time speed, but speeding up the encoders allows for operators to increase the number of <a href="https://developers.google.com/media/vp9/settings/vod/">coding tools</a>, resulting in better quality for the viewers and bandwidth savings for operators.</p><h3>What’s next?</h3><p>We’re energized by the impact that our small team of bounty hunters is having on libvpx performance for the Power architecture and we wanted to share it in this blog post. We look forward to getting even more performance from libvpx on the Power architecture. Expect considerable performance improvement for the Power architecture in the next libvpx release (1.8).</p><p>As IBM targets its Power9 line of systems at heavy cloud computations, it seems natural to also aim all that power at tackling the growing costs of AV1 encodes. This won’t happen without platform specific optimizations and the time to start is now; as the AV1 format is being finalized, everyone is still in the early phases of optimization. We are currently working with our sponsors to set up AV1 bounties, so stay tuned for an upcoming post.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8edf43d440" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[VSX Intrinsics Documentation]]></title>
            <link>https://medium.com/@luc.trudeau/vsx-intrinsics-documentation-7b18e64671cd?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b18e64671cd</guid>
            <category><![CDATA[altivec]]></category>
            <category><![CDATA[simd]]></category>
            <category><![CDATA[powerpc]]></category>
            <category><![CDATA[vsx]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Wed, 16 May 2018 13:10:35 GMT</pubDate>
            <atom:updated>2018-05-16T13:10:35.611Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*4JghpgPFOrWj1TbQcfXPAg.png" /></figure><p>I recently wrote a <a href="https://aomedia.googlesource.com/aom/+/22296e4badeadb84b50238d7f0cd933c1b02fbaf">VSX intrinsic patch for Chroma from Luma (CfL) in AV1 </a>and figured I could write about the experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*-wn7KHBzcPsbuT2a." /><figcaption><a href="https://twitter.com/trudluc/status/981735846360514560">https://twitter.com/trudluc/status/981735846360514560</a></figcaption></figure><p>The biggest issue I had getting started with VSX intrinsics was documentation. Information about VSX intrinsics is spread across multiple sources, which makes it hard to know which documents to follow.</p><p>In this post, we will go over the most helpful resources I’ve come across (so far).</p><h3><a href="https://www.ibm.com/support/knowledgecenter/SSGH2K_13.1.2/com.ibm.xlc131.aix.doc/compiler_ref/vec_intrin_cpp.html">XL C Vector built-in functions</a></h3><p>The most useful documentation source is the section <a href="https://www.ibm.com/support/knowledgecenter/SSGH2K_13.1.2/com.ibm.xlc131.aix.doc/compiler_ref/vec_intrin_cpp.html">vector built-in functions</a> of IBM’s XL C compiler documentation. The table of content (<em>which is hidden by default and can be revealed by clicking on the arrow on the left side of the page</em>) allows to quickly search (<em>using your browser search function</em>) for intrinsics. The documentation given is rather limited, but the supported type tables are often useful.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CK7sJh5QDNiw-FtxaV4IoA.png" /><figcaption>Vector permute instruction (vec_permi) documentation from the vector built-in functions of IBM’s XL C compiler.</figcaption></figure><p>Note: that this documentation is not 100% compatible with GCC, but still very good. For example, vec_perm is not available in the previous list, but it is available in GCC.</p><h3><a href="https://ibm.box.com/s/jd5w15gz301s5b5dt375mshpq9c3lh4u">Power ISA v2.07B</a></h3><p>A good complement to the <a href="https://www.ibm.com/support/knowledgecenter/SSGH2K_13.1.2/com.ibm.xlc131.aix.doc/compiler_ref/vec_intrin_cpp.html">vector built-in functions</a> is the Power ISA. This document does not cover VSX intrinsics directly, it lists the PowerPC instructions (from which the intrinsics are based on).</p><p>Since I was given access to a Power8, I used the <a href="https://ibm.box.com/s/jd5w15gz301s5b5dt375mshpq9c3lh4u">Power ISA v2.07B</a>. In this guide, chapter 6 (p. 213) list all the VSX Integer SIMD instructions. This chapter contains a good amount of documentation on each instruction, and the instructions are grouped by features ( Vector Integer Add Instructions, Vector Integer Multiply Instructions, Vector Integer Compare Instructions…). This is very useful, if you only have a general idea of what you want to do, and you want to learn the different instructions available.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*0vgG59IQ5hnfsGiyCmE2pw.png" /><figcaption>Vector permute instruction in the Power ISA v2.07B</figcaption></figure><h3><a href="https://www.nxp.com/docs/en/reference-manual/ALTIVECPIM.pdf">Altivec PIM</a></h3><p>The <a href="https://www.nxp.com/docs/en/reference-manual/ALTIVECPIM.pdf">Altivec PIM</a> is an old (1999) guide to altivec intrinsics.</p><blockquote>AltiVec is a <a href="https://en.wikipedia.org/wiki/Trademark">trademark</a> owned solely by Freescale, so the system is also referred to as <strong>Velocity Engine</strong> by Apple and <strong>VMX</strong> (<strong>Vector Multimedia Extension</strong>) by IBM and P.A. Semi.<br><a href="https://en.wikipedia.org/wiki/AltiVec">Ref: https://en.wikipedia.org/wiki/AltiVec</a></blockquote><p>Since VSX is an extension of VMX, this guide cover a lot of the core intrinsics I used to write my VSX patch. What I really like about this guide is that it’s well documented and contains a lot of illustrations.</p><p>In chapter 4, at p. 54 you get the operation description sheets of Altivec intrinsics. You can compare the vec_perm documentation of the <a href="https://www.ibm.com/support/knowledgecenter/SSGH2K_13.1.2/com.ibm.xlc131.aix.doc/compiler_ref/vec_intrin_cpp.html">vector built-in functions</a> and the <a href="https://ibm.box.com/s/jd5w15gz301s5b5dt375mshpq9c3lh4u">Power ISA v2.07B</a> in the previous screenshots with that of the <a href="https://www.nxp.com/docs/en/reference-manual/ALTIVECPIM.pdf">Altivec PIM</a> in the following screen shot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/1*0CeFE4R9s32EKcDwBREA_A.png" /><figcaption>Vector permute intrinsics in the Altivec PIM</figcaption></figure><h3><a href="https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W51a7ffcf4dfd_4b40_9d82_446ebc23c550/page/Intel%20SSE%20to%20PowerPC%20AltiVec%20migration">Intel SSE to PowerPC AltiVec migration</a></h3><p>The idea here is great, a <a href="https://en.wikipedia.org/wiki/Rosetta_Stone">Rosetta Stone</a> of intrinsics, but like the Rosetta Stone, parts are missing. I originally planned on using this document as my main source of documentation for my VSX patch, but quickly discovered that a lot of the Intel intrinsics I was looking for weren’t included.</p><p>Don’t get me wrong there’s a lot of Intel intrinsics here, I think the issue is more that there’s not always a 1 to 1 translation between Intel SIMD and VSX.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/1*-l5M2A0F3BZ9jHhUWmSqUQ.png" /><figcaption>Intel SSE to PowerPC Altivec mapping table.</figcaption></figure><p>This resource also contains a section titled GNU/GCC PowerPC AltiVec API quick reference, one thing that’s very useful about it, is the comment column which can give interesting information or indicate other recommended alternative intrinsics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/845/1*uEMCTtwE54Bv5M7d0ivYHw.png" /><figcaption>GNU/GCC PowerPC AltiVec API quick reference</figcaption></figure><h3><a href="https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/PowerPC-AltiVec_002fVSX-Built-in-Functions.html">GCC PowerPC AltiVec Built-in Functions</a></h3><p>This one comes up very often when searching for VSX intrinsics, It’s a list of all the VSX intrinsics found in GCC. You get very little documentation about the functions (if any) and you don’t even get the parameter names, which considerably limits the usefulness of this resource.</p><p>This is helpful, if you want to know if an intrinsic is supported by gcc, don’t forget to check the GCC version in the URL so that it matches your GCC version (I did find some mismatches, for example gcc 7.3.0 support vec_mul, but you won’t find it in the list).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/551/1*4GAOw6F9Di-59i1si54-7w.png" /><figcaption><a href="https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/PowerPC-AltiVec_002fVSX-Built-in-Functions.html">GCC PowerPC AltiVec Built-in Functions</a></figcaption></figure><h3><a href="http://mirror.informatimago.com/next/developer.apple.com/hardware/ve/index.html">Apple Velocity Engine Technical Resources</a></h3><p>This appears to be an archive of a series of guides on Apple’s Velocity Engine, dating back to 2004 and targeting PowerMac G4 and G5.</p><p>Guides like <a href="http://mirror.informatimago.com/next/developer.apple.com/hardware/ve/algorithms.html">Algorithms and Special Topics</a> and <a href="http://mirror.informatimago.com/next/developer.apple.com/hardware/ve/code_optimization.html">Code Optimization</a> give tips on how to optimize for Altivec and SIMD in general. I found these guides very informative, well written and the contain a lot of code examples. However, all this interesting content was not really useful to write my patch.</p><h3>Conclusion</h3><p>The ultimate resource would be : detailed descriptions and illustrations, up to date, easily searchable, list compiler compatibility and even indicate latency and cycles. To be fair, you won’t find that document for Intel or for Arm intrinsics.</p><p>Writing SIMD patches for any platforms involves having multiple tabs opened and cycling between them to find the information you need. The thing with VSX is that you have a few more tabs opened.</p><p>On a side note, finding good (post 2000) video on PowerPC SIMD on YouTube is extremely hard, but stumbled upon the 2004 WWDC presentation of the Velocity Engine. This is probably the best PowerPC SIMD video on Youtube, so definitely worth the watch. Enjoy.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FGx0zY91pJoM%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DGx0zY91pJoM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FGx0zY91pJoM%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/f5d9df07464c1267fdfb4e8b1a915c88/href">https://medium.com/media/f5d9df07464c1267fdfb4e8b1a915c88/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b18e64671cd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fast Averaging of High Color (16 bit) Pixels]]></title>
            <link>https://medium.com/@luc.trudeau/fast-averaging-of-high-color-16-bit-pixels-cb4ac7fd1488?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb4ac7fd1488</guid>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[c]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[rgb565]]></category>
            <category><![CDATA[high-color]]></category>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Tue, 13 Mar 2018 03:58:47 GMT</pubDate>
            <atom:updated>2018-03-20T01:21:23.198Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<p>How did computers in the 90s manage to quickly process color channels that aren’t powers of two? To find out, I set out to write code to quickly average High Color pixels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*9pcGWvsezI5yV7-Mol_jcw.jpeg" /><figcaption>(variously spelled <strong>Highcolor</strong>, <strong>Hicolor</strong>, <strong>Hi-color</strong>, or <strong>Thousands of colors</strong> on a <a href="https://en.wikipedia.org/wiki/Apple_Macintosh">Macintosh</a>)</figcaption></figure><p>Following <a href="https://medium.com/@luc.trudeau/arm-neon-intrinsics-add-functions-explained-with-c-90346d912796">my previous post</a> about <em>Arm Neon Intrinsics Add Functions</em>, Ryan from Intel responded with a little challenge:</p><blockquote>Write code to calculate the average of two pixels in RGB format with 16-bit, in which R, G, B takes 5, 6, 5 bits. You have to calculate them together without extracting each channel and get average separately.</blockquote><h3>Challenge accepted!</h3><p>I was immediately intrigued and after a bit of research, found out that this format is referred to as <a href="https://en.wikipedia.org/wiki/High_color">Hi-Color</a> graphics, which was common in Super VGA video cards in the 90s. Quick side note, adding 1 bit to green makes a lot of sense, since it covers the widest range in the <a href="http://hyperphysics.phy-astr.gsu.edu/hbase/vision/colcon.html">human perception of color</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/477/0*38FpZIu9OOM4G7tY.gif" /><figcaption>Ref: <a href="http://hyperphysics.phy-astr.gsu.edu/hbase/vision/colcon.html">http://hyperphysics.phy-astr.gsu.edu/hbase/vision/colcon.html</a></figcaption></figure><p>The reason you don’t want to extract each color is that important performance gains can be achieved by operating directly on the 16-bit pixels. A<strong> S</strong>ingle 32-bit <strong>I</strong>nstruction can operate over <strong>M</strong>ultiple <strong>D</strong>ata values (in this case 2), this is referred to has <a href="https://en.wikipedia.org/wiki/SIMD">SIMD</a>. Now there’s a catch, with great speed up comes great complexity, as the <a href="https://books.google.ca/books/about/The_PC_Graphics_Handbook.html?id=gZ5SDTpMR-MC">PC Graphics handbook</a> describes:</p><blockquote>the fact that the individual colors are not located at a byte boundary introduces some programming complications. To handle this uneven mapping code must perform bitwise operations to set the corresponding fields to the required values<br> — <a href="https://books.google.ca/books/about/The_PC_Graphics_Handbook.html?id=gZ5SDTpMR-MC">PC Graphics handbook</a></blockquote><h3>Now that’s my kind of a puzzle!</h3><p>First things first, the fastest way to compute an average between <strong>a</strong> and <strong>b</strong> is:</p><pre>avg = (a + b) &gt;&gt; 1</pre><p>For those not familiar, shifting to the right by one (&gt;&gt;1) is equivalent to a division by 2 floored. Note: We need to add 1 to <strong>a</strong> and <strong>b</strong> to offset the sum so that we end up with the same result as a rounded division by 2, but let’s not worry about that for now.</p><h3>A bit of theory about the addition</h3><p>Let’s get the obvious out of the way, computing the average on the 16-bit value directly does not work.</p><p>To understand why, we need talk about how computers add (in theory at least). The following animation is a <a href="https://en.wikipedia.org/wiki/Adder_(electronics)">half-adder,</a> it illustrates that an addition is composed of two parts: the <strong>S</strong>um which is computed using an e<strong>X</strong>clusive <strong>OR</strong>, and the <strong>C</strong>arry which is the result of an <strong>AND </strong>operation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*OTLm4FiK84gKyqBZyoVyaw.gif" /><figcaption>By Marble machine — Own work, CC BY-SA 4.0, <a href="https://commons.wikimedia.org/w/index.php?curid=48017305">https://commons.wikimedia.org/w/index.php?curid=48017305</a></figcaption></figure><p>Notice that adding two n-bit(s) values requires n + 1 bits. Since we want to add 3 distinct colors, we need 3 extra bits, one after each color.</p><pre>   <strong>0</strong> R R R R R <strong>0</strong> G G G G G G <strong>0</strong> B B B B B<br>+  <strong>0</strong> R R R R R <strong>0</strong> G G G G G G <strong>0</strong> B B B B B ----------------------------------------<br>   <strong>0</strong> R R R R R <strong>0</strong> G G G G G G <strong>0</strong> B B B B B </pre><p>Failing to do so will cause the carries to collide into the next color. Simply put, to perform the addition required for the average we need 19 bits and we only have 16.</p><h3>The average fits in the same number of bits, so there’s got to be a way!</h3><p>Let’s go back to our half-adder and try to express it in C:</p><pre>(a xor b) + ((a and b) &lt;&lt; 1)</pre><p>The more picky amongst you might be shocked that I use a + operator inside my definition of the addition, but I don’t mind. Keep your eyes on that left shift, I’m going to make it disappear.</p><p>Let’s replace the addition in the average operation with our C half-adder:</p><pre>avg = (a + b) &gt;&gt; 1</pre><pre>avg = ((a xor b) + ((a and b) &lt;&lt; 1)) &gt;&gt; 1</pre><p>Next, we distribute the right shift:</p><pre>avg = ((a xor b) &gt;&gt; 1) + (a and b)</pre><p>and the left shift has disapeared! Great, we don’t need the extra bit anymore, but we still need to take care of the possible collision between colors.</p><h3>We’re almost there, you’ve read this far in so why stop now?</h3><p>Let’s focus on the <strong>((a</strong> xor <strong>b) &gt;&gt; 1)</strong>, notice that when we perform a right shift, we throw out the rightmost bit. So let’s zero it out:</p><pre>     R R R R R  G G G G G G  B B B B B<br>and  1 1 1 1 <strong>0</strong>  1 1 1 1 1 <strong>0</strong>  1 1 1 1 <strong>0</strong><br>---------------------------------------<br>     R R R R <strong>0</strong>  G G G G G <strong>0</strong>  B B B B <strong>0</strong></pre><p>A simple way to express the value of this mask is via hexadecimal. All we need to do, is compute the value of every four bits:</p><pre>          Mask: 1 1 1 1 <strong>0</strong> 1 1 1 1 1 <strong>0</strong> 1 1 1 1 <strong>0</strong>                      <br>               +-------+-------+-------+-------+<br>     Hex Value:    F       7       D       E</pre><p>If there’s a carry bit between colors it will go in the first zero, so it won’t collide with the next color and it will get right shifted back into its original color:</p><pre>         R R R R <strong>0</strong>  G G G G G <strong>0</strong>  B B B B <strong>0</strong><br>(&gt;&gt; 1)   <strong>0</strong> R R R R  <strong>0</strong> G G G G G  <strong>0</strong> B B B B</pre><h3>Let’s put it all together</h3><p>Here you have it, code that calculates the average of two Hi-Color pixels without extracting each channel.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3eb1b2c72f83351cb1d01c283c78821/href">https://medium.com/media/d3eb1b2c72f83351cb1d01c283c78821/href</a></iframe><p>This is the most commonly used version, but if you are like me, truncating instead of rounding just does not feel right. So Let’s go the extra mile and add in the rounding.</p><h3>Extra credit: Now, about that +1</h3><p>As previously mentionned, the correct code to compute the average is in fact:</p><pre>avg = (a + b + 1) &gt;&gt; 1</pre><p>We need to add one to every color in the pixel</p><pre>+ 0 0 0 0 <strong>1</strong> 0 0 0 0 0 <strong>1</strong> 0 0 0 0 <strong>1</strong></pre><p>Now here’s the interesting part, the +1 will also get shifted out by the right shift. Hang on! It’s not that easy, we can’t just ignore it, because the carry of the +1 could have an impact on the second bit.</p><p>If the first bit of <strong>a</strong> xor <strong>b</strong> is 1, then the +1 would carry into the seconde bit. As it turns out there’s a mask for that:</p><pre>     Mask: 0 0 0 0 <strong>1</strong> 0 0 0 0 0 <strong>1</strong> 0 0 0 0 <strong>1</strong>                      <br>          +-------+-------+-------+-------+<br>Hex Value:    0       8       2       1</pre><p>This version will set you back an extra add and an extra and per pixel, but you get the proper rounding.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2703b00e41ab9a6a682d8a64ce456f06/href">https://medium.com/media/2703b00e41ab9a6a682d8a64ce456f06/href</a></iframe><p>In conclusion, I streamlined this post as much as I could (hopefully not too much) and its still roughly 1000 words to explain two lines of code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb4ac7fd1488" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Arm Neon Intrinsics Add Functions (Explained With C)]]></title>
            <link>https://medium.com/@luc.trudeau/arm-neon-intrinsics-add-functions-explained-with-c-90346d912796?source=rss-3ab9a74ea2fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/90346d912796</guid>
            <category><![CDATA[simd]]></category>
            <category><![CDATA[c]]></category>
            <category><![CDATA[neon]]></category>
            <category><![CDATA[arm]]></category>
            <category><![CDATA[intrinsic]]></category>
            <dc:creator><![CDATA[Luc Trudeau]]></dc:creator>
            <pubDate>Wed, 28 Feb 2018 04:13:56 GMT</pubDate>
            <atom:updated>2018-02-28T14:36:59.709Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h0n_6pR3vvnASNjifOgu_Q.jpeg" /></figure><p>Did you know, <a href="https://developer.arm.com/technologies/neon/intrinsics">Arm Neon Intrinsics</a> have more than 10 different types of vector addition functions? The differences between: <em>Vector Add</em>, <em>Vector Long Add</em>, <em>Vector Wide Add</em>, <em>Vector Rounding Halving Add</em>, <em>Vector Saturating Add</em>, <em>Pairwise Add</em> and <em>Add Across Vector</em> might not appear obvious now, but by the end of this short post they will.</p><p>You might be thinking: “<em>Why don’t I just go read the official Arm documentation instead of this post?</em>”. Because I won’t use pretty graphics and fancy pseudo-code. Yeah that’s right, what we are going to do is look at some C code I wrote (and tested) that mimicks what the intrinsics functions do.</p><h3>Vector Add (vadd)</h3><p>The <em>Vector Add</em> function is exactly what you expect, here’s the 16x4 version:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/186bd2a77a1d8c2e4eed7b5073ef372f/href">https://medium.com/media/186bd2a77a1d8c2e4eed7b5073ef372f/href</a></iframe><h3>Vector Long Add (vaddl)</h3><p>The 16-bit version of <em>Vector Long Add</em> promotes the result of the addition to the next power of 2. In this case 32 bits.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/52ebef81f8d5b3df2663ea25183873cf/href">https://medium.com/media/52ebef81f8d5b3df2663ea25183873cf/href</a></iframe><h3>Vector Wide Add (vaddw)</h3><p>Notice that for this function, the size of parameter <strong>a</strong> is the next power of 2 when compared to parameter <strong>b</strong>. For this addition, parameter <strong>b</strong> gets promoted to 32 bits.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/98bfe7f88e4b556d09e5a3b05c9e1fab/href">https://medium.com/media/98bfe7f88e4b556d09e5a3b05c9e1fab/href</a></iframe><h3>Vector Rounding Halving Add (vrhadd)</h3><p>Although additions are used in the process, what you actually get is an element-wise rounded average.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/88a7c7415e9cbdf9260e13a7c3484cdb/href">https://medium.com/media/88a7c7415e9cbdf9260e13a7c3484cdb/href</a></iframe><h3>Vector Saturating Add (vqadd)</h3><p>Whether you call it clipping or saturating, this add function will limit the value to the maximum value that the output type allows.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a18a3412dec3a6baf142c2178310afbb/href">https://medium.com/media/a18a3412dec3a6baf142c2178310afbb/href</a></iframe><h3>Pairwise Add (vpadd)</h3><p>A beast of an add function. Looking at the C code carefully, we see that it concatenates the additions of subsequent pairs of values in <strong>a</strong> and <strong>b</strong>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1fe6b52dc94099e7acfc63e09c0e7915/href">https://medium.com/media/1fe6b52dc94099e7acfc63e09c0e7915/href</a></iframe><h3>Add Across Vector (vaddv)</h3><p>Returns the sum of vector a:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/72d287656513bed4b0d388bce1d54aa7/href">https://medium.com/media/72d287656513bed4b0d388bce1d54aa7/href</a></iframe><h3>Conclusion</h3><p>Turns out, I totally wrote this for myself. Hopefully it can also help you make optimal choices when picking Neon add functions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=90346d912796" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>