<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Peter Lisák on Medium]]></title>
        <description><![CDATA[Stories by Peter Lisák on Medium]]></description>
        <link>https://medium.com/@peterlisak?source=rss-dfacd6bb4753------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*k4FMhGvUJabuEQa0.</url>
            <title>Stories by Peter Lisák on Medium</title>
            <link>https://medium.com/@peterlisak?source=rss-dfacd6bb4753------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 04:29:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@peterlisak/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How Showmax got into live streaming]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/how-showmax-got-into-live-streaming-537011143a79?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/537011143a79</guid>
            <category><![CDATA[redundancy]]></category>
            <category><![CDATA[live-streaming]]></category>
            <category><![CDATA[media-engineering]]></category>
            <category><![CDATA[unified-streaming]]></category>
            <category><![CDATA[ffmpeg]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:27:42 GMT</pubDate>
            <atom:updated>2024-01-19T10:27:42.771Z</atom:updated>
            <content:encoded><![CDATA[<h3>PART II — SOMETIMES THE ONLY WAY IS THE WAY OF THE FORCE</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QyYkVK4kOuYRk-km" /></figure><p>In the <a href="https://showmax.engineering/2018/02/how-showmax-got-into-live-streaming/">first part of this series</a>, we explained how Showmax got into the live streaming business and the problems we faced while developing our redundant setup. Now, the fun stuff — here’s a little dive into why our live encoding checker was sometimes forced to kill <a href="https://www.ffmpeg.org/">FFmpeg</a> with a SIGKILL signal.</p><h3>Use the force</h3><p>As we mentioned, our encoders have a component that checks the state of a live stream, identifying the transcoding process as broken in these cases:</p><ul><li>Network issues (high latency, network connection interruption, etc.)</li><li>Input stream corrupted (e.g. <a href="https://showmax.engineering/2018/02/how-showmax-got-into-live-streaming/#issue-3-dry-runs-vs-production">remote source overheats</a>)</li><li>Encoder resource issues (encoder exhausts its RAM, CPU load gets too high, etc.)</li></ul><p>All of these issues have one thing in common: encoding speed falling below 1.0x. This is why we use the speed of live encoding as the main decision metric. Interestingly, to fix a broken live stream, the live checker relies on a simple rule of thumb: try turning it off and on again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*XbR9cki-GmghIQdqDD_VtA.jpeg" /></figure><p>This mechanism looked promising! We had tried to simulate a failure scenario and were expecting that the players would recover. Unfortunately, this wasn’t always the case. Sometimes the players stopped and never restarted — mainly because the USP received an <a href="http://docs.unified-streaming.com/documentation/live/encoders/configuring-encoders.html#end-of-stream-eos-signal-should-be-sent">End-Of-Stream (EOS)</a> sequence from FFmpeg when it stopped.</p><p>The EOS sequence consists of the following 8-bytes, called an empty MFRA (Movie Fragment Random Access) Box, signaling that there is no further content:</p><pre>00 00 00 08 6d 66 72 61</pre><p>In our self-healing scenario, we did not want the stream to be marked as “finished.” We only wanted to restart the encoder and keep the players waiting until it comes back up. Even though this would create a discontinuity in the stream, the stream would eventually resume.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*-fHJO3uVibhiee1-C2G8aQ.jpeg" /></figure><p>As it turns out, this required a somewhat heavy-handed approach. Instead of gently quitting the encoder in the event that the live checker deems the stream as broken, it would send a <a href="https://en.wikipedia.org/wiki/Signal_(IPC)#SIGKILL">SIGKILL signal</a> to prevent the process from sending the EOS sequence. Although the USP was not getting any data, it did not think the live stream was finished, and everything worked as expected. Sometimes, the easiest solution is the best solution.</p><h3>Why so hacky?</h3><p>We are pretty sure that you think that’s quite hacky. First, you’re right. But…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*JwoPLLTfHX_K7NkBGBHoYQ.jpeg" /></figure><p>…When we held a meeting about writing this post, we went back through the <a href="https://ffmpeg.org/ffmpeg.html">FFmpeg documentation</a> to gather some source materials. Then, we searched the source code on <a href="https://github.com/FFmpeg/FFmpeg">Github</a> and we found something familiar. <a href="https://github.com/FFmpeg/FFmpeg/blob/n3.4.2/libavformat/movenc.c#L79">An undocumented flag</a> called skip_trailer: Skip writing the mfra/tfra/mfro trailer for fragmented files. Just a quick proof-of-concept gave us a glimmer of hope.</p><pre>$ ffmpeg -v 0 -i Sintel.2010.1080p.mkv -movflags isml+frag_keyframe -f ismv - | tail -c 8 | hexdump<br>00 00 00 08 6d 66 72 61<br>$ ffmpeg -v 0 -i Sintel.2010.1080p.mkv -movflags isml+frag_keyframe+skip_trailer -f ismv - | tail -c 8 | hexdump<br>00 00 00 00 00 00 de a2</pre><p>We thought we had found a way to replace SIGKILL. We prepared changes for the encoder, adding +skip_trailer to the FFmpeg command line so that it did not send an EOS signal anymore. We also appended a new part of the code to the live-encoding pipeline to post an empty MFRA box instead of FFmpeg at the point when we knew it’s really done.</p><p>The first tests showed us that some of the streams were not stopped even if the USP returned the HTTP response 200 OK on ingesting EOS bytes. We tried to check the states of streams via /state and /statistics at the <a href="http://docs.unified-streaming.com/documentation/live/publishing-points.html#retrieving-additional-publishing-point-information">USP’s endpoints</a> and retried posting EOS if needed. But some of the streams were still magically turned back to started state from stopped. At the moment we are still not able to say why, but we’re eager to change that.</p><p>Some unconfirmed theories are that it may have something to do with <a href="http://docs.unified-streaming.com/documentation/live/advanced-live-options.html#dual-ingest-setup-failover">USP’s dual ingest</a> (EOS is sent from the new connection), or the expiration time from last-ingested stream data (found in the SQLite database where the USP stores stream metadata). Sending EOS bytes after some delay seems to be working as desired.</p><p>We know we didn’t find everything we needed to know in the documentation. We need to spend more time with source code browsing and reverse engineering to understand every nuance in this complicated process.</p><h3>Next up</h3><p>We have applied the same try-and-see approach for our next technical challenge — trimming the live stream. You can read about it in our <a href="https://showmax.engineering/2018/03/trim-the-fat-not-the-meat/">next piece in this series</a>.</p><p><em>Originally published at </em><a href="https://showmax.engineering/articles/how-showmax-got-into-live-streaming1"><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=537011143a79" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/how-showmax-got-into-live-streaming-537011143a79">How Showmax got into live streaming</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Showmax got into live streaming]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/how-showmax-got-into-live-streaming-723d8fdf3cee?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/723d8fdf3cee</guid>
            <category><![CDATA[ffmpeg]]></category>
            <category><![CDATA[live-streaming]]></category>
            <category><![CDATA[http-live-streaming]]></category>
            <category><![CDATA[media-engineering]]></category>
            <category><![CDATA[unified-streaming]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:27:11 GMT</pubDate>
            <atom:updated>2024-01-19T10:27:11.794Z</atom:updated>
            <content:encoded><![CDATA[<h3>PART I — ADVENTURES IN ACHIEVING STREAMING REDUNDANCY</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hHuvDcPXxtEa5KAN" /></figure><p>This is the first in a series of articles about how Showmax made a radical departure from its video-on-demand roots and took a live streaming platform from scratch to production in less than three months. In particular, we’ll share some of the key issues we struggled with. In this first article, we describe how we had to downgrade our initial plans for full redundancy because things didn’t work out as expected.</p><p>Let’s start with some brief background. Showmax started life in mid-2015 as a subscription video on demand (SVOD) service. The reason for the SVOD-only approach was to capitalise on the major shift in viewing habits to binge-watching multiple episodes (and even multiple seasons) in a single sitting. Live streaming was never envisaged as part of the offering.</p><p>Fast-forward to 2017 and our business in Poland achieved much consumer love by offering original Polish content. In fact, this local content was more popular than heavyweight Hollywood shows and a major differentiator for the service. Based on this, the Poland team had the idea to bring a localised version of the world-famous <a href="https://en.wikipedia.org/wiki/Saturday_Night_Live">Saturday Night Live</a> to Poland, and after negotiating with owners NBCUniversal <a href="https://blog.showmax.com/showmax-debuts-live-streaming-saturday-night-live-polska/">SNL Polska</a> was born.</p><p>Well, with one slight wrinkle — as the <em>live</em> part of the SNL name suggests, we suddenly had to shift gears from SVOD-only to having live streaming capability. This was a major technical challenge, and one where failure would be played out in real time to the general public. The task ahead was both exhilarating and terrifying.</p><h3>Initial idea</h3><p>The initial idea of a fully redundant live streaming setup that we drew on a whiteboard looked promising. It would bolt-on to the existing SVOD architecture that uses FFmpeg for transcoding video to multiple bitrates and <a href="http://www.unified-streaming.com/">Unified Streaming Platform (USP)</a> for streaming the content across any platform and device (including protection, e.g. DRM).</p><p>We had two single bitrate stream inputs provided by <a href="http://atmsystem.pl/en/">ATM SYSTEM</a>, with the master in Warsaw and the slave in Wroclaw. Each of the streams would be transcoded by two encoders into multiple bitrate streams and ingested to USP origins as per the diagram. This was exactly as recommended by <a href="http://docs.unified-streaming.com/documentation/live/advanced-live-options.html#redundancy">USP’s documentation</a> on redundancy and dual ingest (failover).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*9j1AXziUsXSrDVloODW7uQ.png" /></figure><p>This setup would produce two fully redundant output streams, <a href="https://developer.apple.com/library/content/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/UsingHTTPLiveStreaming/UsingHTTPLiveStreaming.html#//apple_ref/doc/uid/TP40008332-CH102-SW22">primary and backup stream</a>.</p><pre>#EXTM3U <br>#EXT-X-STREAM-INF:BANDWIDTH=592000,AVERAGE-BANDWIDTH=538000,CODECS=&quot;mp4a.40.2,avc1.4D401F&quot;,RESOLUTION=640x360,AUDIO=&quot;audio-aacl-98&quot;,CLOSED-CAPTIONS=NONE <br>https://usp-live01.showmax.com/l/4545a5ba-8c7c-457d-a136-2010140f0442/4545a5ba-8c7c-457d-a136-2010140f0442-audio=98304-video=409000.m3u8 <br>#EXT-X-STREAM-INF:BANDWIDTH=592000,AVERAGE-BANDWIDTH=538000,CODECS=&quot;mp4a.40.2,avc1.4D401F&quot;,RESOLUTION=640x360,AUDIO=&quot;audio-aacl-98&quot;,CLOSED-CAPTIONS=NONE <br>https://usp-live02.showmax.com/l/f3f39fe5-a51e-4719-81e0-2bd05414b2fa/f3f39fe5-a51e-4719-81e0-2bd05414b2fa-audio=98304-video=409000.m3u8 <br>#EXT-X-STREAM-INF:BANDWIDTH=3735000,AVERAGE-BANDWIDTH=3396000,CODECS=&quot;mp4a.40.2,avc1.4D401F&quot;,RESOLUTION=1280x720,AUDIO=&quot;audio-aacl-131&quot;,CLOSED-CAPTIONS=NONE <br>https://usp-live01.showmax.com/l/4545a5ba-8c7c-457d-a136-2010140f0442/4545a5ba-8c7c-457d-a136-2010140f0442-audio=131072-video=3072000.m3u8 <br>#EXT-X-STREAM-INF:BANDWIDTH=3735000,AVERAGE-BANDWIDTH=3396000,CODECS=&quot;mp4a.40.2,avc1.4D401F&quot;,RESOLUTION=1280x720,AUDIO=&quot;audio-aacl-131&quot;,CLOSED-CAPTIONS=NONE <br>https://usp-live02.showmax.com/l/f3f39fe5-a51e-4719-81e0-2bd05414b2fa/f3f39fe5-a51e-4719-81e0-2bd05414b2fa-audio=131072-video=3072000.m3u8</pre><h3>Issue #1: Redundancy and dual ingest setup</h3><p>The proposed setup did not work as expected. When the camera was aimed at our small improvised stage featuring a desk clock, the clock produced a broken live stream. The second hand jumped backwards and forwards, and playlists contained discontinuities after every chunk. Even though the <a href="http://docs.unified-streaming.com/documentation/live/advanced-live-options.html#synchronizing-the-archive-from-the-first-origin-with-the-second">documentation</a> describes this situation thoroughly, it took us a while to actually understand its implications. It turned out that ingested streams must be time-aligned very precisely because USP origin keeps the chunk that arrives first while the second one to arrive is discarded. In our case, it was ingesting chunks alternately from both encoders, which was the root cause of the trouble. Unfortunately, we were not able to synchronize the FFmpeg encoders correctly. Here is where our fully redundant setup started to fall apart. Each USP now only had a single encoder feeding it data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*aqU2CYCxue-kEMPH4VhZDQ.jpeg" /></figure><h3>Issue #2: HLS failover</h3><p>We still had a chance for failover on the client side video player by using <a href="https://developer.apple.com/library/content/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/UsingHTTPLiveStreaming/UsingHTTPLiveStreaming.html#//apple_ref/doc/uid/TP40008332-CH102-SW22">HLS Redundant Streams</a>. We’ve configured our infrastructure to take the playlists generated by the USP and inject the URLs of streams served by the other USPs as redundant streams. In essence we were creating meshed playlists.</p><p>Despite the big expectations, still no luck — the video player on the client side crashed when the primary stream was manually killed. In practice, neither of the players we tested ( <a href="https://github.com/video-dev/hls.js/">HLS.js</a>, <a href="https://github.com/google/ExoPlayer">ExoPlayer</a> and <a href="https://developer.apple.com/documentation/avfoundation/avplayer">AVPlayer</a>) could play without interruption. What we learned is that HLS failover is meant to fall back between CDNs serving completely identical streams. The only consolation was that we found we’re not alone in <a href="https://github.com/videojs/videojs-contrib-hls/issues/753#issuecomment-231264858">misunderstanding alternate streams</a> in HLS playlists.</p><h3>Issue #3: Dry-runs vs. production</h3><p>In the buildup to the big day of the SNL Polska debut we performed countless dry-runs. Our list of testing scenarios covered the whole pipeline from ingesting on encoders to playback on the client side. We tried to simulate production broadcasting as much as possible to be prepared properly for D-Day.</p><p>Fast-forward in time, with only a few minutes to go we got a call from ATM telling us to have only one encoder consume the stream from each source. Apparently, their <a href="https://teradek.com/products/cube_155">Teradek Cube 155</a> was overheating — two encoders connected were simply too much.</p><p>During dry-runs, everything always worked like a charm. But there was one key difference — we were using a much newer <a href="https://teradek.com/products/cube-655">Teradek Cube 655</a> in tests which had no issues with multiple simultaneous connections. Which leads to…</p><p>Lesson learned: Always test exactly the same setup you’re going to use in production.</p><h3>Final setup</h3><p>After shaving off all non-working parts we ended with two redundant pipelines of Teradek, FFmpeg and USP origin (as outlined in the diagram below). Although USP and HLS provide some kind of redundancy/failover approaches, in practice we were forced to deliver our custom solution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*77Ok0LNs6SWQhtB8tqnwPA.png" /></figure><p>We implemented the custom failover on our Playback API where we added an asynchronous health check of all the redundant live streams. For the lack of better name, we called it the <em>live watcher</em>. The stream that is not updated regularly is considered as dead. For that purpose, USP provides /state and /statistics publishing endpoints to <a href="http://docs.unified-streaming.com/documentation/live/publishing-points.html#retrieving-additional-publishing-point-information">retrieve additional information</a> about live streams.</p><pre>$ curl -v https://usp-live01.showmax.com/l/4545a5ba-8c7c-457d-a136-2010140f0442/4545a5ba-8c7c-457d-a136-2010140f0442.isml/state<br>&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&gt;<br>&lt;!-- Created with Unified Streaming Platform(version=1.7.10) --&gt;<br>&lt;smil<br>  xmlns=&quot;http://www.w3.org/2001/SMIL20/Language&quot;&gt;<br>  &lt;head&gt;<br>    &lt;meta<br>      name=&quot;updated&quot;<br>      content=&quot;2015-05-06T01:57:12.313929Z&quot;&gt;<br>    &lt;/meta&gt;<br>    &lt;meta<br>      name=&quot;state&quot;<br>      content=&quot;started&quot;&gt;<br>    &lt;/meta&gt;<br>  &lt;/head&gt;<br>&lt;/smil&gt;</pre><p>To increase the precision and reaction speed of Playback’s live watcher we added another type of live streaming health check to the encoders. Hence <a href="https://ffmpeg.org/ffmpeg.html">FFmpeg’s parameter</a> -progress output.txt was added to transcoding command-line, which supplies program-friendly progress information.</p><pre>frame=1887368<br>fps=25.0<br>stream_0_1_q=32.0<br>stream_1_1_q=30.0<br>stream_2_1_q=28.0<br>stream_3_1_q=35.0<br>bitrate=913.2kbits/s<br>total_size=8618218800<br>out_time_ms=75496746667<br>out_time=20:58:16.746667<br>dup_frames=0<br>drop_frames=0<br>speed=1.0x<br>progress=continue</pre><p>The decision to stop/restart encoding is made by observing the FFmpeg progress output and parsing out the encoding speed, which must stay above 1.0 (otherwise the stream might start to stutter). For debugging purposes we also logged round trip times between our encoders and the Teradeks.</p><p>The incoming player always gets a healthy live stream playlist from the Playback API. However, If the playlist dies whilst playing, the player implements a retry logic that asks the customer to reload the video. This means that a working playlist is returned again and the video playback is resumed.</p><p>The dead stream is still being monitored and if it springs back to life, it will be put back into rotation. This in essence forms a simple self-healing logic.</p><h3>Next Up</h3><p>Next we will talk about <a href="https://showmax.engineering/2018/03/sometimes-the-only-way-is-the-way-of-the-force/">why we sometimes need to kill FFmpeg with SIGKILL</a>.</p><p><em>Originally published at </em><a href="https://showmax.engineering/articles/how-showmax-got-into-live-streaming"><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=723d8fdf3cee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/how-showmax-got-into-live-streaming-723d8fdf3cee">How Showmax got into live streaming</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pushing video bitrate to the limit]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/pushing-video-bitrate-to-the-limit-9b0d7ade6269?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/9b0d7ade6269</guid>
            <category><![CDATA[bandwith-saving]]></category>
            <category><![CDATA[media-engineering]]></category>
            <category><![CDATA[ffmpeg]]></category>
            <category><![CDATA[encoding]]></category>
            <category><![CDATA[video-streaming-service]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:19:16 GMT</pubDate>
            <atom:updated>2024-01-19T10:19:16.739Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*r53_xFcx5LyIDjNX" /></figure><p>When you subscribe to a VOD service, you expect the best visual quality possible. For our customers across Sub-Saharan Africa, it’s no different. But, in a region with developing infrastructure and expensive data, OTT services like ours face an inevitable challenge: Providing a service that is both high-quality and affordable.</p><h3>Africa — continent of challenges</h3><p>Showmax is part of the Multichoice Group, and with our sister company DStv, we are the leading OTT service on the African continent. It’s a unique market relative to North America and Europe, and you really have to look at things differently. Ideas and approaches that wouldn’t work in other markets may be just the thing for certain parts of Africa.</p><p>The African VOD/OTT market is developing quickly, and infrastructure is being improved significantly, but there are still a lot of places — with millions of potential viewers — where data is a luxury. In Africa, no matter where you are, mobile data (all data, really) is expensive. As such, we are constantly looking for ways to deliver a premium streaming experience while consuming as little data as possible. In fact, Showmax was among the first VOD services <a href="https://www.fin24.com/Tech/News/showmax-offers-offline-viewing-on-android-20151214">to introduce a “download” feature</a>.</p><p>The reason is simple — mobile is a huge part of our business:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*JAf5nvCAIp-nTOH4urfBWg.jpeg" /></figure><p>It’s important to remember that customers pay for our service twice — for their Showmax subscription and for data. That means that every transferred bit counts.</p><p>Here’s a rundown of how we are working to squeeze as much as we can into every last bit.</p><h3>Next-gen codecs</h3><p>For years now, AVC/H.264 by MPEG-LA has been the primary codec for video streaming — it’s widely used and supported. But it has efficiency limits with 4K/8K resolutions, so two next-gen codecs were introduced to meet these requirements:</p><ul><li>Direct successor — HEVC/H.265: Currently supported by Apple devices, the Safari browser, and some smart TVs (since it became a part of the DVB-T2 standard).</li><li>Direct competitor — VP9 by Google: currently supported by Android devices, web browsers, and some smart TVs.</li></ul><p>We ran some tests and experiments, and decided to lower the bitrate ladder by 33% for both codecs — HEVC/H.265 and VP9 — and pushed them to <a href="https://mybroadband.co.za/news/broadcasting/298240-netflix-vs-showmax-new-data-saving-codec-tested.html">production</a> for new content.</p><p>It wasn’t so easy.</p><p>New codecs can save bandwidth, but they require more power to encode. In our case, it’s ~4x for HEVC and ~8x for VP9, respectively. So, we came up with a parallel encoding approach that we lay out in detail in a two-part blog post <a href="https://tech.showmax.com/2019/01/divide-encode-1/">here</a> and <a href="https://tech.showmax.com/2019/03/divide-encode-2/">here</a>. It helps us keep encoding time within a reasonable range for next-gen codecs (below is a time chart of one asset from our catalogue).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*15Avb_6Qn57e_dDkcQ4OPg.jpeg" /></figure><p>Where next-gen codecs really show their full power is in bandwidth-saving for higher resolutions. HEVC outperforms VP9 here as well. Below, we split a scene from the movie <em>Sintel</em> into thirds to show the difference. 33% in between H.264 and HEVC/VP9 is a reasonable balance for our content set that preserves quality and saves as much bandwidth as possible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*qTdGsFLmsqtyxZWClyZXog.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*2d6QM7CJ4tx95D4TTyK0uw.jpeg" /></figure><p>Then, there’s the decoding aspect. In theory, players should choose the variant from the DASH manifest or HLS playlist based on their capabilities and supported codecs. The reality is that some smart TVs and smartphones crash when they don’t recognize the codec in the manifest/playlist, or think they are able to play that codec but can’t. To get around this, we filter out next-gen codecs from the manifests/playlist before passing it to the player on devices where issues may occur, or occurred previously. Beyond that, next-gen codecs can also be omitted if the device has only a software decoder, because it consumes too much battery and could drain it in a matter of minutes.</p><p>The future of codecs is interesting. As computational power continues to increase for devices of all kinds, there will certainly be another evolutionary generation of the VCC and AV1 codecs. Recently, AV1 was added to Android with version 10, and it’s slowly being pushed to production around the world. VCC is just slightly behind and should be standardized later this year.</p><h3>Per-title encoding</h3><p>Constant bitrate (CBR) encoding with a fixed bitrate ladder is the keystone of all OTT/VOD services. But videos contain fast and slow scenes, different levels of detail, and other attributes that affect complexity and the required amount of bits to encode an individual scene. In most cases, bits are wasted. To avoid waste, we built a per-title optimization system to select the correct bitrate for the best viewing experience, but no more. Nowadays, all of the big cloud encoding services like <a href="https://bitmovin.com/per-title-encoding/">Bitmovin</a>, <a href="https://www.brightcove.com/en/blog/2017/09/context-aware-encoding-building-better-mousetrap">Brightcove</a>, JWPlayer, and <a href="https://mux.com/blog/instant-per-title-encoding/">Mux</a>, have their own approach.</p><p>FFmpeg’s capped Constant rate factor (CRF) is a simple variant of the per-title approach. It lacks the ability to change the number of rungs in the ladder, but it’s a “one pass” method, and advanced methods usually require an extra analysis pass. As Jan Ozer says in <a href="http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/One-Title-at-a-Time-Comparing-Per-Title-Video-Encoding-Options-121493.aspx">One Title at a Time: Comparing Per-Title Video Encoding Options</a>, the results of capped CRF are comparable to advanced technologies.</p><p>Capped CRF mode is supported by every codec we use — H.264, HEVC, and VP9. The simplified command looks like this:<br>ffmpeg -i input_file -crf 23 -maxrate 5000k output_file<br>Where FFmpeg will encode video at quality level 23 and cap the bit rate at 5000kbps. On the image below, you can see from the screen cap of <em>Sintel</em> that the resulting bitrate is far from the cap.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*kyJWQuLnSoX6csS5y685pA.png" /></figure><p>The table below contains a few examples of final bitrates for all three codecs of 1080p. The original bitrates for CBR mode were 5000kbps and 3340kbps, respectively. <em>Simon’s cat</em> is a simple black and white animation, and the bitrate savings are enormous here. Surprisingly, H.264 outperforms both HEVC and VP9.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eiwfNPxFf44EEeezRpRMuQ.png" /></figure><h3>Mobile only</h3><p>Originally, our bitrate ladder started with a resolution of 360p and bitrate of 200Kb/s — which was still not low enough on Edge connectivity. So we introduced two lower steps starting at 144p resolution with a 75Kb/s bitrate focus on <a href="https://mybroadband.co.za/news/broadcasting/328789-showmax-launches-r49-mobile-plan-in-south-africa.html"><strong>mobile devices</strong></a>. Up to this point, audio bandwidth was only a small fraction of overall bitrate, but with such a low video bitrate, we also had to re-think audio codecs. We added HE-AACv2 with 32Kb/s for very low data profiles to pass through the pipes with poor Edge connectivity.</p><p>It may sound silly if you live in an area with Fiber or 5G connectivity, but imagine you live in a rural area with limited connectivity and <a href="https://techpoint.africa/2019/05/01/mobile-data-cost-africa/"><strong>expensive data</strong></a>. The introduction of our Mobile Only subscription gives people on budget the opportunity to consume up to 20 hours of content with a 1GB data plan. It’s important to note that people tend to be more sensitive to lower audio quality than video. On smaller phone screens, the lack of video quality is mitigated while the HE-AACv2 codec keeps audio quality at a pleasant level — even with an extremely low 32kb/s bitrate.</p><p>In general, configuring VP9 is quite complicated and it often does not behave as documented. We had the biggest issue with overshooting the maximum bitrate. In the end, we switched back to CBR for very low resolutions.</p><h3>Switching from VP9 to HEVC on Android</h3><p>In the previous section, I noted that we serve VP9 for only ~2% of Android devices. While we were using the CBR approach, we preferred VP9 on Android as both come from Google and the HEVC and VP9 bitrates were the same. But HEVC with capped CRF provides better results than VP9.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*F_J95BZrdJZVJQ4-rCVYug.jpeg" /></figure><p>Moreover, we found that most android devices that support VP9 support HEVC as well. That allows us to transfer these devices from VP9 to HEVC and save more bandwidth. Before the change, 25% of androids played VP9 and 23% HEVC. After change it was 2% and 45%, respectively.</p><h3>Conclusion</h3><p>After all of these changes, we had to make the important decision to re-encode the whole catalogue — even though viewership is uneven and only a small fraction of assets account for most of the actual content consumed on the app. After only a few weeks of fully utilizing our encoding farm, we can now save bandwidth that will add up to more than 40 thousands hours of content (that’s more than four years).</p><p>At the beginning, we were able to save 33% by using new codecs. After switching from CBR to capped CRF mode, we save almost 25% with mature H.264. For the next-gen codecs, the savings reached 39% and 48%, respectively. Modern codecs and per-title techniques were the main drivers of how we managed to reduce bit rate significantly while preserving video quality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*p1IAL1ugOa5aQMam4l3OIA.jpeg" /></figure><p><em>Originally published at </em><a href="https://showmax.engineering/articles/pushing-video-bitrate-to-the-limit"><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9b0d7ade6269" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/pushing-video-bitrate-to-the-limit-9b0d7ade6269">Pushing video bitrate to the limit</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Showmax Engineering story, Part III.]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/the-showmax-engineering-story-part-iii-4381905d1b8c?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/4381905d1b8c</guid>
            <category><![CDATA[encoding]]></category>
            <category><![CDATA[cdn]]></category>
            <category><![CDATA[drm]]></category>
            <category><![CDATA[media-engineering]]></category>
            <category><![CDATA[network-topology]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:18:17 GMT</pubDate>
            <atom:updated>2024-01-19T10:18:17.800Z</atom:updated>
            <content:encoded><![CDATA[<h3>DELIVERING CONTENT SECURELY, WITH BEST POSSIBLE USER EXPERIENCE</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AxXrlZm9fETLBX0S" /><figcaption>Photo by <a href="https://unsplash.com/@charlesdeluvio?utm_source=medium&amp;utm_medium=referral">charlesdeluvio</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>On the Showmax Engineering blog, we try to explain solutions to a pretty specific problem: how to deliver a high quality video streaming in Africa. To make our stories more intelligible, we started the series The Showmax engineering story, with the goal of explaining the meanings of in-depth media engineering terms that we use so commonly.</p><p>Now, with the very last piece of vocabulary at hand, you have the tools to understand it all completely. If you missed the previous parts, you can read about encoding terms in <a href="https://showmax.engineering/2020/11/media-engineering-vocabulary/"><strong>The Showmax Engineering story, Part I</strong></a>, and streaming terms in <a href="https://showmax.engineering/2021/04/media-engineering-vocabulary-2/"><strong>The Showmax Engineering story, Part II.</strong></a> In this very last edition, we describe the terminology around content protection and quality evaluation.</p><h3>Content protection-related terms</h3><p>Let’s have a look behind the curtain of content protection against piracy.</p><p><strong>Digital rights management (DRM)</strong> is a technology-based control over copyright-protected content, and it’s a fundamental part of the content protection environment. Its rules are defined within the contracts with content owners — in our case, that means contracts with studios. Basically, for every piece of content we have device constraints (what device we can display the content on and in which quality/resolution) and time constraints (how long we’re allowed to host it).</p><h3>DRM types</h3><p>There are three main DRM types used in the market.</p><p><strong>Widevine</strong> by Google is mainly supported by web browsers (Chrome, Firefox, Edge), Android devices, and SmartTV devices. Google provides a cloud solution for the license server, which is free to use, and includes no charges for license generation or device integration. It also provides an SDK to implement your own license server.</p><p><strong>PlayReady</strong> by Microsoft is mainly supported by the Edge web browser, SmartTV devices and gaming consoles (Xbox, PlayStation). Microsoft provides the SDK to implement your own license server, or you can use any third party DRM provider that supports PlayReady.</p><p><strong>FairPlay</strong> by Apple is mainly supported by the Safari web browser and iOS/tvOS devices, and was originally developed to protect music in the iTunes Store. Apple itself doesn’t provide any license server. You can either follow the guide on how to implement your own server, or use a third party DRM provider. Further details and specifications that go beyond the scope of our vocabulary can be found in the <a href="https://ottverse.com/drm-glossary/"><strong>OTT Verse glossary</strong></a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/0*RZBfRaFBt6jFFnLJ" /></figure><h3>DRM encryption algorithms</h3><p>All three variants use a family of AES encryption algorithms, the difference is only in supported mode — full sample encryption in CTR mode (cenc), or pattern encryption in CBC mode (cbcs). This is defined as part of <strong>Common Encryption Scheme (CENC)</strong>, which defines the standards for encryption and key mapping.</p><p>If DRM systems support CENC, content can be decrypted by any of these systems as long as the same key IDs and content keys are used. This allows content providers to serve the same content encrypted once to a broader range of devices using different DRM systems. It’s also called <strong>multiDRM</strong>.</p><p>To close this section, we need to mention two MPEG standards, <strong>MPEG Common Encryption</strong>, which defines encryption and container formats; and <strong>MPEG-A Part 19: Common media application format (CMAF)</strong> which handles conversion between them without re-encryption and re-encoding. Unfortunately, it does not define which AES mode should be used. While Apple decided to support “cbcs”, Widevine and PlayReady started with “cenc” and added “cbcs” mode later. Therefore, implementation of CMAF in practice is slow as you still need to have both variants to support older devices.</p><p>Technically the content is encrypted by a key, which is shared as part of the <strong>license</strong> that defines the conditions on which a device can use the key(s) to decrypt the content. Limited validity, requested security levels of the device, or the minimal supported HDCP are all examples of such conditions (a decryption key is also referred to as a <strong>content key</strong>). To find the right key, the device must provide a <strong>content ID</strong> (ID identifying a piece of content in the register of the DRM provider), and/or <strong>key ID</strong> (ID of the key used for content encryption) to the license server. Those values are usually baked into playlists/manifests, or directly in media segments. The <strong>License server</strong> then issues the actual license with the key to the device. The license is processed in a <strong>trusted execution environment (TEE)</strong> of the device — a secure area of the processor that ensures that the code and loaded data are protected, and provides hardware security for trusted applications.</p><p><strong>Content decryption module (CDM)</strong> can be a software (compiled binary library) stored in a browser or hardware solution. The CDM decrypts the DRM-protected content using the key issued by the DRM license server.<br><strong>Encrypted Media Extensions (EME)</strong> is specified as a communication layer between CDM and web browsers to play HTML5 video files protected by DRM. This obsoletes third party plugins like Adobe Flash or Microsoft Silverlight provided this functionality in the past.</p><h3>DRM layers of protection</h3><p><strong>Security level</strong> defines the minimal level of DRM security requested, and specifies where the decryption can be processed. Levels of security can differ across DRMs, but in general, it can be defined as software DRM (less secure, usually up to SD quality) and hardware DRM (required for HD and higher).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ds0e-UZW3zHUUiibKYS-LQ.png" /></figure><p><strong>High-bandwidth content protection (HDCP)</strong> is a security measure used to transfer content to a display using compatible digital outputs including DisplayPort, DVI or HDMI. HDCP prevents unauthorized devices from reading high-quality content. Both the display and the digital outputs must be certified HW, and must support HDCP, otherwise the content is downgraded to a lower quality. If no HDCP is found on any side, it leads to a failure. HDCP can be restricted based on the <strong>content type</strong> (0 — any version, 1–2.0 and higher), or by <strong>version</strong> directly.</p><p>Fairplay and PlayReady use content type, and Widevine uses versioning.</p><p>Be aware that DRM encryption is not the same as what’s used for the HDCP encryption when sending the signal over the cable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/0*yDpPknsLtjyOSjtx" /></figure><h3>Forensic / digital watermarking</h3><p><strong>Forensic watermarking</strong> (or <strong>digital watermarking</strong>) is the practice of adding invisible markers into video tracks to uniquely identify authorized users. It can be understood as something like a signature. A simplified explanation is that, at the beginning of the packaging process we have two variants of the stream, one signed with A char, and the other with B char. The playlist is then composed as a unique sequence of segments from both streams. This signature can’t be removed by re-encoding the content, so it can be easily used for tracing leaked streams.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/0*P5yfYe6GNeKHh63o" /></figure><h3>Insight into content delivery</h3><p><strong>Content delivery network</strong> is a set of servers distributed across the country or continent to increase the speed of content distribution. It usually consists of many powerful cache servers connected into layered/tree topology.</p><p><strong>Origin</strong> servers contain all of the catalog content, and are therefore a kind of source of truth. They have massive disk capacity and don’t serve the content directly to customers — they are protected behind <strong>edge</strong> servers, or <strong>mid tier</strong> servers that sit between origins and edges. <strong>Mid tier/edge servers</strong> hold the most popular content in local cache with less capacity but more speed (RAM or quick disks like SSDs), in order to serve higher demand from popular content very quickly. The effectiveness of a given CDN architecture can be evaluated by <strong>cache-hit ratio</strong>. The higher the ratio, the more requests are served from the local caches (faster response) with fewer requests passed to lower layers (and, eventually, to origins). Such architectures, just at a larger scale, are also used by big third party CDN providers like Akamai, Limelight and Cloudfront.</p><p>You can find out more about the Showmax content delivery network in the <a href="https://showmax.engineering/tag/cdn/"><strong>CDN series</strong></a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/0*Q0u8fmj6SErubGFf" /></figure><h3>Quality of experience</h3><p>All of the terms mentioned within this series describe factors that influence the quality of our customers’ experience. <strong>Quality of experience (QoE)</strong>, our very own metric, describes how our customers are satisfied with the product we provide them. It’s an attempt to objectively measure subjective opinion.</p><p>Some general QoE metrics worth mentioning:</p><p><strong>Buffering time</strong> is the time when the player waits for data/content while playback is paused. A related value is <strong>re-buffering</strong>, which is calculated as the proportion of buffering time to consumed/played time. It can be influenced by an overloaded CDN, network issues between CDN and the device, and several other things. In general, lower video bitrates are less prone for buffering.</p><p>Another one is the number of <strong>failures</strong> during or before real playback that cause unrecoverable interruption. They can be caused by bugs on the backend or frontend, issues with the network between CDN and the device, damaged encoded video titles, and more.</p><p>Last but not least is <strong>average bitrate</strong>, which tells us the average video quality the user watched — which is often tightly-connected to buffering. Lower average bitrate usually means that viewers watched lower quality videos, which can be a precursor of buffering issues. It can be caused by overloaded CDN or device, network capacity, etc.</p><p><strong>Mean opinion score (MOS)</strong> is an arithmetic combination of individual values/metrics that expresses a user’s QoE in one value. It’s very useful to define the relative importance of the metrics, and it helps us set priorities for improvements — as an improvement in one metric can cause degradation in others and vice versa. For example, we can decrease the buffering time and get a smoother stream by lowering the average bitrate. On the other hand, with the lower bitrate we also decrease the picture quality. So we need to clearly identify the higher priority, whether it is lower buffering or better picture quality.</p><p>In the <a href="https://showmax.engineering/2021/04/media-engineering-vocabulary-2/"><strong>edition</strong></a> about encoding parameters, we omitted to mention how you can evaluate the outputs produced by different encoder configurations. Popular mathematical approaches are <strong>Peak Signal-to-Noise Ratio (PSNR)</strong>, or <strong>Structural Similarity Index Measure (SSIM)</strong>. They evaluate picture quality from their exact perspective, and do not necessarily take into account the subjective perception of humans. Despite this, they are still heavily used as base evaluation methods, where anything under 35 PSNR score can be considered as ugly (e.g., with artifacts), while scores beyond 45 are considered to be unrecognizable from the original source (therefore wasting bits).</p><p>Subjective video quality can be predicted by <strong>Video Multimethod Assessment Fusion (VMAF)</strong>, which takes into account the perception of quality by human eyes. A well-balanced score of quality and reasonable data consumption may impact the aforementioned QoE metrics and lead to their improvement. The trend is to keep bitrate low and quality high — which are in most cases two contradictory characteristics. More about using VMAF to find optimal bitrate ladder you can read in Jan Ozer’s posts <a href="https://streaminglearningcenter.com/encoding/optimal_encoding_ladder_vmaf.html"><strong>1</strong></a> and <a href="https://www.linkedin.com/pulse/finding-just-noticeable-difference-netflix-vmaf-jan-ozer"><strong>2</strong></a>.</p><h3>Keep on reading…and learning</h3><p>After finishing this series introducing the common terms from the media engineering world — the core of our business — it’s easy to see why providing a quality SVOD service requires the joint effort of several teams. It’s challenging, and we love it.</p><p>Continue your discovery path by reading another series of blog posts, this one about delivering the content the right way. Our great CDN series starts here: <a href="https://showmax.engineering/2021/06/cdn-automation-1/"><strong>Selecting the right URL</strong></a>. In case you’ve already read the beginning, you can continue with the <a href="https://showmax.engineering/2021/08/cdn-automation-2/"><strong>second</strong></a> or <a href="https://showmax.engineering/2021/12/cdn-automation-3/"><strong>third part</strong></a>.</p><p><em>Originally published at </em><a href="https://showmax.engineering/articles/the-showmax-engineering-story-part-iii."><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4381905d1b8c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/the-showmax-engineering-story-part-iii-4381905d1b8c">The Showmax Engineering story, Part III.</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Showmax Engineering story, Part II.]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/the-showmax-engineering-story-part-ii-6b0bdd4f6ba9?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b0bdd4f6ba9</guid>
            <category><![CDATA[video-streaming-online]]></category>
            <category><![CDATA[vocabulary]]></category>
            <category><![CDATA[encoding]]></category>
            <category><![CDATA[video-streaming-service]]></category>
            <category><![CDATA[media-engineering]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:16:41 GMT</pubDate>
            <atom:updated>2024-01-19T10:16:41.205Z</atom:updated>
            <content:encoded><![CDATA[<h3>GETTING CONTENT TO THE CUSTOMER AS EFFICIENTLY AS POSSIBLE</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sw8Y1z2ol0uRVZI_" /><figcaption>Photo by <a href="https://unsplash.com/@jamie452?utm_source=medium&amp;utm_medium=referral">Jamie Street</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>On our blog we usually discuss our work in pretty in-depth terms, or terms that we set on our own. At times, we may speak a language that is unfamiliar, or even unintelligible, to some people. So, we decided to put together a vocabulary resource to help people understand the magic of the SVOD business. This is the second part in this series, again focused on Media Engineering, one of the most important areas of our work. If you missed the first one, you can <a href="https://tech.showmax.com/2020/11/media-engineering-vocabulary/">read it here</a>.</p><h3>Advanced terms in media encoding</h3><p>To encode video, you can use different approaches for bitrate control, depending on the encoding scenarios (archiving, streaming, or storing on devices with limited capacity like CDs or DVDs). <strong>Constant bitrate (CBR)</strong> is a type of encoding with a specified target bitrate value where the encoder uses adequate compression to produce a stream with the requested bitrate. It can waste bits on scenes with low complexity, and it is not suitable for archiving. But, it’s useful for streaming when you need to predict and control the file size.</p><p>The inverse approach is called <strong>variable bitrate (VBR)</strong>, a process of encoding in which you define the resulting quality. The encoder tries to achieve it using the lowest amount of bits possible, within some defined constraints. <strong>Average bitrate (ABR)</strong> is a variant of VBR, where the goal is to create files with predictable long-term average bitrate. This is achieved with 2-pass encoding, where the first pass is used to analyze the content, and the second one for actual encoding based on the previous analysis. It’s best for devices with limited storage space.</p><p>Another variant of VBR is <strong>Constant Quality (CQ) / Constant Rate Factor (CRF)</strong>, which, as the name suggests, produces the stream with full-length constant quality and controlled file size. It is very useful for content archiving, as it preserves content in the highest quality.</p><p>For streaming, it is important to have predictable bitrate and peaks within a given constraints. This is possible thanks to the <strong>Constrained Encoding (VBV)</strong> method, which is applicable for 2-pass VBR or CRF (also called <strong>capped CRF</strong>). It can control bitrate peaks and variability range. The advantage of capped CRF is that it requires only 1-pass. However, it’s important to find a reasonable CRF value in order to avoid hitting the maximum bitrate too often, and to avoid quality loss.</p><p>All of these methods are related to the encoding process. The decoding process is always the same regardless of bit allocation in the stream. Most of the codecs and encoders (free ones like <strong>FFmpeg</strong> or commercial solutions like <strong>AWS Elementals</strong>) support all of the above-mentioned methods.</p><h3>Bonus terms</h3><p><strong>Sintel</strong> and <strong>Big Buck Bunny</strong> are short, animated, “open movies” from the Blender Foundation. Thanks to their licensing, they are very popular within the video engineer community for testing and to compare encoding configurations. Note: watching the same video or scene over and over may have a dark side, check out <a href="https://www.youtube.com/watch?v=KDDif85hiUg">this lightning talk</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*TnfTbCSw7GrSH-03Sn-BmA.jpeg" /></figure><p>A <strong>mezzanine file</strong>, also known as an intermediate file, is a compressed version of the original source video without any perceivable quality loss. It has a form of standardized file format in terms of codecs, container, bitrate, FPS and more, and it’s both faster, and requires less bandwidth, to process mezzanine files into transcoded files. Movie files may be ingested in mezzanine format as part of the deal with studios, or you may create mezzanine files yourself as a preliminary step in your encoding workflow.</p><h3>Basic keywords</h3><p><strong>Progressive download</strong> is a technique of streaming video content used before Adaptive bitrate streaming. The file is transferred from server to client via HTTP and can be played before the download is complete if the required metadata is at the beginning of the media file. It’s not a stretch to say that fetching segments of ABR streaming is based on progressive download, so ABR streaming is just a sequence of progressive downloads.</p><p><strong>Adaptive bitrate streaming (ABR)</strong> is video streaming over HTTP, where players are provided with a list of streams in various codecs, resolutions, and bitrates, but containing the same content. The files are segmented into short, usually 2–10 seconds long parts (also called segments or chunks), that are described in the manifest/playlist file which the player uses for playback.</p><p>As a user, you’ve surely noticed that the quality of streams is usually worse at playback start but improves after 2–10 seconds. The client player downloads the segments based on current network state and device capabilities, continuously monitoring the state of the network, and may switch between streams to keep playback running smoothly and without rebuffering (read more about our <a href="https://tech.showmax.com/2021/02/keep-the-stream-live-despite-failure/">second stream backup strategy</a>. The player often does not have enough information at the beginning of the playback session, so it starts with lower bitrates. But, this depends on the player’s configuration, as individual segments must be in such a format that they can be played standalone.</p><p><strong>Bitrate</strong>, or <strong>encoding ladder</strong>, is the set of profiles (codec, resolutions, bitrate, etc.), or in other words, the set of encoding configurations used to create multiple files for Adaptive bitrate streaming to make video available on a range of devices using various connection bandwidth. The number of, and spaces between, steps of the ladder must be chosen based on the target devices and network conditions of the target market to be optimal for delivery.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TOszY1UW_CgsyMt09_qcjg.png" /><figcaption><em>Showmax bitrate ladder — our choice of resolution and bitrate, focused on users with lower connectivity. We are using capped CRF mode, so bitrates are upper caps.</em></figcaption></figure><p>For encoding within the bitrate ladder, two methods — CBR or VBR — and a set of constant parameters were used for a decade. The parameters were chosen based on the content set as a one-size-fits-all solution. This changed in late 2015, when Netflix introduced <strong>per-Title</strong> encoding. This approach adjusts the bitrate ladder to the actual content asset to achieve the highest possible quality. It was quickly adopted industry-wide in various ways, either as parameter optimization techniques, or by changing the number of rungs in the ladder and their resolutions. For example, if two consecutive rungs are comparable in quality, the one with higher bitrate can be dropped. The follow-up step is <strong>shot-based</strong> encoding, again introduced by Netflix, where encoding parameters are configured per the actual scene rather than the whole movie asset.</p><p><strong>HTTP Live Streaming (HLS)</strong> is a variant of HTTP-based adaptive bitrate streaming protocol introduced by Apple in 2009. It uses an extended M3U playlist (master playlist) to serve a list of available streams in the form of links to another extended M3u8 playlist (segment playlist) containing links to actual segments. It supports MPEG-TS or fragmented MP4 as file containers for media segments. HLS is used mainly by all Apple devices and the Safari browser, but is also supported on Android devices.</p><pre>#EXTM3U<br>#EXT-X-STREAM-INF:BANDWIDTH=150000,RESOLUTION=416x234,CODECS=&quot;avc1.42e00a,mp4a.40.2&quot;<br>http://example.com/low/index.m3u8<br>#EXT-X-STREAM-INF:BANDWIDTH=240000,RESOLUTION=416x234,CODECS=&quot;avc1.42e00a,mp4a.40.2&quot;<br>http://example.com/lo_mid/index.m3u8<br>#EXT-X-STREAM-INF:BANDWIDTH=440000,RESOLUTION=416x234,CODECS=&quot;avc1.42e00a,mp4a.40.2&quot;<br>http://example.com/hi_mid/index.m3u8<br>#EXT-X-STREAM-INF:BANDWIDTH=640000,RESOLUTION=640x360,CODECS=&quot;avc1.42e00a,mp4a.40.2&quot;<br>http://example.com/high/index.m3u8<br>#EXT-X-STREAM-INF:BANDWIDTH=64000,CODECS=&quot;mp4a.40.5&quot;<br>http://example.com/audio/index.m3u8</pre><p><em>Example of playlist with bitrate variants.</em></p><pre>#EXTM3U<br>#EXT-X-MEDIA-SEQUENCE:0<br>#EXT-X-TARGETDURATION:2<br>#EXTINF:2,<br>seq-0.ts<br>#EXTINF:2,<br>seq-1.ts<br>#EXTINF:2,<br>seq-2.ts<br>#EXTINF:2,<br>seq-3.ts<br>#EXTINF:2,<br>seq-4.ts<br>#EXTINF:2,<br>seq-5.ts<br>#EXT-X-ENDLIST</pre><p><em>Example of variant playlists with actual segments.</em></p><p><strong>Dynamic Adaptive Streaming over HTTP (DASH)</strong>, also known as <strong>MPEG-DASH</strong>, is another variant of HTTP-based adaptive bitrate streaming protocol. It is similar to Apple’s HLS solution, but uses an XML-based manifest to describe available streams. Segments are served in fragmented MP4 containers. It’s supported by a wide range of web browsers, Smart TVs, Android devices, and more.</p><pre>&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&gt;<br>&lt;MPD xmlns=&quot;urn:mpeg:dash:schema:mpd:2011&quot; type=&quot;static&quot; minBufferTime=&quot;PT1.5S&quot; mediaPresentationDuration=&quot;PT0H10M0.00S&quot; profiles=&quot;urn:mpeg:dash:profile:full:2011&quot;&gt;<br>    &lt;Period start=&quot;PT0S&quot; duration=&quot;PT0H10M0.00S&quot;&gt;<br>        &lt;AdaptationSet segmentAlignment=&quot;true&quot; bitstreamSwitching=&quot;true&quot; maxWidth=&quot;1920&quot; maxHeight=&quot;1080&quot; maxFrameRate=&quot;25&quot; par=&quot;16:9&quot;&gt;<br>            &lt;ContentComponent id=&quot;1&quot; contentType=&quot;video&quot;/&gt;<br>            &lt;SegmentTemplate timescale=&quot;1000&quot; duration=&quot;10000&quot; media=&quot;video-$RepresentationID$$Number$.m4s&quot; startNumber=&quot;1&quot; initialization=&quot;video_init.mp4&quot;/&gt;<br>            &lt;Representation id=&quot;low&quot; mimeType=&quot;video/mp4&quot; codecs=&quot;avc1.64000d&quot; width=&quot;320&quot; height=&quot;180&quot; frameRate=&quot;25&quot; sar=&quot;1:1&quot; startWithSAP=&quot;4&quot; bandwidth=&quot;67001&quot;&gt; &lt;/Representation&gt;<br>            &lt;Representation id=&quot;mid&quot; mimeType=&quot;video/mp4&quot; codecs=&quot;avc1.64001e&quot; width=&quot;640&quot; height=&quot;360&quot; frameRate=&quot;25&quot; sar=&quot;1:1&quot; startWithSAP=&quot;4&quot; bandwidth=&quot;228204&quot;&gt; &lt;/Representation&gt;<br>            &lt;Representation id=&quot;hd&quot; mimeType=&quot;video/mp4&quot; codecs=&quot;avc1.64001f&quot; width=&quot;1280&quot; height=&quot;720&quot; frameRate=&quot;25&quot; sar=&quot;1:1&quot; startWithSAP=&quot;4&quot; bandwidth=&quot;580017&quot;&gt; &lt;/Representation&gt;<br>            &lt;Representation id=&quot;full&quot; mimeType=&quot;video/mp4&quot; codecs=&quot;avc1.640028&quot; width=&quot;1920&quot; height=&quot;1080&quot; frameRate=&quot;25&quot; sar=&quot;1:1&quot; startWithSAP=&quot;4&quot; bandwidth=&quot;899712&quot;&gt; &lt;/Representation&gt;<br>        &lt;/AdaptationSet&gt;<br>        &lt;AdaptationSet segmentAlignment=&quot;true&quot; bitstreamSwitching=&quot;true&quot; lang=&quot;und&quot;&gt;<br>            &lt;AudioChannelConfiguration schemeIdUri=&quot;urn:mpeg:dash:23003:3:audio_channel_configuration:2011&quot; value=&quot;1&quot;/&gt;<br>            &lt;ContentComponent id=&quot;1&quot; contentType=&quot;audio&quot;/&gt;<br>            &lt;SegmentTemplate timescale=&quot;1000&quot; duration=&quot;9520&quot; media=&quot;audio$RepresentationID$$Number$.m4s&quot; startNumber=&quot;1&quot; initialization=&quot;audio_init.mp4&quot;/&gt;<br>            &lt;Representation id=&quot;aaclc_low&quot; mimeType=&quot;audio/mp4&quot; codecs=&quot;mp4a.40.02&quot; audioSamplingRate=&quot;44100&quot; startWithSAP=&quot;1&quot; bandwidth=&quot;19042&quot;&gt; &lt;/Representation&gt;<br>            &lt;Representation id=&quot;aaclc_high&quot; mimeType=&quot;audio/mp4&quot; codecs=&quot;mp4a.40.02&quot; audioSamplingRate=&quot;44100&quot; startWithSAP=&quot;1&quot; bandwidth=&quot;66341&quot;&gt; &lt;/Representation&gt;<br>        &lt;/AdaptationSet&gt;<br>    &lt;/Period&gt;<br>&lt;/MPD&gt;</pre><p><em>Example of DASH manifest.</em></p><p><strong>SmoothStreaming</strong> is another variant of HTTP-based adaptive bitrate streaming protocol developed by Microsoft as an IIS Media Services extension. It uses an XML-based manifest enlisting available streams. The principle is the same as HLS or DASH, differing only in Manifest syntax and used media file containers. It is supported by devices implementing Streaming Client software development kits for Windows, and a Smooth Streaming Porting Kit for other operating systems.</p><pre>&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&gt;<br>&lt;SmoothStreamingMedia MajorVersion=&quot;2&quot; MinorVersion=&quot;1&quot; Duration=&quot;1209510000&quot;&gt;<br>    &lt;StreamIndex Type=&quot;video&quot; Name=&quot;video&quot; Chunks=&quot;61&quot; QualityLevels=&quot;8&quot; MaxWidth=&quot;1280&quot; MaxHeight=&quot;720&quot; DisplayWidth=&quot;1280&quot; DisplayHeight=&quot;720&quot; Url=&quot;QualityLevels({bitrate})/Fragments(video={start time})&quot;&gt;<br>        &lt;QualityLevel Index=&quot;0&quot; Bitrate=&quot;2962000&quot; FourCC=&quot;AVC1&quot; MaxWidth=&quot;1280&quot; MaxHeight=&quot;720&quot; CodecPrivateData=&quot;...&quot;/&gt;<br>        &lt;QualityLevel Index=&quot;1&quot; Bitrate=&quot;2056000&quot; FourCC=&quot;AVC1&quot; MaxWidth=&quot;992&quot; MaxHeight=&quot;560&quot; CodecPrivateData=&quot;...&quot;/&gt;<br>        &lt;QualityLevel Index=&quot;2&quot; Bitrate=&quot;1427000&quot; FourCC=&quot;AVC1&quot; MaxWidth=&quot;768&quot; MaxHeight=&quot;432&quot; CodecPrivateData=&quot;...&quot;/&gt;<br>        &lt;QualityLevel Index=&quot;3&quot; Bitrate=&quot;991000&quot; FourCC=&quot;AVC1&quot; MaxWidth=&quot;592&quot; MaxHeight=&quot;332&quot; CodecPrivateData=&quot;...&quot;/&gt;<br>        &lt;c d=&quot;20020000&quot;/&gt;<br>        ...<br>        &lt;c d=&quot;6670001&quot;/&gt;<br>    &lt;/StreamIndex&gt;<br>    &lt;StreamIndex Type=&quot;audio&quot; Index=&quot;0&quot; Name=&quot;audio&quot; Chunks=&quot;61&quot; QualityLevels=&quot;1&quot; Url=&quot;QualityLevels({bitrate})/Fragments(audio={start time})&quot;&gt;<br>        &lt;QualityLevel FourCC=&quot;AACL&quot; Bitrate=&quot;128000&quot; SamplingRate=&quot;44100&quot; Channels=&quot;2&quot; BitsPerSample=&quot;16&quot; PacketSize=&quot;4&quot; AudioTag=&quot;255&quot; CodecPrivateData=&quot;1210&quot;/&gt;<br>        &lt;c d=&quot;20201360&quot;/&gt;<br>        ...<br>        &lt;c d=&quot;8126985&quot;/&gt;<br>    &lt;/StreamIndex&gt;<br>&lt;/SmoothStreamingMedia&gt;</pre><p><em>Example of SmoothStreaming manifest.</em></p><p>The files for these streaming protocols usually use the same encoded content (codec), but the container differs. To produce streaming files for various streaming protocols, the tool called <strong>packager</strong> may come in handy. It can remux/repackage the encoded source file into the desired streaming protocol. It also often supports DRM encryption or on-the-fly packaging, useful for live streaming.</p><p>A popular free tool is <strong>Shaka Packager</strong>. Here at Showmax, we use a commercial tool called <strong>Unified Origin</strong> or <strong>Unified Packager</strong>. Note that encoders can also produce files in streaming protocol formats, but as implied above, packagers can prevent you from wasting time and storage due to multiple encoding.</p><p>We hope you find this resource useful, and encourage you to read (or re-read) some of our other posts that may now be a bit more clear. <a href="https://tech.showmax.com/2020/06/pushing-video-bitrate-to-the-limit/"><strong>Pushing video bitrate to the limit</strong></a> explains the challenges of African infrastructure we face, and how we manage to stream the video at the top possible quality while saving customer data. Another related post, <a href="https://tech.showmax.com/2021/02/keep-the-stream-live-despite-failure/"><strong>Keep the stream live, despite any infrastructure failure</strong></a>, should also be a bit more accessible after going through these terms.</p><p>Enjoy!</p><p><em>Originally published at </em><a href="https://showmax.engineering/articles/the-showmax-engineering-story-part-ii."><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b0bdd4f6ba9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/the-showmax-engineering-story-part-ii-6b0bdd4f6ba9">The Showmax Engineering story, Part II.</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Showmax Engineering story, Part I.]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/the-showmax-engineering-story-part-i-89e599cd7418?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/89e599cd7418</guid>
            <category><![CDATA[video-streaming-online]]></category>
            <category><![CDATA[media-engineering]]></category>
            <category><![CDATA[vocabulary]]></category>
            <category><![CDATA[encoding]]></category>
            <category><![CDATA[video-streaming-service]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:15:40 GMT</pubDate>
            <atom:updated>2024-01-19T10:15:40.939Z</atom:updated>
            <content:encoded><![CDATA[<h3>HERE’S A RUNDOWN OF THE MAIN TERMS WE USE AT SHOWMAX ENGINEERING</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eTGKeYa59D0jMnz-" /><figcaption>Photo by <a href="https://unsplash.com/@cookiethepom?utm_source=medium&amp;utm_medium=referral">Cookie the Pom</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Showmax delivers a top-shelf VOD service across the African continent. On top of Hollywood hits, TV series, and local telenovelas, we offer news, music and live sports.</p><p>Here on the Showmax tech blog, we write about the many challenges that we encounter with media engineering, improving UX, managing peaks during live events, facing infrastructure, hackers, and much (much) more. Our posts are typically pretty in-depth, and we often use terms we see as basic, but may need proper explanation.</p><p>So, we’ve put together a list of terms and definitions for the core part of our business — media engineering. Here, we start with basics, but we will get in to more complex terms and topics in part II of this series.</p><h3>The basic terms of Showmax Engineering</h3><p><strong>Over the top (OTT)</strong> — When the content is transferred to viewers over the internet.</p><p><strong>Video on demand (VOD)</strong> — A service that allows users to access video they want to watch directly without sticking to the TV guide schedule as known from regular TV broadcasting.</p><h3>VOD business models:</h3><p><strong>Advertising-based (AVOD)</strong> — Access is usually free for customers and the service is supported by advertisers.<br><strong>Subscription (SVOD)</strong> — Access is behind paywall and customers pay for monthly, weekly, or daily access. Subscribers can consume as much content as they want — this is what we focus on at Showmax.<br><strong>Transactional (TVOD)</strong> — Access is behind paywall, but it’s pay-per-view, where customers pay for access to particular pieces of content or set amount of time.<br><strong>Premium (PVOD)</strong> — Premium video on demand is a version of TVOD, but the price is much higher and the rules are more strict. You can watch the movie at the same time as it’s being aired in US cinemas, but you can watch it only once.</p><h3>Encoding &amp; Decoding</h3><p>Accessing content and making content available for users is the core of media engineering. What do we do with the media to get it ready for transfer and playing on whatever player? We encode it and prepare it to be decoded.</p><p><strong>Video &amp; Audio codecs/formats</strong> are algorithms that encode or decode a digital data stream or signal. While encoders compress video data (e.g. to transfer), the decoders do the opposite, they uncompress data (e.g. to display). That’s why you have an encoder in your camera and decoder in your TV. The most popular video codec is H.264/AVC1, but it’s being replaced by its successor, H.265/HEVC, and royalty-free VP9 is being replaced by its successor, AV1. Audio codecs include Opus, the family of AAC codecs (AAC-LC, HE-AAC), or the line of Dolby codecs like Dolby Digital/AC-3, Dolby Digital Plus/E-AC-3, or Dolby Atmos/AC-4.<br><strong>File Formats/Containers</strong> are like shells that wrap encoded video, audio and other data into one file. That data is also called “tracks” within the container, and the process of creating/changing the container is called “transmuxing.” Popular containers are:</p><ul><li>MPEG-TS used in broadcasting for streaming</li><li>MPEG-4 popular for streaming video over the internet and storing video files (in all of its variants, MP4, FMP4, MOV)</li><li>MKV, a versatile format used mostly for distribution of whole video files, supporting vast amount of audio and video codecs while being royalty free</li><li>WEBM which is quickly gaining popularity as a format for video content on websites, replacing GIF images.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*WY1Jgxsqj8RFJ0fBQqOjfQ.png" /></figure><p><strong>Bit rate</strong> is the amount of bits used for coding video/audio data within one second (bps). It determines what throughput or bandwidth is required to stream the video in a particular quality on the network. More bits preserve more information from the signal, fewer mean that the information from the signal is lost. The quality of codecs is measured by how well they compress, in other words, how much data is able to store/compress into a given amount of bits. Better encoding compression requires more computational power; the same is often true for decoding.</p><p><strong>Resolution</strong> is the amount of pixels in X and Y dimension used for coding frames in video. There are several standardized resolutions based on height:</p><ul><li>Standard Definition = SD = 576p</li><li>High Definition = HD = 720p</li><li>FullHD = FHD = 1080p</li><li>4K Ultra High Definition = 4K UHD = 3840p</li><li>8K Ultra High Definition = 8K UHD = 7680p</li></ul><p>Resolution and bit rate are often used when describing quality. The ratio between width and height is called <strong>aspect ratio</strong>, commonly expressed as two numbers separated by a colon — most-commonly 4:3 and 16:9. We recognize 3 kinds of <strong>aspect ratio</strong>:</p><ul><li>Display aspect ratio (DAR)</li><li>Storage aspect ratio (SAR)</li><li>Pixel aspect ratio (PAR)</li></ul><p>The relationship within them can be expressed as PAR = DAR/SAR.<br>The encoded video data actually consists of a series of compressed still images called <strong>frames</strong>. Codecs encode either as the whole image ( <strong>I-frames</strong>), with reference to the previous images ( <strong>P-frames</strong>) to be decoded, or the previous and subsequent images ( <strong>B-frames</strong>). I-frames require the most bits, and B-frames lowest. The amount of consecutive I/B/P frames may impact things like overall visual quality and seeking speed. A lack of I-frames may cause corruptly displayed frames like gray areas after seeking, but it usually heals itself once it hits an I-frame and redraws the whole display.</p><h3>Maximising encoding power with scaling, and minimising encoding time with an in-house scheduler</h3><p>The Showmax Media Engineering team maximises the performance of our encoding software by using horizontal scaling ( <a href="https://tech.showmax.com/2019/01/divide-encode-1/">read how</a>). The system is powerful even if the encoding runs on hardware that isn’t necessarily high-end. Even if we use a server or a developer’s laptop, the output remains the same.</p><p>A parallel encoding pipeline goes hand-in-hand with encoding split into small, logical units — a nice thing to have when things go wrong. Encoding jobs in parallel pipelines are scheduled by our very own scheduler using the API. We considered using solutions like Celery or Apache Airflow, but concluded that it would be harder to fit these to our needs — reading the existing schedulers — than to just build our own custom solution.</p><p>In the Content Management System, we incorporated <a href="https://tech.showmax.com/2019/03/divide-encode-2/">our own simple scheduler</a> that helps us reduce encoding time significantly. To share files between the encoders, we use GlusterFS.</p><p>To learn more, and to get a wider perspective on media engineering at Showmax, wait for part II. of this vocabulary exercise. It will focus on decoding, live streaming, data saving, bitrates and more.</p><h3>More frequently used terms in media encoding</h3><p><strong>Frame rate</strong> (measured as frames per second; FPS) defines how many images are compressed in one second, and then displayed in that frequency. It is usually expressed as a number, but can also be expressed as a fraction, e.g. 25/1. Higher FPS means a smoother overall video, but also larger video files and higher bitrates. Modern video standards are usually 24, 25, and 30 FPS, 23.97, 24.97, and 29.97 respectively. High values like 120 are ideal for slow motion — they can be slowed down 4x to 30 FPS and playback is still smooth.</p><p><strong>Interlacing</strong> is a technique of visually doubling the frame rate by inserting two consecutive frames into one. This is usually applied when it’s beneficial to have a higher frame rate, but there are space and bandwidth limitations. However, decoded interlaced frames are visually very unpleasant and have to be deinterlaced. Most modern codecs are not optimized for interlaced scan but rather progressive scan, since most video files are transmitted and stored digitally.</p><p><strong>Group of Pictures (GoP)</strong>, expressed as a whole number, is a group of ordered inter-frames and intra-frames. Each GoP starts with intra-frame, I-frame, followed by bipredictive and predictive frames. Even though GoP is usually expressed as one number, it can sometimes be described as two numbers — the distance between two intra-frames and the distance between the intra-iframe and predictive frame. A group of pictures can be either opened or closed. Predictive and bi-predictive frames within an <strong>opened GoP</strong> can reference frames from other GoPs, but predictive and bi-predictive frames within a <strong>closed GoP</strong> can only reference frames within the same GoP. This is important, especially for streaming video with adaptive bitrate, because streaming with open GoPs can lead to issues with referencing invalid frames and the video can be distorted, hurting UX.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*ji5RfEFxDy4kRDgzzjFQcg.png" /></figure><p><strong>Color space</strong> is an organization of colors and their combinations. Depending on the application, we have to choose from two models: subtractive and additive. Using the subtractive model, a color is removed from key color to increase the level of light that can pass through. Using the additive model, a color is added to key color to increase the level of light passing through or reflecting. Colors are analog values, so we have to use a discrete representation of combinations to store colors digitally. The most common color spaces used today are the RGB additive model and CMYK subtractive model.</p><p><strong>Lip syncing</strong> is the technique of matching audio of people speaking or singing to actual video of those people. The most common form of lip syncing is dubbing or voice-overs. In video streaming, non-matching lip syncing is a sign of uneven audio and video alignment due for various reasons, like slow decoding of the video stream, or frame drops that render the video behind audio.</p><p><strong>Decoding timestamp (DTS)</strong> tells us when we need to decode a frame. For example, we have a sequence of bi-predictive frames referencing a predictive frame in the future. To be able to decode these B-frames, we have to decode the P-frame first.</p><p><strong>Presentation timestamp (PTS)</strong> expresses the actual time when a decoded frame should be displayed. This is often different from DTS.</p><p><strong>Are you interested in Media Engineering, join us we are hiring! Check out </strong><a href="https://tech.showmax.com/open-positions/2091441-python-developer---video-encoding/"><strong>the open position in our team</strong></a><strong>.</strong></p><p><em>Originally published at </em><a href="https://showmax.engineering/articles/the-showmax-engineering-story-part-i."><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=89e599cd7418" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/the-showmax-engineering-story-part-i-89e599cd7418">The Showmax Engineering story, Part I.</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Divide & Encode: How to Encode Videos Blazingly Fast]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/divide-encode-how-to-encode-videos-blazingly-fast-e2df44bfc8a2?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2df44bfc8a2</guid>
            <category><![CDATA[media-engineering]]></category>
            <category><![CDATA[ffmpeg]]></category>
            <category><![CDATA[scalability]]></category>
            <category><![CDATA[encoding]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:14:43 GMT</pubDate>
            <atom:updated>2024-01-19T10:14:43.395Z</atom:updated>
            <content:encoded><![CDATA[<h3>PART II — UNDER THE HOOD</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*D22E6OWMoaFajXKX" /><figcaption>Photo by <a href="https://unsplash.com/@thevanegmond?utm_source=medium&amp;utm_medium=referral">Nathan Van Egmond</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This is the second (and final) part of our blog mini-series about boosting encoding speed. In <a href="https://tech.showmax.com/2019/01/divide-encode-1/">the first part</a> we wrote about how we managed to maximize the power of our encoding farm via parallelization of the encoding pipeline. Here, we will give you a look under the hood of our implementation, and the issues we encountered along the way.</p><p>Implementation has two main parts. First, we took the original pipeline in the encoder service and split it in to small, logical units — this helps keep things simple and reliable in the processing pipeline. You will appreciate this approach when things go wrong. Bugs strike and you must debug a lot very quickly. The second part, the heavy lifting of the correct definition of the parallel pipelines, and the scheduling of respective jobs, is left on the API. It takes dependencies between jobs, priorities, and also jobs’ and workers’ features into account when assigning jobs to the individual encoder machines. To share files between workers we use GlusterFS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*YyhYZDzCKLiY0vqi-NQLfA.png" /></figure><p>Our implementation allows us to run on a physical machine (e.g. a server or developer’s laptop), or in the cloud (e.g. a Docker container). It is designed generically so that even our Analytics Team can use it for their deep learning computations on movies and series. The scene-detection algorithm uses multiple neural networks to detect what is happening in the video, and runs on dedicated GPU-equipped machines, but, from the global perspective, it behaves like any other encoder task.</p><h3>Under the hood</h3><p>Initially, we were looking for some existing scheduling and enqueueing tools, like <a href="http://www.celeryproject.org/">Celery</a> or <a href="https://airflow.apache.org/">Apache Airflow</a>. tl;dr, we probably suffer from <a href="https://en.wikipedia.org/wiki/Not_invented_here">not-invented-here</a> (NIH) syndrome a little bit. We considered fitting our demands to existing schedulers harder than building a custom solution, so we implemented our own simple scheduler which is incorporated into our Content Management System (CMS). Before the parallel encoding pipeline, our CMS already provided our Content Team with basic tools for video editing, cropping, trimming, and setting up external audio tracks.</p><p>We designed the scheduler to support the following features:</p><ul><li><strong>Simple notation of the pipeline:</strong> Pipeline is programmatically defined and dynamically changeable by video properties per individual encoding tasks. See the code snippet below. All classes ending with Recipe hold the configurations of jobs and methods for root_node, spread_children_node, and collect_children_node generates nodes and edges between them.</li></ul><pre>def generate(self, **kwargs):<br>    r&quot;&quot;&quot;Generate parallel encoding pipeline.<br><br>    Brief diagram of pipeline:<br>         split_segment<br>        /   || \   \<br>    images [*chunks] [*audios]<br>     |    \|/     //<br>     |  [*merge_store]<br>     |      \|/<br>     | [*drm_store]<br>     |  |//<br>     clean<br>    &quot;&quot;&quot;<br>    # Setup split segment job.<br>    ...<br>    split_segment_recipe = SplitSegmentRecipe(**split_segment_opts)<br>    split_segment_node = self.root_node(recipe=split_segment_recipe)<br>    # Setup audio encode jobs.<br>    ...<br>    # Setup video chunk encode jobs.<br>    ...<br>    for video_stream_opts_group in encode_video_chunk_groups:<br>       for i in range(chunk_range_start, chunk_range_end + 1):<br>           ...<br>           encode_chunk_recipe = EncodeVideoChunkRecipe(<br>               **encode_video_opts<br>           )<br>           encode_chunk_recipes.append(encode_chunk_recipe)<br><br>    encode_chunk_nodes = self.spread_children_node(<br>       parent=split_segment_node, recipes=encode_chunk_recipes)<br><br>    # Setup image encode job.<br>    ...<br>    # Setup merge encode jobs.<br>    for streams_opts in merge_mux_stream_groups:<br>       ...<br>       merge_mux_recipe = MergeMuxStoreRecipe(<br>           **merge_mux_opts_group<br>       )<br>       merge_mux_node = self.collect_children_node(<br>           parents=merge_mux_recipe.filter_nodes(<br>               encode_audio_nodes + encode_chunk_nodes),<br>           recipe=merge_mux_recipe)<br>       merge_mux_nodes.append(merge_mux_node)<br><br>    # Setup DRM jobs.<br>    ...<br><br>    # Setup clean job.<br>    ...<br>    clean_recipe = CleanRecipe(<br>       **clean_opts<br>    )<br>    self.collect_children_node(<br>       parents=drm_store_nodes + [image_node], recipe=clean_recipe)</pre><ul><li><strong>Advanced scheduling:</strong> The scheduler reflects the queue prioritization and other specific requirements for the job (e.g. requires GPU, etc.) when it is acquired by the concrete machine. In other words, we can target a specific job on a specific set of workers.</li><li><strong>Simple management control:</strong> The management tool should be able disable and enable workers, monitor the progress of job processing (via heartbeats), cancel processing prematurely if some job of the pipeline fails (again via heartbeats), or recover/restart from the job where the failure occurred. Certainly, it expects that the worker knows how to accept and process the responses from the API (eg. heartbeats) properly.</li></ul><h3>Performance debugging</h3><p>Our CMS is built on Django, so it was easy to customize the admin views. We created two interactive widgets. The first one is for displaying relationships between different jobs of the encoding task in the form of a directed acyclic graph (DAG), shown in Image 2. It is very useful when checking the pipeline definition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*NaOyV_cYmcgEAZyWXNorUg.gif" /></figure><p>For the latter, in Image 3, we used Gantt graph (again interactive). This helps us debug and improve the performance of the encoding pipeline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*sr29CK8njEXRE5zLBgoi_Q.jpeg" /></figure><h3>Logging</h3><p>At Showmax, we log everything. We have a fully-fashioned logging infrastructure using ELK stack. Your logging process should be defined so you can monitor your software as easily as possible. It’s easy for us to monitor the processing of the encoding task with more encoders — and in real time — in Kibana (see Image 4).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*-wC9IPqMxzcxSZ1ZKjUj1w.png" /></figure><h3>Watch for state</h3><p>In order to observe the actual state of encoding remotely, or by some external utility, FFmpeg provides an option: -progress.</p><pre>$ ffmpeg -progress progress.log -i ...<br>-progress url (global)<br> Send program-friendly progress information to url.</pre><p>Progress information is written approximately every second, and at the end of the encoding process. It is made of “key=value” lines made up of alphanumeric characters. The last key of a sequence of progress information is always “progress”. ( <a href="https://ffmpeg.org/ffmpeg.html">FFmpeg</a>)</p><p>Then, without affecting the encoder application, you can easily watch encoding speed and progress like this:</p><pre>$ tail -f progress.log | egrep &quot;(speed|out_time)=&quot;<br>out_time=00:00:01.706667<br>speed= 3.4x<br>out_time=00:00:03.221333<br>speed=3.18x<br>out_time=00:00:04.458667<br>speed=2.95x<br>out_time=00:00:05.162667<br>speed=2.54x</pre><p>I personally use it often in both the developing and testing phase, and when investigating issues in production.</p><h3>New software, new bugs, and new fails</h3><p>Unfortunately, not everything went perfectly. We hit two major issues, both related to processing segmented video. Let’s repeat briefly. In the first step, the video is segmented to chunks with a length divisible by group of pictures (GoP) of mezzanine file, every chunk is encoded by one worker, and, after processing, all of the chunks are merged to the final video.</p><h3>Cutting issue</h3><p>A basic example of GoP of a mezzanine file is one-second, and chunks encoded by one encoder are 10 seconds long. In Image 5., you can see the encoding without trimming, which produces the video correctly with a regular GoP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*dLSIxzx397TrTJ8Rlzfw9A.png" /></figure><p>In Image 6., trimming is all set up. Only the first and the last chunk is encoded, as trimmed and middle chunks are encoded in full length as before. The video has irregular GoP because the start cut is out of the key frame.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*I_PpED4W1yrn9tF3VlJmag.png" /></figure><pre>frame|key_frame=1|pkt_pts_time=5.000000<br>frame|key_frame=1|pkt_pts_time=6.000000<br>frame|key_frame=1|pkt_pts_time=7.600000     &lt;---- irregularity<br>frame|key_frame=1|pkt_pts_time=8.600000<br>frame|key_frame=1|pkt_pts_time=9.600000</pre><p>To prevent that, all encoders must produce 10-second chunks (except the last one, of course), as shown in Image 7. Therefore, to encode one chunk, the encoder has to get more segments and cut only specific 10-second clips of video off of segments. In other words, the encoders will process overlapping segments and make a precise cut-off while encoding. In order to mitigate overlap and transferred video segments, we split video to smaller parts (for brevity, one second) at the splitting step.</p><p>In fact, we split to 10-seconds segments, which are encoded into chunks of minimum one-minute, with a maximum count of chunks-per-video of 50, as we currently have 50 workers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*h6m2W7NjU9MzetgHgWmrIA.png" /></figure><p>But wait, there’s more. Start/end cuts must be aligned on frames, otherwise FFmpeg aligns it auto-magically in order to keep the desired length of chunk, but the frame count does not have to correspond the duration. After concatenating to the final MP4 file, the duration can actually be shorter.</p><p>Short disclaimer about our solution and Showmax mezzanine files: We could start creating segments from the farthest-left I-frame of the start cut to minimize overlapping of segments to one second (the frequency of I-frames in our mezzanine file). But, we plan to move the segmentation step to the phase where the mezzanine file is created (before our parallel encoding pipeline). We must also admit that there are other possibilities to specify a mezzanine file with <a href="https://en.wikipedia.org/wiki/DNxHR_codec">DNxHR</a> codec, or encoding with I-only frames in order to mitigate (or completely avoid) that cutting issue. However, years ago, we made a compromise and tried to find the balance between the size and flexibility of a mezzanine file. The best option for us is H.264 (high profile and level 5.2), with bitrate of 16Mbps, and a regular closed GoP. It gives us higher transfer speed and storage capacity for mezzanine files, and minimal quality loss.</p><h3>Merging issue</h3><p>This issue was a bit of a mystery. During the QA process, we found that some Nexus TV devices, and older Android devices with Widevine DRM, failed to play…sometimes. The Widevine DRM error message said:</p><pre>W/MDRMOemCrypto: duplicate SPS NALU (skipping)<br>E/MDRMOemCrypto: ERROR: multiple PPS NALUs (not supported by video decoder)<br>E/MDRMOemCrypto: Failed to process subsamples<br>E/WVCdm: Decrypt error result in session sid43 during encrypted block: 178</pre><p>Respectively:</p><pre>Player error (videoId: 290ebde0-e14e-46e2-bc16-7a56ec844fdb; currentPosition: 5344000; error: error (UNKNOWN, Type: 260, Extra: -1004))</pre><p>Further investigation revealed that the issue occurs only when using DRM when streams are switched. The multi-bitrate/resolution stream without DRM plays just as well as the Single-bitrate/resolution stream without DRM. After more Googling, we found a very familiar <a href="https://github.com/google/ExoPlayer/issues/2254#issuecomment-273801032">issue in exoplayer</a>.</p><p>We started experimenting with various FFmpeg flags when splitting and merging videos, but eventually it helped to replace FFmpeg with the mkvmerge command in order to merge chunks to one video properly. It is also worthy to mention that we are using Matroska container format for all intermediate steps in the pipeline like splitting audio and video tracks, transcoding and etc. We struggled with tricky A/V sync issue years ago and MKV file format helped us with keeping A/V in-sync.</p><h3>Conclusion</h3><p>Let’s recap. Horizontal scaling is very powerful way how to boost the performance of video encoding software, even if you don’t have high-end hardware. We managed to lower encoding time significantly by using a simple, custom-built scheduler. That allows us to run encoding workers in different environments, such as the cloud or on a farm of physical machines. Moreover, it’s applicable to any costly computations which can be parallelized and distributed.</p><p>There are, however, some constraints in this approach to parallel video encoding. The input/mezzanine file must have regular and closed GoP in order to be able split video to smaller segments properly so they can be encoded by more workers. But, if you do it correctly, you will rejoice.</p><p><a href="https://https//twitter.com/ShowmaxDevs/status/1103281249525944321">Follow the article</a> to get updates and comment.</p><p><em>Originally published at </em><a href="https://showmax.engineering/articles/divide--encode-how-to-encode-videos-blazingly-fast1"><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2df44bfc8a2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/divide-encode-how-to-encode-videos-blazingly-fast-e2df44bfc8a2">Divide &amp; Encode: How to Encode Videos Blazingly Fast</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Divide & Encode: How to Encode Videos Blazingly Fast]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/divide-encode-how-to-encode-videos-blazingly-fast-cc182505ef6a?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc182505ef6a</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[media-engineering]]></category>
            <category><![CDATA[ffmpeg]]></category>
            <category><![CDATA[scalability]]></category>
            <category><![CDATA[encoding]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:14:11 GMT</pubDate>
            <atom:updated>2024-01-19T10:14:11.349Z</atom:updated>
            <content:encoded><![CDATA[<h3>PART I — NEED FOR SPEED</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ufc6d_XFGibWIYBH" /><figcaption>Photo by <a href="https://unsplash.com/@adrianvalverdem?utm_source=medium&amp;utm_medium=referral">Adrián Valverde</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Here at Showmax, we’re addicted to speed and performance. In this post, I’ll go through a few techniques we used to improve the speed of our encoding pipeline, maximize the computing power of our encoders, and shortening our encoding time.</p><p>When it was all said and done, we reduced encoding times from several hours to several minutes. That means a drastically-reduced delay between our receiving the content from the providers and publishing it on our platform — so our customers can rejoice in all of this great content sooner.</p><p>I’ll begin by introducing our legacy encoding pipeline and all of its pros and cons. Then, I’ll get into our motivations to change it … in this case with our idea to use a parallel encoding pipeline.</p><h3>Once Upon a Time…</h3><p>In 2015, as Showmax was still in its very early stages, our encoding pipeline was created as simply as possible, following the <a href="https://en.wikipedia.org/wiki/KISS_principle">KISS</a> principle, to process content especially reliably. The Showmax encoder supports several pipelines dedicated to different purposes, like preparing mezzanine file, encoding video, and more.</p><p>Mezzanine assets — digital assets created in an intermediate step, especially in the video and broadcast industry. — <a href="https://en.wikipedia.org/wiki/Mezzanine_(disambiguation)">Wikipedia</a></p><p>In fact, these encoding pipelines are just a set of Python scripts serializing the individual steps of processing shown on Image 1. Technically, those steps either send HTTP requests to the various internal and external API endpoints, or call some UNIX utilities like <a href="https://ffmpeg.org/ffmpeg.html">FFmpeg</a> and <a href="https://rsync.samba.org/">rsync</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*DjYgYjN9KoX8FKbxSV1MEA.png" /></figure><p>NOTE: The actual encoding is done by FFmpeg — the Swiss army knife of encoding tools. It has many (many) options — more than most users/developers would ever need to use. As my colleague Jiri Brunclik puts it: “FFmpeg can do anything, even make coffee. But there are only two people in the world who know the right parameters. :-) “</p><p>Between 2015 and 2017, our entire pipeline was running on a bunch of physical machines, with one instance per machine. Each instance was only capable of processing one video at a time. We used to have 140 <a href="https://www.hetzner.com/dedicated-rootserver/ex40">EX40</a> machines hosted at Hetzner; later, we moved to using 50 <a href="https://www.hetzner.com/dedicated-rootserver/px61">PX61</a> machines.</p><p>In order to save on storage space, the encoders produce only MP4 files which are later encrypted ( <a href="https://www.widevine.com/solutions/">Widevine Modular</a>, <a href="https://www.microsoft.com/playready/">PlayReady</a> and <a href="https://developer.apple.com/streaming/fps/">HLS FairPlay</a>) and packaged to streams ( <a href="https://dashif.org/">DASH</a>, <a href="https://www.microsoft.com/silverlight/smoothstreaming/">SmoothStreaming</a> or <a href="https://developer.apple.com/streaming/">HLS</a>) with <a href="https://www.unified-streaming.com/software-makes-delivery-online-video-simple">Unified Streaming Platform</a>. This is done on-the-fly, and then packaged files are stored in cache on the CDN edge servers.</p><p>Simplicity is always great, but it has its downsides. Serial processing is slow … sometimes very slow. Encoding time depends on the length of the video and the complexity of image compression (animation is usually easier to compress than live-action video). In our case, the encoding process usually took <strong><em>3–4x</em></strong> the length of the video.</p><p>Generally, the slow encoding pipeline was not really a big deal. You know, <a href="https://en.wiktionary.org/wiki/if_it_ain%27t_broke,_don%27t_fix_it">if it ain’t broke, don’t fix it</a>. But there are situations when encoding time really does matter:</p><ul><li><strong>Publishing high priority content:</strong><br>Sometimes we acquire content shortly after its premiere and it needs to be published as soon as possible. The time gap between ingesting videos to our system and expected publish time is limited. Tasks in the queue can be reordered by prioritization. But even prioritized task still must wait for a free machine if the queue is full. For extra urgent content, we had dedicated queues with more powerful machines. However, both solutions are suboptimal.</li><li><strong>Maximizing utilization of the encoder farm:</strong><br>When the encoding queue is full (size is greater than the count of encoders in the farm), it means every encoder is processing one video and computing power is being used to the max. But when the encoding queue is not full, a part of the encoders is just slacking. Our encoding farm of 140 machines used to have only CPU utilization on average.</li><li><strong>Inexpensive scaling:</strong><br>More expensive machines could improve the encoding speed, but the price can be too high in comparison to the relatively small gain (eg. in our tests of encoding performance PX61 was comparable to the twice-as-expensive PX121). It’s also called <a href="https://en.wikipedia.org/wiki/Scalability#Horizontal_and_vertical_scaling">vertical scaling</a> and just means adding more power (CPU, RAM) to an existing machine.</li><li><strong>Incorporating next-generation codecs:</strong><br>Codecs like H.265/HEVC, VP9, and AV1 have a great impact on quality, as they can compress video with lower bitrate while keeping the same quality as H.264. But the encoding speed is terribly slow (but it’s improving). The speed of VP9 encoding is still ~20x worse than for H.264 and it would cause an increase of the encoding time from hours to days.</li></ul><h3>Divide &amp; Encode</h3><p>We concluded that vertical scaling was not the way we wanted to go, mainly because of the small improvement relative to cost. We did some experiments with encoding video to all variants at once with one FFmpeg instance and it yielded a performance boost of about 22% with really minimal effort in pipeline rewriting (thanks to the decoding of input video only once). That’s a good start.</p><p>We decided to focus on horizontal scaling more and engaging more machines to process one pipeline. If every variant could be processed by one encoder, it would consume 16 machines (that’s the count of our variants). But it’s not sufficient for us — currently our farm of encoders contains 50 PX61 machines. By splitting video to smaller parts (segments) we could use more (or all of the) machines to encode and we’d utilize our encoding farm to its full potential.</p><p>Caveats: Well, it’s not our idea originally, it’s just about taking an existing approach and applying it to our ecosystem. Despite this, we’d like to share some of our design details and the struggles we had.</p><p>The proposed parallel pipeline can be displayed as <a href="https://en.wikipedia.org/wiki/Directed_acyclic_graph">directed acyclic graph</a> (DAG) shown on Image 2 below. The nodes represent parts of pipeline (AKA “jobs”) that can be processed independently on separate machines, and the edges define dependencies between nodes (define the order of processing of jobs). From the top of the image, the video is split to chunks. The processing of chunks is distributed to the encoding farm (note that the count of chunks can vary based on the length of video). After that, the chunks are merged to the final video MP4 file and uploaded to the storage servers. Then, content is registered with our DRM providers. The last/bottom node of the DAG is just the internal garbage collector — the results of individual jobs are stored in the shared file system. Moreover, every job can have defined requested features specifying what capabilities the machine must have to acquire the job. Based on that, we are able to have machines with different hardware and power, and control what jobs they should process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*St-QPlyu0yLzeuzdPTjhGA.jpeg" /></figure><p>All of the issues I mentioned before were solved by using the parallel approach:</p><ul><li><strong>Publishing high priority content:</strong><br>The feature of prioritization is still valid, but much improved in this case, as the individual jobs are shorter from start to finish. The jobs submitted later, and with higher priority, can start being processed sooner in the full queue. As a consequence we don’t need the dedicated machines either.</li><li><strong>Maximizing utilization of the encoder farm:</strong><br>The parallel encoding pipeline is designed to minimize idle power, which results in improvement in processing speed. By splitting work between more machines, we’re always (trying to) use the maximum number of free machines.</li><li><strong>Inexpensive scaling:</strong><br>As we designed a fully-scalable parallel encoding pipeline, it can be run in the cloud or directly on a developer’s notebook. We can easily adjust the count and performance of machines to minimize cost, and maximize the power and fine-tune by defining requested features as well.</li><li><strong>Incorporating next-generation codecs:</strong><br>Encoding speed is still 20x slower. However, with parallelization, it’s not days but hours and can be improved by scaling up (adding more machines to the farm). In addition, we can distribute the encoding of variants of chunks to more machines, or change the chunk size, since our proposed parallel pipeline is fully-scalable horizontally.</li></ul><h3>Speed</h3><p>The naive expectation was that the speed gain factor would be a linear function of the count of machines, like using two machines means it’s twice as fast, three machines is three times faster. and so on. But parallel processing requires sharing files between machines, as well as splitting/merging video to chunks and vice versa. All those parts are necessary overheads and every level of pipeline can utilize the different count of encoders. So what was the real speed gain?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*3z952Ps6CahC9ci-As387w.jpeg" /></figure><p>The graph on Image 3 shows the performance statistics of our different configurations. Values are expressed relatively as multiples of the original video length. Using the approach of encoding all variants at once helps minimize overheads, and is why processing parallel pipelines via one encoder can perform better in comparison to the legacy serial pipeline (1st and 2nd rows in graph).</p><p>Increasing the encoder count actually does boost — almost linearly — up to a point, but it depends on the count of jobs generated for one video to process in parallel (2nd-4th rows), as well as on the overall count of jobs generated for more videos in the queue (5th row). Then, for a queue of 5 videos, the relative average time of encoding of one video is 0.46x / 5 = 0.09x. All of them are testing cases in ideal conditions.</p><p>After a month in production, the running numbers are as follows. Pipelines of “encode” type represent approx. 31% of all encoding pipelines, but consume more than 71% of overall computing time on average. After applying parallelization on those pipelines in November 2018, we managed to lower their overall computing time to 33% and relative average encoding time from ~3.9x to 1x total video length. The numbers for parallelization also calculate time spent by waiting for a free worker after a part of the pipeline was already done, therefore they are actually even lower than presented here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*vGgr_yX_4T4oHhTpU9QO3A.jpeg" /></figure><h3>Next Up</h3><p>In the next piece in this series, we’ll give you a look under the hood, share some practical tips, and of course go through some issues we’ve had so far.</p><p><em>Originally published at </em><a href="https://showmax.engineering/articles/divide--encode-how-to-encode-videos-blazingly-fast"><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc182505ef6a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/divide-encode-how-to-encode-videos-blazingly-fast-cc182505ef6a">Divide &amp; Encode: How to Encode Videos Blazingly Fast</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Life of Image]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/life-of-image-686e18621692?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/686e18621692</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[resize-images-api]]></category>
            <category><![CDATA[scalability]]></category>
            <category><![CDATA[graphicsmagick]]></category>
            <category><![CDATA[media-engineering]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:13:13 GMT</pubDate>
            <atom:updated>2024-01-19T10:13:13.618Z</atom:updated>
            <content:encoded><![CDATA[<h3>HOW TO EFFICIENTLY SERVE IMAGES IN VARIOUS SHAPES AND FORMATS</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gIOCjQo8n8WR87gl" /><figcaption>Photo by <a href="https://unsplash.com/@frostroomhead?utm_source=medium&amp;utm_medium=referral">Rodion Kutsaiev</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Here at Showmax, we use images all over our streaming platform. They can have various forms — background, hero or poster image, carousels, etc. After years of production experience with image management we have a lot to share with you, so here’s a look at the images workflow at Showmax.</p><p>The life of every image begins in our Content Management System (CMS). Basically, it’s the heart of our content metadata workflow. Image management is built in to the CMS, where images are stored on GlusterFS volume and served to clients via Image API as shown on the image here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*ZIVAheMC16cU6t2jBYVLPw.png" /></figure><h3>Image workflow</h3><p>A content operator usually creates and uploads several pictures for each movie or TV series. Then they prepare carousels, posters and hero images with different aspect ratios, localized per each country where the asset is licensed, and more. Based on the dominant colors of an image, the system automatically computes for font colors for overlaying text and background color gradient. These images are then served to users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*y-P-Y7hjcCjlS6E0tkUzkw.jpeg" /></figure><p>Moreover, we have another type of image — “Image overlays”. They are something like the image snippets and served only as stuck on prepared pictures. Most of them have text marks like <em>“Coming soon”</em>, <em>“Editor’s choice”</em>, <em>“Last chance”</em>, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*tq2f9VwMyCSj_ii6k8Auug.png" /></figure><p>Subsequently, the images are served to the customers via a microservice called Image API. It’s a place where the actual magic of efficient transformation happens. As stated in the title, we use one origin source image and transform it to different shapes and formats on-the-fly. The Image API knows how to do some transformations, like:</p><ul><li><strong>Nothing</strong>, only image quality is set to the default value of 75.</li><li><strong>Resizing</strong> just by appending parameters “width=”, “height=” or “800x500” to the URL. Transformation-by-default keeps the same aspect ratio as the original picture, but you could enforce resizing to exact dimensions with a special mark “f”.</li><li><strong>Cropping</strong> by appending parameters “crop_width=”, “crop_height=” or “crop=x” to the URL.</li><li><strong>Desaturating</strong> by appending boolean switch “desaturate” to the URL.</li><li><strong>Padding</strong> by appending parameters “pad=x” and “pad_color=” to URL.</li><li><strong>Sticking pre-defined overlays</strong> by appending parameter “overlay=” to the URL. The value is a JSON string encoded in base64 containing overlay attributes like: position, opacity, and scale.</li><li><strong>Setting quality</strong> by appending parameter “q=” to the URL.</li><li><strong>Changing the output</strong> format by changing the suffix in the URL.</li></ul><p>All of those options can be applied on their own, or arbitrarily combined with others. Front end applications then ask the image service for a picture and, after it’s generated, it is cached by a CDN and delivered to the client.</p><h3>Where the magic(k) happens</h3><p>Our Image API is built on <a href="https://falconframework.org/">Falcon framework</a> and <a href="http://www.graphicsmagick.org/">GraphicsMagick</a> tool as an image processing engine, hidden behind CDN caches. The main benefit of GraphicsMagick for us was the processing of format — something other tools such as Pillow or ImageMagick did not have implemented at that time.</p><p>Here are some interesting tips and unexpected troubles we had during the implementation.</p><h3>Streaming the images efficiently</h3><p>When composing image with overlay (another image), we have to process several images together. The process consists of several steps during which new images are created (as mezzanine) and stored on disk after each call of GraphicMagick. They are then merged back together by another call of GraphicMagick. Every call of GraphicMagick invokes the reading of the image from disk and writing the results to disk — that’s a lot of expensive I/O operations. We needed to find out how to read every image only once and store intermediate and final results in memory.</p><p>Luckily, GraphicMagick supports a format called <strong>MIFF</strong> — it means, <em>“Magick Image File Format”</em>, and is used for storing bitmap images. You could join many images into a single one, or use it as a stream of images like using pipes in Unix systems.</p><p>Our more-efficient image workflow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*hRpFDs8qnEci8-r5nbKLVg.png" /></figure><p><em>1. Read one original picture from a filesystem<br>2. Read overlay image from a filesystem<br>3. Resize the overlay image based on the size of overlayed image<br>4. Stick overlay image to the original picture<br>5. Resize the result and send it to the user</em></p><h3>Dominant color</h3><p>Earlier you read about dominant color of images and where we use it. The problem was that we had to compute the dominant color using GraphicsMagick — which does not have such a function.</p><p>The solution is quite simple (with a bit of out-of-the-box thinking). There are steps:</p><ol><li>Transform the input image to the same color space (as other color spaces like CMYK will provide different results than RGB color space).</li><li>Drop all profiles and text attributes from the image.</li><li>Remove the matte channel if the image has one.<br>undefined</li><li>Reduce the colors of the image to the minimum — one.</li><li>Resize the image to 1x1 pixels in MIFF format.</li></ol><p>Now we just need to read the color palette containing one color from 1x1 pixel image. We don’t even have to parse the file, just read the last 3 bytes where the color code is always placed. The length and structure of 1x1 MIFF images is always the same.</p><p>The GraphicsMagick commands could looks like this:</p><pre># Transform to RGB color space with one color - steps from 1 to 4<br>/usr/bin/env gm convert - -strip +matte -colorspace RGB -colors 1 -compress none miff:-<br><br># Resize image to 1x1 - step 5<br>/usr/bin/env gm convert miff:- -resize 1!x1 -compress none miff:-</pre><h3>GraphicsMagick as a computational engine</h3><p>Here’s why we use GraphicsMagick as a calculator. We’ve already mentioned that several image transformations can be applied to an image in one request on Image API.</p><p>Sometimes a front-end application needs to process more operations in one call. It needs to resize image → crop → stick overlay — everything — while preserving aspect ratio in the resize step, and the app specifies only the required width.</p><p>Resize and crop is done in one step, so we need to know what the image’s height would be after resizing in order to adjust crop parameters properly. We can’t compute it in python due to different approach to rounding — we need to use the same kind of rounding as GraphicsMagick. So, we prepare an empty canvas with the original dimensions of the input image, try to resize, and find out the new dimensions.</p><p>This trick can be done like this:</p><pre># Resize empty canvas<br>/usr/bin/env gm convert -size ${IMAGE_WIDTH}x${IMAGE_HEIGHT} xc:none -monochrome -resize ${NEW_IMAGE_WIDTH:-}${FORCE}x${NEW_IMAGE_HEIGHT:-} -compress none miff:-<br><br><br># Get new image properties<br>/usr/bin/env gm identify -verbose -format %m:%w:%h -</pre><h3>Conclusion</h3><p>The image API helps our front-end teams with basic image transformations without the need to upload the different variants of images to our CMS. Our content teams can mark images with overlays without having to use an image editor to burn in an overlay and then upload a new image.</p><p>Image API currently works with an image library containing more than 300,000 images, and growing. Transforming images on-the-fly is quite costly, therefore every variant of a given image is computed only once and then cached on a CDN with an 85% cache hit ratio.</p><p>The last (but certainly not least) advantage of our in-house solution is the cost savings. Commercial services providing the same functionalities usually have payment models that consist of a fee for every single transformation. Ours has a semi-fixed cost, as we’re caching the images on the CDN and the network traffic is almost negligible. Finally, it is easy to maintain, since it has small codebase of less than 1000 lines.</p><p><em>Originally published at </em><a href="https://showmax.engineering/articles/life-of-image"><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=686e18621692" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/life-of-image-686e18621692">Life of Image</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Delivering content the right way, Part I.]]></title>
            <link>https://medium.com/showmax-engineering-developers-news/delivering-content-the-right-way-part-i-3a5fc64fee16?source=rss-dfacd6bb4753------2</link>
            <guid isPermaLink="false">https://medium.com/p/3a5fc64fee16</guid>
            <category><![CDATA[cdn]]></category>
            <category><![CDATA[media-engineering]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[content-delivery-network]]></category>
            <category><![CDATA[vod]]></category>
            <dc:creator><![CDATA[Peter Lisák]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 10:09:13 GMT</pubDate>
            <atom:updated>2024-01-19T10:09:13.571Z</atom:updated>
            <content:encoded><![CDATA[<h3>SELECTING THE RIGHT URL</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IVtfVDaArp7lMlqj" /><figcaption>Photo by <a href="https://unsplash.com/@herrherrmann?utm_source=medium&amp;utm_medium=referral">Sebastian Herrmann</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Video on Demand (VOD) platforms usually support a wide range of devices. Despite the attempts to unify streaming protocols, DRMs, and other parts of the tech stack, it’s almost impossible to have only one playlist/manifest that fits everyone.</p><p>The problem actually goes beyond supporting different streaming protocols or DRM types. The list is long and may contain codecs, performance, or levels of protection. At Showmax, we tackle this challenge with a component called Playback API.</p><h3>Playback API</h3><p><strong>TL;DR: Playback API is a service that provides a content URL based on the information provided by a device or application. Content is then delivered to the device/application in a format that can be played back in the best available quality.</strong></p><p>In the early days of Showmax, the main purpose of the Playback API was to select a playlist/manifest from formats like DASH, SmoothStreaming, or HLS, with the right DRM system. The device or application let developers (read: the Frontend teams) know what streaming protocol and DRM is supported, so they just call the API that provides this information and receive the content URL. For example: Android/AndroidTV always get a DASH manifest with Widevine DRM, and iOS/tvOS get an HLS playlist with FairPlay. However, web browsers or smart TVs are not so simple and we need to use some sort of detection.</p><p>It’s worth noting that all Showmax APIs that serve content-related data use the Showmax Content Management System (CMS) as the source of truth. In addition to managing actual content and scheduling by content operators, the CMS is also used for configuring content protection, and to decide which CDN should be used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*NmdFOQhfVM7eyfK6PX7-WQ.png" /></figure><p>The content URL is chosen based on the information provided by the caller and the platform-related restrictions, both of which impact the URL that is returned. One such example is called Streaming Profile Restriction, which is basically a configuration matrix in our CMS that includes dimensions like device, streaming protocol, DRM, country, and maximum allowed resolution in the playlist. We have several packages (configurations generating playlists by a packager) with different sets of resolutions.</p><p>This solution was introduced when we had different content contracts for European and African markets. Even though we are now providing our services mainly in Africa, this feature still has some applications. We used it for limiting quality during <a href="https://www.linkedin.com/pulse/bandwidth-strategy-showmax-peter-lis%C3%A1k/">lockdowns</a> to relieve overloaded ISPs and their networks, and to drop higher profiles in playlists within our bandwidth-capping option saving data in applications. Now, this is mostly done directly by the players themselves.</p><p>When we added <a href="https://tech.showmax.com/2020/06/pushing-video-bitrate-to-the-limit/#next-gen-codecs">next-gen codecs</a> into the pipeline, we introduced a feature called “capabilities.” We found out during testing that providing the playlist with all codecs and letting players choose what they can play works in theory, not in practice. Capabilities are extra information detected and provided by the device to the API-they can ask for a preferred codec, for example. The process of selecting a preferred codec has two steps. The first is to detect codecs the device supports, then, if the most suitable codec is one of the next-gen codecs, test it in practice. The device may lie, which is especially true for cheaper, low-end Androids. The most suitable codec should save bandwidth, provide better picture quality, and be efficient with battery usage. Once a preferred codec is chosen and provided, the Playback API will adjust the content URL to reflect device capabilities.</p><p>In the future, we may introduce 4K capability to serve 4K resolution only on devices which have enough performance to decode such a resolution. It may sound defensive, but it’s really useful when covering lots of various edge cases.</p><h3>CDN application in CMS</h3><p>We always try to find the optimal path for a specific customer at the time of request to serve content in the highest-possible quality. To do it right is kind of the holy grail for CDN engineers.</p><p>Our CMS allows us to configure the preferred third party CDN (or CDNs) edges for specific countries or ISPs, which helps us pick the best CDN for the customer. It’s usually the closest Point of Presence (PoP), which, in reality, means we are using an in-house CDN for South Africa and nearby countries. Countries farther away from our PoPs are served by third party CDNs with a local PoP. Backup CDNs can be also configured as described in this blog post: <a href="https://tech.showmax.com/2021/02/keep-the-stream-live-despite-failure/">Keep the stream live, despite any infrastructure failure</a>.</p><p>Another use-case of preferring an ISP-specific CDN is when we have a special arrangement with an ISP, and the requirement is to use a specific network path for delivering content.</p><p>A special use-case is the configuration for so-called “hot assets”- a small subset of content in high demand. Increased demand requires us to involve extra CDN edges to serve that content (most recently, we experienced this with Game of Thrones and Chernobyl). This is automatically-detected based on the thresholds, and we use the amount of data-per-time window, or manually curated list of assets.</p><p>When entering a new territory or just adding a new “cold” CDN into the stack, we have the option to start prewarming that CDN by slowly increasing the amount of traffic routed there. This mitigates possible buffering issues caused by lots of MISS requests, and protects our origins from getting DDoSed by a CDN.</p><p>Finally, all of these configuration options can be time-based, with the traffic spread across more CDNs during weekends and evenings, and the in-house CDN during off-peaks hours for cost efficiency.</p><h3>Conclusion</h3><p>The process of selecting the right content URL is controlled on both sides-backend/CMS and frontend/applications-and the final mechanism of selecting the URL is done in the Playback API. A hybrid approach like this is beneficial because it leverages the advantages of both options while mitigating their respective disadvantages. With respect to flexibility, the backend changes can be released faster than changes in frontend applications. However, curating a list of devices with capabilities on the backend would be very tedious.</p><p>That’s all for today. In future installations, we will walk you through our incorporation of consistent hashing into CDN edge selection or some of the automations related to the CDN selection process aka our journey multi-CDN.</p><p><em>Originally published at </em><a href="https://showmax.engineering/articles/delivering-content-the-right-way-part-i."><em>https://showmax.engineering</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3a5fc64fee16" width="1" height="1" alt=""><hr><p><a href="https://medium.com/showmax-engineering-developers-news/delivering-content-the-right-way-part-i-3a5fc64fee16">Delivering content the right way, Part I.</a> was originally published in <a href="https://medium.com/showmax-engineering-developers-news">{//} Showmax Engineering: Archived Knowledge Base</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>