<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Axel de Romblay on Medium]]></title>
        <description><![CDATA[Stories by Axel de Romblay on Medium]]></description>
        <link>https://medium.com/@axelderomblay?source=rss-83077b56a8a4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*pjRB8jgehX6KI3TKVFQ5OQ.png</url>
            <title>Stories by Axel de Romblay on Medium</title>
            <link>https://medium.com/@axelderomblay?source=rss-83077b56a8a4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 13:07:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@axelderomblay/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[On topic annotation: how to extract relevant labels from videos?]]></title>
            <link>https://medium.com/dailymotion/topic-annotation-automatic-algorithms-data-377079d27936?source=rss-83077b56a8a4------2</link>
            <guid isPermaLink="false">https://medium.com/p/377079d27936</guid>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Axel de Romblay]]></dc:creator>
            <pubDate>Mon, 04 Feb 2019 09:04:17 GMT</pubDate>
            <atom:updated>2020-11-23T18:39:32.565Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dZVR53FTlcCHIDreIMZ2cA.jpeg" /></figure><p><a href="https://www.dailymotion.com/en"><strong>Dailymotion</strong></a><strong> is a global video platform hosting hundreds of millions videos in more than 20 languages. Our purpose is to share the most compelling music, entertainment, news and sports content around, thanks to partnerships with the world’s leading publishers and content creators, such as France Télévisions, Le Parisien, CBS, BeIN Sports, CNN, GQ, Universal Music Group, VICE, and more.</strong></p><p>In order to be able to <strong>characterize the content</strong> of our video catalog, the Data Team is working on <strong>automatic algorithms to extract topics from the video</strong>.</p><blockquote>“But why do we care at Dailymotion about being able to accurately categorize content at scale?”</blockquote><ul><li><strong>Watching</strong> <strong>interface</strong>: push videos with trending and<em> </em>popular topics, recommend videos related to a topic.</li><li><strong>Search engine</strong>:<strong> </strong>retrieve videos from a given topic.</li><li><strong>SEO and acquisition</strong>: increase the external visibility of our video catalog and get more new visitors.</li></ul><blockquote>“What technical challenges do we face for a relevant topic annotation algorithm?”</blockquote><ul><li><strong>Relevance &amp; quality of the topics</strong>: we want to have relevant and specific/precise topics. <em>E.g : “2018 FIFA World Cup” vs “Football”.</em></li><li><strong>High precision/coverage tradeoff</strong>: we want to tag (or cover) the maximum videos with at least one topic and with a minimum error rate. <em>E.g: we can have a 100% coverage with (almost) random topics vs 50% coverage with very accurate topics.</em></li><li><strong>Fast and up-to-date annotation</strong>: we need a fast annotation pipeline that proposes updated topics. <em>E.g: “Juventus” for a video related to Cristiano Ronaldo vs “Real Madrid”.</em></li><li><strong>Multi-lingual annotation</strong>: we need to tag videos for all the languages. <em>E.g: French, English, Korean videos…</em></li></ul><h3>Demystifying our topic annotation pipeline</h3><p><em>In this section, we will present the different steps of the actual topic annotation pipeline running at dailymotion.</em></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F8YHmc8luwmJJjFY7zh%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2F8YHmc8luwmJJjFY7zh&amp;image=https%3A%2F%2Fmedia0.giphy.com%2Fmedia%2F8YHmc8luwmJJjFY7zh%2F200.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/e3aea8523a016d318db06349ec788da4/href">https://medium.com/media/e3aea8523a016d318db06349ec788da4/href</a></iframe><p>There exist two types of data describing a video:</p><ul><li><strong>Raw data (aka metadata): </strong>the description, the frames and the audio.</li><li><strong>Additional data</strong>: the channel, the country/language, date, …</li></ul><p>In order to have a <strong>robust</strong> <strong>model</strong>, it is important to work as much as possible on (raw) <strong>metadata</strong>.</p><p>Before going into details, here is an overview of the actual pipeline:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mK6z68_R_vw0i1AqXJNHPA.png" /><figcaption>Schema of the actual pipeline running at dailymotion. Each step is presented below.</figcaption></figure><blockquote><strong>NB</strong>: In this post, we will focus on the pipeline <strong>running on the video descriptions only.</strong></blockquote><h3>1. Text extractor and language detector</h3><p><em>This step takes all the metadata related to the video as input and outputs the description and the associated language.</em></p><p>This first step is very important since it impacts all subsequent steps of the pipeline. It is a complex task we won’t detail and you can find all the explanations <a href="https://medium.com/dailymotion/on-language-detection-classification-x-user-consumption-3b3f1e97503b">here</a>:</p><p><a href="https://medium.com/dailymotion/on-language-detection-classification-x-user-consumption-3b3f1e97503b">On Language Detection: Classification x User Consumption</a></p><h3><strong>2. Topic maker on text</strong></h3><p><em>This step takes the video description and the corresponding detected language as inputs and outputs candidate topics related to the description.</em></p><p>Our approach is different from traditional <a href="https://en.wikipedia.org/wiki/Natural_language_processing">Natural Language Processing</a> <em>(aka NLP)</em> tasks with semantic analysis of the text. Instead, we used a framework that maps an <strong>unordered</strong> <strong>set of words</strong> (<em>the video description</em>) to <strong>universal Wikidata entities </strong><em>(candidate topics).</em></p><blockquote><strong>NB</strong>: this task is called “<a href="https://en.wikipedia.org/wiki/Entity_linking">Named Entity Linking</a>” (aka NEL).</blockquote><p><a href="https://www.wikidata.org/wiki/Wikidata:Main_Page">Wikidata</a> is an open source <strong>knowledge graph </strong>with billions of <strong>updated</strong> and <strong>inter-connected </strong>entities:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LCW7uy5PJeS0z6hUamgbmw.png" /><figcaption>Example of a Wikidata entity defined by its id: “Q15869”. More details <a href="https://www.wikidata.org/wiki/Q15869">here</a>.</figcaption></figure><p>Now, let’s explain how we do the mapping:</p><h4><strong>Preprocessing phase</strong></h4><p><em>As usual, we first need to clean the descriptions and tokenize the words</em>.</p><p>We remove low-frequency words, one-character words, … Also, we deal with <strong>overlapping words</strong> since words may overlap or be a substring of one another, we need to detect their boundaries.</p><blockquote>Example: “jaguar cars”, we discard the token “jaguar” and keep “jaguar cars” instead.</blockquote><h4><strong>Disambiguation phase</strong></h4><p><em>For a given word, we would like to pick the appropriate sense in case of polysemy.</em></p><blockquote>For example: “jaguar” can refer to the car or the animal… Actually, there exists a lot of Wikidata ids referring to this word: “<a href="https://www.wikidata.org/wiki/Q35694">Q35694</a>”, <a href="https://www.wikidata.org/wiki/Q26742231">Q26742231</a>”, “<a href="https://www.wikidata.org/wiki/Q2382933">Q2382933</a>”, “<a href="https://www.wikidata.org/wiki/Q650601">Q650601</a>”, …</blockquote><p>The disambiguation process is a combination of two features:</p><ul><li>The <strong>commonness (or prior-probability) of the sense given the word. </strong>It is computed as the probability that the word points to the sense by parsing the Wikipedia dataset.<strong> </strong><em>Back to the example, the animal is more likely to be the appropriate sense…</em></li><li>The <strong>relatedness score between the word and the sense</strong>. The idea is to pick the sense that best fits the context of the text: we use a <em>voting scheme</em> where all other words vote for the sense. The vote of a word to a sense is based on the overlap between their in-linking pages in Wikipedia. <em>Back to the example, if our description is “Tata Motors acquisition of Jaguar”, the word jaguar is related to the car…</em></li></ul><p>Then, we pick the sense with the <strong>highest disambiguation score</strong>, which is the sum of the two weighted features.</p><h4><strong>Pruning phase</strong></h4><p><em>Once we have mapped each word to a unique sense (or Wikidata entity), we want to select only the meaningful entities to generate candidate topics.</em></p><blockquote>Back to our example: “<a href="https://www.wikidata.org/wiki/Q188514">Tata Motors</a> acquisition of <a href="https://www.wikidata.org/wiki/Q26742231">Jaguar</a>”, we might discard “<a href="https://www.wikidata.org/wiki/Q3604747">acquisition</a>” which is not meaningful enough…</blockquote><p>The pruning phase is a combination of two features:</p><ul><li><strong>The link probability of a word</strong>: the probability that an occurrence of the word in <a href="https://en.wikipedia.org/wiki/Main_Page">Wikipedia</a> is a word pointing to some Wikipedia page. <em>Back to our example: “</em><a href="https://en.wikipedia.org/wiki/Tata_Motors"><em>Tata Motors</em></a><em> acquisition of </em><a href="https://en.wikipedia.org/wiki/Jaguar_Land_Rover"><em>Jaguar</em></a><em>”, the word “Tata Motors” is very likely to be a link in Wikipedia as we can see in figure below:</em></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*9jRgnULwYy2rNBJagouwxw.png" /><figcaption><em>“Tata Motors” is very likely to be a link in Wikipedia.</em></figcaption></figure><ul><li><strong>The (averaged) coherence of the word</strong> within the description. For each word (and its unique associated sense thanks to the disambiguation phase), we compute a coherence score with each other words and average all the score. The coherence score between two words (or senses) is also based on the overlap between their in-linking pages in Wikipedia. <em>Back to our example: “</em><a href="https://en.wikipedia.org/wiki/Tata_Motors"><em>Tata Motors</em></a><em> acquisition of </em><a href="https://en.wikipedia.org/wiki/Jaguar_Land_Rover"><em>Jaguar</em></a><em>”, the words “Tata Motors” and “Jaguar” are coherent between each other since both of them are car companies.</em></li></ul><h3>3. Topic <strong>Filter</strong> on text</h3><p><em>This step takes a set of candidate topics as input and outputs/selects the accurate ones.</em></p><p>Finding accurate topics related to a video is not an easy task. Very often, our candidate topics are <strong>not</strong> <strong>accurate enough</strong> for several reasons:</p><ul><li>The <strong>precision</strong> threshold allowed is very <strong>high</strong> <em>(for example if we set a 99% precision for our topics)</em>.</li><li>The description is <strong>ambiguous</strong> or not <strong>representative</strong> of the video <em>(this is the case if accurate topics come from other metadata like the frames or the audio)</em>.</li></ul><p>Thus, we need to build an <strong>automatic</strong> (machine learning) <strong>model</strong> that first <strong>ranks</strong> the candidate topics and then <strong>selects</strong> the accurate ones.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*A8rEmKudHpxCiPlU06cIIA.jpeg" /><figcaption>Our model will mimic human decision.</figcaption></figure><h4>Ranking of the candidate topics</h4><ul><li><strong>Feature engineering</strong></li></ul><p>For each candidate topics, we <strong>compute features</strong> that will help the model to decide whether the (candidate) topic is <strong>central</strong> or <strong>off</strong>.</p><blockquote><strong>NB:</strong> a “central” candidate topic is a topic associated to a video, otherwise if it is “off”, we discard it.</blockquote><p>In addition to all the previous features <em>(disambiguation score, averaged coherence score, …)</em>, we compute features related to the (Wikipedia) <strong>popularity</strong> of the topic, the <strong>match</strong> <em>(occurrence and position in the description/title)</em>, the <strong>affinity</strong> between the candidate topics and the <strong>coherence with the channel </strong><em>(for example a candidate topic about Basketball is more likely to be central for the channel BeIN Sports than CNN…).</em></p><p>We are also working on integrating <a href="https://medium.com/dailymotion/bag-of-words-representation-for-video-channels-semantic-structuring-4f2777591e4a"><strong>topic and channel embeddings</strong></a>:</p><p><a href="https://medium.com/dailymotion/bag-of-words-representation-for-video-channels-semantic-structuring-4f2777591e4a">Bag-of-words representation for video channels’ semantic structuring</a></p><ul><li><strong>Training of an optimal machine learning model on the features</strong></li></ul><p>In order to build our training set, we first had to <strong>manually</strong> <strong>validate which candidate topics are central or not</strong> on a small set of videos <em>(I know that’s a boring task but Amazon Mechanical Turk is doing it for you ;))</em></p><blockquote><strong>NB</strong>: in fact, validating topics is not an exact science. In some ambiguous cases, people might argue a topic is central to the video whereas others might argue the contrary! Fortunately, we can overcome this issue by averaging the votes…</blockquote><p>Then, we <strong>train (or fit) a model </strong>on a<strong> </strong>subset of the training set<strong> </strong>and<strong> find the optimal hyper-parameters </strong>on the rest<strong> </strong>of the training set<strong> </strong><em>(aka validation set)</em>: the optimal model has learnt (complex) rules on the features that will automatically decide for a new video which candidate topics are likely to be central or off. Indeed, the estimator will output a <strong>centrality score for all candidate topics for a given video, which allows us to do the ranking.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jaW7BE9Inp5qQg9Gvp4aWQ.png" /><figcaption>Schema of the machine learning model</figcaption></figure><h4>Selection of the candidate topics</h4><p><em>The selection allows us to push accurate topics on the dailymotion interface.</em></p><p>For the last step, we <strong>compute a centrality threshold on the validation set </strong>above which candidate topics are considered to be central to the video. We determine it by <strong>setting a minimum precision for our topics</strong>.</p><p>Once we obtain the centrality threshold, we can apply it and deduce the <strong>associated coverage on our video catalog</strong> and we are done! :)</p><blockquote><strong>NB</strong>: the coverage on our video catalog is relatively close to the coverage on the validation set.</blockquote><h3>What’s next?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*w7iG5P_BRtyC0k_Fs6LqvQ.gif" /></figure><p>Our solution meets <strong>all the criteria for a relevant topic annotation pipeline</strong> <em>(relevance of the topics, good precision/coverage tradeoff, up-to-date &amp; multi-lingual annotation).</em></p><p>In addition to other exciting projects <em>(e.g: recommendation systems)</em>, the Data Team keeps on working on improving the<strong> characterization of our video catalog </strong>by:</p><ul><li><strong>Working on the frames </strong>to<strong> </strong>cover the videos with no descriptions (or incomplete ones) or<strong> </strong>to<strong> </strong>get more topics per videos.</li><li><strong>Working on the topics</strong> to build an automatic categorization of our topics <em>(example: “2018 FIFA World Cup” &lt; “Association Football” &lt; “Sports”).</em></li><li><strong>Working on both descriptions and topics/categories </strong>to get a “contextual” categorization of our videos.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=377079d27936" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dailymotion/topic-annotation-automatic-algorithms-data-377079d27936">On topic annotation: how to extract relevant labels from videos?</a> was originally published in <a href="https://medium.com/dailymotion">Dailymotion</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>