<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Luke Lefebure on Medium]]></title>
        <description><![CDATA[Stories by Luke Lefebure on Medium]]></description>
        <link>https://medium.com/@luke.lefebure?source=rss-9425101ee142------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*V1bPfH3iRAlKmkavR8bFQw.png</url>
            <title>Stories by Luke Lefebure on Medium</title>
            <link>https://medium.com/@luke.lefebure?source=rss-9425101ee142------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 20:56:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@luke.lefebure/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Exploring the UN General Debates with Dynamic Topic Models]]></title>
            <link>https://medium.com/data-science/exploring-the-un-general-debates-with-dynamic-topic-models-72dc0e307696?source=rss-9425101ee142------2</link>
            <guid isPermaLink="false">https://medium.com/p/72dc0e307696</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[united-nations]]></category>
            <category><![CDATA[topic-modeling]]></category>
            <category><![CDATA[text-analytics]]></category>
            <dc:creator><![CDATA[Luke Lefebure]]></dc:creator>
            <pubDate>Wed, 17 Oct 2018 14:35:18 GMT</pubDate>
            <atom:updated>2018-10-17T14:35:18.136Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Vr-ySFQjZv4ad8Hz" /><figcaption>Credit: <a href="https://unsplash.com/@lemonvlad?utm_source=medium&amp;utm_medium=referral">Vladislav Klapin</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In this article, I introduce the intuition behind Dynamic Topic Models and demonstrate this method’s ability to discover evolving narratives in unstructured text through an application to a corpus of speeches.</p><p>By reading, I hope you will pick up an understanding of Dynamic Topic Models, the problem that they try to solve, practical tips for working through a project with text data, and some insights about modern world affairs! If you would like to follow along with the code, you can find that <a href="https://github.com/llefebure/un-general-debates">here</a>.</p><p>Each fall, world leaders gather in New York to participate in the General Debate of the United Nations General Assembly. This year’s session concluded earlier this month after discussion around issues such as nuclear weapons and the war in Syria.</p><p>Speeches given at the General Debate form a historical record of the issues that have commanded the attention of the international community. Hidden in this text data are insights about how the dialogue changes over time, and Dynamic Topic Models can help us quantitatively model that evolution.</p><h3>Dynamic Topic Models</h3><p>Topic modeling is a generic term used to describe the process of finding topics in a corpus of unstructured text. A popular method for this is Latent Dirichlet Allocation (LDA) which is a generative model that learns a predefined number of latent topics, where each topic is represented as a distribution over terms and each document as a distribution over topics. The article below does a good job at explaining the idea, and the <a href="https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation">Wikipedia page</a> is also a good reference.</p><p><a href="https://medium.com/@lettier/how-does-lda-work-ill-explain-using-emoji-108abf40fa7d">Your Guide to Latent Dirichlet Allocation</a></p><p>LDA is a very powerful technique for the qualitative analysis of large corpora because of its highly interpretable topics, and it is also useful for dimensionality reduction as it transforms sparse document-term matrices into fixed low dimensional document-topic matrices. However, LDA ignores the temporal aspect present in many document collections.</p><p>Consider a topic about US politics found by LDA in a corpus of news from the past decade. You might expect the most probable terms to be “congress”, “election”, “vote”, etc. However, the model treats a document from 2010 the same as one from 2018 with respect to the term probabilities, so it is unable to learn that the term “trump”, for example, was almost nonexistent in the context of politics until relatively recently.</p><p>Dynamic Topic Models (DTMs), as presented by Blei and Lafferty <a href="https://mimno.infosci.cornell.edu/info6150/readings/dynamic_topic_models.pdf">here</a>, address this problem by extending the idea of LDA to allow topic representations to evolve over fixed time intervals such as years. Specifically, the documents within each time slice are modeled with a topic model of the same dimension, and each topic in time slice t evolves from a corresponding topic in time slice t-1.</p><p>The generative process of LDA remains more or less the same aside from the key difference that the overall topic distribution and the term distribution for each topic differ depending on the time slice. In particular, the parameters for these distributions “evolve” at each time slice by being drawn from distributions centered around the corresponding values from the previous time slice.</p><p>The end result of this is a series of LDA-like topic models that are sequentially tied together. A topic learned by a DTM is thus a sequence of related distributions over terms. With this intuition, I move now to its application.</p><h3>The General Debate Dataset</h3><p>A corpus consisting of speeches at the General Debate from 1970 through 2015 is <a href="https://www.kaggle.com/unitednations/un-general-debates/home">hosted on Kaggle</a> (7,507 in total). The dataset was originally released last year (see <a href="https://arxiv.org/pdf/1707.02774.pdf">here</a> for the paper) by researchers in the UK and Ireland who used it to study the position of different countries on various policy dimensions.</p><p>Each speech is tagged with the year and session it was given and the ISO Alpha-3 code of the country that the speaker represented.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1012/1*vAcrlLo2VRGqyBIzgBTM_w.png" /><figcaption>Sample rows from the dataset</figcaption></figure><h3>Data Preprocessing</h3><p>A key observation in this dataset is that each of these speeches consists of discussion on a multitude of topics. If every speech contains discussion on poverty and terrorism, a topic model trained on entire speeches as documents in a bag of words representation will have no way of understanding that terms like “poverty” and “terrorism” should be representative of different topics.</p><p>To counter this problem, I tokenize each speech into paragraphs and treat each paragraph as a separate document for the analysis. A simple rule based approach that looks for sentences separated by a newline character performs reasonably well on the task of paragraph tokenization for this dataset. After this step, the number of documents jumps from 7,507 (full speeches) to 283,593 (paragraphs).</p><p>After expanding each speech into multiple documents, I word tokenize, normalize each term by lowercasing and lemmatizing, and trim low frequency terms from the vocabulary. The end result is a vocabulary of 7,054 terms and a bag of words representation for each document that can be used as input to the DTM.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dab863b8ecfb9a2f6b610ef4e6b551e9/href">https://medium.com/media/dab863b8ecfb9a2f6b610ef4e6b551e9/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lgKG4R4s2Z7ivyuqkh1iHw.png" /><figcaption>Sample rows after paragraph tokenization</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yl5CrY3poT3CjXcglzap5Q.png" /><figcaption>Sample rows after word tokenization of paragraphs</figcaption></figure><h3>Inference</h3><p>To determine the number of topics to use, I ran LDA on a few different time slices individually with different numbers of topics (10, 15, 20, 30) to get a feel for the problem. Through manual inspection, 15 seemed to produce the most interpretable topics, so that is what I settled on for the DTM. More experimentation and rigorous quantitative evaluation could certainly improve this.</p><p>Using <a href="https://radimrehurek.com/gensim/models/wrappers/dtmmodel.html">gensim&#39;s Python wrapper</a> to the <a href="https://github.com/blei-lab/dtm">original DTM C++ code</a>, inferring the parameters of a DTM is then straightforward, albeit slow. Inference took about 8 hours on an n1-standard-2 (2 vCPUs, 7.5 GB memory) instance on Google Cloud Platform. However, I ran on a single core, so this time can probably be cut down if you can get the parallelized version of the original C++ to compile.</p><h3>Results</h3><p>The model discovered very interpretable topics, and I examine a few in depth here. Specifically, I show for a few topics the top terms at a sample of time slices as well as plots of probabilities of notable terms over time. A complete list of the topics discovered by the model and their top terms can be found in the appendix at the end of this article.</p><h4>Human Rights</h4><p>The <a href="http://www.un.org/en/sections/un-charter/preamble/index.html">Preamble of the UN Charter</a> states:</p><blockquote>We the peoples of the United Nations determined … to reaffirm faith in fundamental human rights, in the dignity and worth of the human person, in the equal rights of men and women and of nations large and small.</blockquote><p>It is no surprise then that human rights is a perennial topic at the General Debate and one that the model was able to discover. Despite the notion of gender equality appearing in the charter quoted above, the model shows that it still took quite some time for the terms “woman” and “gender” to really catch on. Also, note the rising use of “humankind” coupled with the decline of the use of “mankind”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*XzriDe8Z0OKyOnc46hP4_g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sDbLgtYobSCbmrjqPHwWpA.png" /><figcaption>Topic 3 (Human Rights): Top ten terms by probability at sampled time slices (left) and selected term probabilities over time (right)</figcaption></figure><h4>Israel</h4><p>Ever since the creation of modern Israel in 1948, there has been a constant debate in which the UN has been intimately involved around Arab-Israeli conflict, borders, settlements, and the treatment of the Palestinian people. In fact, there is an entire <a href="https://en.wikipedia.org/wiki/List_of_United_Nations_resolutions_concerning_Israel">Wikipedia page</a> devoted to the matter.</p><p>The model shows an increasing focus on Gaza after its blockade by Israel and Egypt in 2007 and a constant discussion around Israeli settlements, a practice the <a href="https://www.un.org/press/en/2016/sc12657.doc.htm">UN has deemed illegal</a> under international law. The rise of the term “two” is interesting, likely coming from growing discussion around a “two-state solution”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*wBoY5rql26EcznGzNztwZA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nNsMfla7Fop5lscSu8pGVg.png" /><figcaption>Topic 5 (Israel): Top ten terms by probability at sampled time slices (left) and selected term probabilities over time (right)</figcaption></figure><h4>Africa</h4><p>The discussion on Africa has evolved from its colonial past to independence, the era of apartheid, and finally the birth of new governments, elections, and democracy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*sEAZTs8d_JrmncUOsUD0CA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NS3RqrORrofpTJm2Mf9lNQ.png" /><figcaption>Topic 8 (Africa): Top ten terms by probability at sampled time slices (left) and selected term probabilities over time (right)</figcaption></figure><h4>Nuclear Weapons</h4><p>The Cold War era brought an arms race between the US and the Soviet Union and subsequent discussion on disarmament. More recently, the focus has been on Iran.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*5dCdCcLviAaYextTGRWRZg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_-TzdzrufU8ADMF-18pA8A.png" /><figcaption>Topic 2 (Nuclear Weapons): Top ten terms by probability at sampled time slices (left) and selected term probabilities over time (right)</figcaption></figure><h4>Development</h4><p>The Millennium Development Goals and the Sustainable Development Goals are UN initiatives introduced in 2000 and 2015 respectively to focus on solutions to issues such as poverty, education, and gender equality. This topic seems to have evolved from generic discussion around international development towards these structured programs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*kmwjCQzJZYZABQRu9pUPdg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OfYb7fmnrWy8oMRrysBk9A.png" /><figcaption>Topic 4 (Development): Top ten terms by probability at sampled time slices (left) and selected term probabilities over time (right)</figcaption></figure><h3>Conclusions</h3><p>This exercise demonstrated how DTMs represent topics and this method’s incredible ability to discover narratives in unstructured text. Remember, the model did all of the above without any human intelligence!</p><p>I could potentially improve the methods used by experimenting with a larger vocabulary size, different numbers of topics and evaluation thereof, phrase extraction (e.g. “human_rights” could be one token instead of having “human” and “rights” separately), and different stemmers or lemmatizers. Other methods that go beyond the bag of words representation and use word embeddings could be interesting to explore as well.</p><p>One aspect of DTMs not leveraged here (but one that I would like to explore further) is the learned vector representations for documents. These are not only useful for computing similarity values between documents but also for finding representative documents for each topic. For example, it would be interesting to get all of the speeches from US representatives, pick out the most representative paragraphs for each time slice for, say, the human rights topic, and use those paragraphs to construct a thread on the evolution on US discourse on human rights.</p><p>Overall, this turned out to be a fascinating exercise. If you would like to experiment yourself or dive deeper into the implementation details, all of the code for this analysis can again be found in the repo <a href="https://github.com/llefebure/un-general-debates">here</a>. Thanks for following along, and if you have additional ideas or questions, please leave a comment!</p><h3>Appendix</h3><p>Below is a complete list of the topics discovered by the DTM. For each topic, the top ten terms from the 1975, 1995, and 2015 time slices are shown.</p><pre>Topic 0<br>            1975           1995           2015<br>0       security        council        council<br>1            non       security       security<br>2          state         member         member<br>3        council          state         reform<br>4         cyprus        general         united<br>5  international         reform         nation<br>6      principle         nation      permanent<br>7         nation         united          state<br>8        aligned  international           must<br>9        charter           must  international<br><br>Topic 1<br>            1975           1995           2015<br>0            sea  international          state<br>1            law          state         island<br>2          state         island          small<br>3          latin           drug  international<br>4        america        country        country<br>5       resource          small           sids<br>6       american        america          crime<br>7        country      caribbean      caribbean<br>8     conference     convention     developing<br>9  international          crime        pacific<br><br>Topic 2<br>          1975           1995           2015<br>0      nuclear        nuclear         weapon<br>1  disarmament         weapon        nuclear<br>2       weapon         treaty           iran<br>3          arm    disarmament         treaty<br>4        state  proliferation            arm<br>5       soviet            non      agreement<br>6       treaty            arm  international<br>7        power          state            use<br>8        union           test            non<br>9         race  international       chemical<br><br>Topic 3<br>            1975           1995           2015<br>0          right          right          right<br>1          human          human          human<br>2         people        respect            law<br>3  international      democracy          woman<br>4        freedom         people         people<br>5      principle  international        respect<br>6        justice            law        freedom<br>7      political        freedom           rule<br>8        respect      principle  international<br>9           must        society          peace<br><br>Topic 4<br>            1975           1995           2015<br>0       economic       economic       economic<br>1  international    development        country<br>2          world  international          world<br>3        country          world         global<br>4            new            new  international<br>5          order         social         social<br>6    development      political    development<br>7       relation        country      political<br>8      political         global         crisis<br>9             co    cooperation            new<br><br>Topic 5<br>          1975         1995           2015<br>0         east        peace          state<br>1       middle         east          peace<br>2         arab       middle       solution<br>3       israel   resolution    palestinian<br>4        peace       people       conflict<br>5  palestinian  palestinian  international<br>6       people       israel          syria<br>7        right      process         people<br>8    territory       region           east<br>9    palestine    agreement         israel<br><br>Topic 6<br>         1975          1995           2015<br>0    republic        united         united<br>1      people        nation         nation<br>2      united         peace  international<br>3       state     operation          peace<br>4      nation       keeping       conflict<br>5  government      republic           role<br>6       korea      conflict   peacekeeping<br>7    peaceful        bosnia        mission<br>8        viet  organization        support<br>9     country   herzegovina       republic<br><br>Topic 7<br>        1975         1995           2015<br>0  operation  cooperation         region<br>1         co       region    cooperation<br>2      peace     regional  international<br>3    country     european        country<br>4     europe      country       security<br>5   security       europe       regional<br>6   relation        union       european<br>7   european        state      terrorism<br>8     region     security          union<br>9    detente    stability         effort<br><br>Topic 8<br>           1975        1995        2015<br>0        africa      africa     african<br>1         south     african     country<br>2       african     country      people<br>3        people  government  government<br>4        regime       peace    republic<br>5      southern   community      africa<br>6       namibia      people   political<br>7     apartheid  democratic   community<br>8  independence     process  democratic<br>9        racist      effort       peace<br><br>Topic 9<br>        1975      1995       2015<br>0      world       war    climate<br>1          u     world     change<br>2        one  conflict      world<br>3        war      cold     global<br>4    problem      many     threat<br>5       time         u  challenge<br>6  situation     peace       must<br>7      peace       one    country<br>8      power   problem     crisis<br>9   conflict      must   conflict<br><br>Topic 10<br>           1975        1995       2015<br>0        people      people     people<br>1      struggle     country        war<br>2       country     refugee       year<br>3  independence        year      world<br>4      national       state    country<br>5         force       force        one<br>6    liberation         war  terrorist<br>7         state  government      today<br>8   imperialist         one      state<br>9   imperialism        life       many<br><br>Topic 11<br>            1975           1995       2015<br>0         nation         nation     nation<br>1         united         united     united<br>2   organization   organization      world<br>3          world          world          u<br>4          peace           must       must<br>5         member  international  challenge<br>6  international              u     future<br>7        charter           role      peace<br>8           role          peace       year<br>9          state            new     global<br><br>Topic 12<br>          1975         1995         2015<br>0      country      country  development<br>1   developing  development      country<br>2  development     economic         goal<br>3     economic   developing  sustainable<br>4        world        trade          per<br>5    developed     resource       agenda<br>6     resource      economy         cent<br>7        trade    financial      poverty<br>8        price    programme       health<br>9   assistance   assistance    education<br><br>Topic 13<br>         1975       1995       2015<br>0          mr    general    general<br>1     general    session   assembly<br>2   president   assembly    session<br>3    assembly         mr  president<br>4     session  president         mr<br>5  delegation  secretary       like<br>6        also       also      would<br>7        like       like  secretary<br>8        wish   election       also<br>9    election       wish      sixty<br><br>Topic 14<br>         1975         1995         2015<br>0     session   conference  development<br>1    assembly  development         2015<br>2     general      general           70<br>3  conference         year       agenda<br>4  resolution       agenda         year<br>5        year     assembly  sustainable<br>6     special      meeting   conference<br>7  government       summit           pv<br>8      united         held           15<br>9     meeting         last         post</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72dc0e307696" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/exploring-the-un-general-debates-with-dynamic-topic-models-72dc0e307696">Exploring the UN General Debates with Dynamic Topic Models</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>