<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Moses Boudourides on Medium]]></title>
        <description><![CDATA[Stories by Moses Boudourides on Medium]]></description>
        <link>https://medium.com/@mosabou?source=rss-b3c587159c74------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*KQG8qZkyOSy5d-zHcT8IWw.jpeg</url>
            <title>Stories by Moses Boudourides on Medium</title>
            <link>https://medium.com/@mosabou?source=rss-b3c587159c74------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 11:54:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mosabou/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Michel Foucault in the Cambridge Lexicon and ChatGPT]]></title>
            <link>https://medium.com/@mosabou/michel-foucault-in-the-cambridge-lexicon-and-chatgpt-996bb09892e0?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/996bb09892e0</guid>
            <category><![CDATA[network-science]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[social-network-analysis]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Tue, 28 Feb 2023 03:50:10 GMT</pubDate>
            <atom:updated>2023-02-28T03:57:13.056Z</atom:updated>
            <content:encoded><![CDATA[<p>This is an ongoing project in collaboration with Chenchen Zhang.</p><p>Are ChatGPT outputs trustful? Of course, to answer such a question one would need to cross-check or validate ChatGPT responses with whatever knowledge or information is accepted, established or considered in some sense to be true or valid. However, this may not be always a simple task, because the veracity of a statement often depends on the context of the input or the question to which the statement was produced. A general question might generate a general answer, which might be rather ill-defined to be assessed accurately. Conversely, a concretely specified question, which would happen to be clearly delineated inside a particular state or context of facticity, might be more unequivocally replied or the accuracy of a given replied to such question might be more assuredly cross-checked. For instance, asking “<em>What is power?</em>”, ChatGPT would respond in an essayistic way without giving any explicit references to notions that could be double-checked within other sources. Nevertheless, asking “<em>What is power according to Plato?</em>” chatGPT is certainly going to return a reference to “The Republic” and what we are suggesting is that it could be on the ground of such references that one might securely gauge the validity of responses generated by AI and LLMs.</p><p>Therefore, the venue for a possibly trustful validation of ChatGPT could be a contextual structure of associations, references or relationships among a group of cognate concepts, which would be taken as the input of queries, so that the given responses might be evaluated over possible occurrences of contextually adjacent notions in their content. Actually, the previously described contextual structure is often identified to a knowledge graph or a semantic network among such entities as things, events, situations, concepts, words etc., tied together by a pattern of contextual connections among them. In our study here, as such a contextual structure, we are taking the network of terms of a dictionary, which provides a relational pattern of references among all included terms.</p><p>In particular, we are considering “<em>The Cambridge Foucault Lexicon</em>,” edited by L. Lawlor &amp; J. Nale (Cambridge University Press, 2014), which contains clear and incisive definitions and descriptions of 91 Michel Foucault’s major terms, written by a number of Foucault’s scholars from a variety of disciplines. Moreover, for every entry of the Lexicon, the discussion of the corresponding term closes with the quotation of other relevant terms (in a paragraph named “<em>See Also</em>”). Thus, one may derive the Foucault (Lexicon) network, which is a directed graph consisting of the 91 terms as nodes and 427 references as links. The out-degree of a node/term is the number of nodes/terms to which the former is referring. The in-degree of a node/term is the number of nodes/terms from which the former is referred. In the Foucault network, the mean and standard deviation (std) of the out-degree is 4.69 and 1.47 (resp.) and for the in-degree are 4.69 and 4.86 (resp.). Moreover, the density (defined as the ratio of existing links/references to all possible ones) is 427/(91x90) = 0.052. In addition, 14.988% of links/references were found to be reciprocated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*xOKnxz5zcZgxjwkCKTAREg.png" /><figcaption>The Foucault Lexicon directed graph of terms and references among them.</figcaption></figure><p>Next, we were querying ChatGPT about Michel Foucault and each one of the 91 lexical terms and collecting the given responses. In particular, we used the following query “<em>What do you know about Michel Foucault and term X?</em>” Furthermore, we were searching inside ChatGPT’s responses for the occurrences of terms to which term X was referring to in the Lexicon. Thus, we found that ChatGPT validated 70 terms/nodes (77%) of the Foucault network nodes. The ChatGPT discovered terms, which were referred to by term X, were called “ChatGPT validated out-references” for X. Similarly, the ChatGPT discovered terms, which were referring to term X, were called “ChatGPT validated in-references” for X. In addition, the number of ChatGPT validated out- and in-references defined the “ChatGPT out- and in-degree” (resp.). In the plots below, ChatGPT validated references are colored red. Furthermore, the ratio of the ChatGPT out(in)-degree with the term X out(in)-degree (resp.) defined what we called “ChatGPT out(in)-coverage ratio.” Thus, we computed the mean and std of the ChatGPT out-degree equal to 1.07 and 0.94 (resp.) and for the ChatGPT in-degree 1.07 and 3.5 (resp.). Moreover, the mean and std of the ChatGPT out-coverage ratio was found 0.23 and 0.19 (resp.) and for the ChatGPT in-coverage ratio 0.19 and 0.32 (resp.). Totally, among the 427 links/references of the Foucault network 97 were validated by ChatGPT, making the “ChatGPT coverage density” equal to 97/427=0.23.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/959/1*oEQnn62IWyEim6t8zBownA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qpUEqKlzfgYY_rl8sHlpGg.png" /></figure><p>Apparently, the density of ChatGPT validated references is moderately low. However, it is differentiated in the ego-nets of particular nodes/references. For example, the ego-net of the term “<em>Technology (of Discipline, Governmentality, and Ethics)</em>” has ChatGPT coverage density equal to 0.43. Three other examples of ego-nets are displayed below. In conclusion, as one may observe in the boxplots and histograms of the ChatGPT out(in)-degrees the ChatGPT validation is rather low.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*mSVB7DYI9s7BY2Dcjxr46g.png" /><figcaption>The “<em>Technology (of Discipline, Governmentality, and Ethics)</em>” ego-net in the Foucault Lexicon network.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/1*2fU97UYNTf8aQzh9OVmuVA.png" /><figcaption>The “Governmentality” ego-net in the Foucault Lexicon network.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*tItAYmGZNLtDSAj9cJuAAA.png" /><figcaption>The “Power” ego-net in the Foucault Lexicon network.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*vbdSmQze65zvWMdypWvYkg.png" /><figcaption>The “Resistance” ego-net in the Foucault Lexicon network.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=996bb09892e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Topic Modeling and Networks of Co-Occurring Terms in Courses Learning Objectives]]></title>
            <link>https://medium.com/@mosabou/topic-modeling-and-networks-of-co-occurring-terms-in-courses-learning-objectives-cd283ee78669?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/cd283ee78669</guid>
            <category><![CDATA[topic-modeling]]></category>
            <category><![CDATA[network-science]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Fri, 17 Apr 2020 15:55:35 GMT</pubDate>
            <atom:updated>2020-04-17T15:55:35.207Z</atom:updated>
            <content:encoded><![CDATA[<p>The purpose here is to present the results of a data/network analysis of the text of the Learning Objectives of 15 Core STS courses taught at NYUAD in Spring 2020. To do so, we are trying to answer three rudimentary questions:</p><p>1. What are the salient or important words in the text of these Learning Objectives and how can one retrieve them through automated text-mining of the corpora?</p><p>2. What are the aggregate patterns, the relational adjacencies and the central foci that might emerge from the textual-semantic entanglement of these words in the Learning Objectives?</p><p>3. Can the overall structure, the common orientation and the shared objectives of the ensemble of the whole set of the Core STS courses be discerned through a computational NLP<a href="#_ftn1">[1]</a>-based analysis of the dataset of the Learning Objectives of these courses?</p><p>To answer these questions, we are employing the following standard computational techniques:</p><p>1. <em>Topic Modeling</em> calculating important words (or keywords), called “<em>terms</em>,” in the corpora of the Learning Objectives and classifying them into a small number of certain interpretable “<em>topics</em>.”</p><p>2. <em>Networks of co-occurring terms</em>, i.e., graphs among terms (derived by Topic Modeling) linked through their co-occurrences in the text of particular Learning Objectives.</p><p>3. <em>Community partition </em>of networks of co-occurring terms into a number of discrete groups (“communities”) of terms which lump together existing structural similarities of these terms so that the resulting community assortment might disclose the common characteristics of the Learning Objectives of the Core STS courses program.</p><p><strong>1.</strong> <strong>Topic Modeling</strong></p><p>Inside the corpus of the Learning Objectives of 15 Core STS courses taught at NYUAD in Spring 2020, we were able to locate the following four topics of terms:</p><p>· <em>Urbanity</em>,</p><p>· <em>History</em>,</p><p>· <em>Science</em> and</p><p>· <em>Revolution</em>.</p><p>These topics are displayed in the following bar plots, in which the length of each bar indicates the probability of the corresponding term to belong to a topic. Notice that all terms are <em>stemmed</em> and <em>lemmatized </em>(i.e., they are short of grammatical suffixes).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*uRc2QqIH-z4N145k9hrLyA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*PunC8EqgaRJzp9zr9geeKQ.png" /></figure><p><strong>2.</strong> <strong>Co-Occurring Networks of Topic Modeling Terms (TM)</strong></p><p>Below is the graph of the 31 top TM terms. These terms are the nodes of a connected graph and they are linked together through 149 edges (co-occurrences among terms). The width of a link between two nodes is variable and it corresponds to the counted number of co-occurrences of a pair of terms in the text of the Learning Objectives. The diameter of the circle of a node is proportional to the degree of that node (which is the total number of textual co-occurrences of the corresponding term). Notice that central nodes (terms) are placed in the core (the center) of this graph, while the less frequently observed terms are positioned in the periphery of the graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*KnYNe_63SToj4WWoZJCK8g.png" /></figure><p>Next, by grouping TM terms in each one of the Core STS courses, one can construct the graph of courses sharing common TM terms in their Learning Objectives so that one might be able to understand which courses are close to which other ones. Thus, the graph of courses sharing common TM terms is a weighted connected graph consisting of 15 nodes (courses) and 81 edges (pairs of courses sharing at least one TM term, with the edge weight being equal to the total number of shared terms between pairs of courses). Notice that edge weights are indicated by the width and the color opacity of the link joining a pair of nodes (courses).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/518/1*pPeBtcwJ5Y3kjnaUDUyrXw.png" /></figure><p>3. <strong>Communities in Co-Occurring Networks of Topic Modeling Terms (TM)</strong></p><p>The following are the five “fluid”<a href="#_ftn2">[2]</a> communities of the graph of co-occurring TM terms in the Learning Objectives of the Core STS Courses:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*_t59HqrzIsSpHgYtRdF2_Q.png" /></figure><p>The corresponding partition in fluid communities of the graph of Core STS courses, which are sharing common TM terms in their Learning Objectives, gives rise to the following three communities:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*M27dFYuPIArD0lQAXN6P1w.png" /></figure><p>From the above, one can interpret the structural clustering of communities as a grouping of the Core STS courses inside three blocks of courses:</p><p>· one dealing with societal issues (purple),</p><p>· another one mainly concerning scientific issues (lime) and a</p><p>· third one tackling political issues (chocolate).</p><p>Notice that the strongly interdisciplinary character of the course “Urban Violence: The Middle East” in the description of its Learning Objectives tends to bring it into the block of courses focusing on scientific issues.</p><p><a href="#_ftnref1">[1]</a> Natural Language Processing.</p><p><a href="#_ftnref2">[2]</a> The name comes from the application of the so-called “fluid communities” algorithm of F. Parés et al. (2017).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cd283ee78669" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Networks of Co-Occurring Proper Nouns in Anne Frank’s Diary]]></title>
            <link>https://medium.com/@mosabou/the-networks-of-co-occurring-proper-nouns-in-anne-franks-diary-f2ef42779d41?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/f2ef42779d41</guid>
            <category><![CDATA[network-science]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[digital-humanities]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Tue, 03 Mar 2020 19:15:55 GMT</pubDate>
            <atom:updated>2020-03-03T19:37:56.250Z</atom:updated>
            <content:encoded><![CDATA[<p>This is the draft of a report on the work I am doing this period with Mark McGown.</p><p><strong>Introduction</strong></p><p>Anne Frank’s diary is the perspective of a world shifting into conflict during World War II from the eyes of a 13-year-old Jewish girl. It is as compelling as a first-hand account of a race of people facing extermination as it is human in the innocence of the narrator. Rarely in modern history does a first-hand account of so many disparate communities, sentiment, and political forces emerge to tell a story of conflict, heroism, and tragedy. The contents of this diary blend both the day-to-day interests of a German adolescent hiding in Amsterdam with the unfolding events of a German-occupied Netherlands. Through this lens, the context of how each concept relates to one another from Anne’s perspective can be known to the reader.</p><p>The purpose of this project was to glean further insights into the context of this period in Anne Frank’s life and, through this process, develop the tools necessary to extract similar information from other texts. The NetworkX library for Python was used extensively in this project to visualize and investigate the relationships of co-occurrent proper nouns as well as the sentiment between them. Tags were placed on each of these terms regarding whether they were people, organizations, products, or nationalities. Community detection methods were employed to group these terms into both clusters of their community as well as their tag type. Finally, the metrics of these graphs in addition to metrics on the community grouping methods are presented.</p><p><strong>Historical Context</strong></p><p>Anne was born in Frankfurt, Germany on June 12, 1929, to parents Otto &amp; Edith and sister Margot while Jewish persecution was growing. Shortly after the first concentration camp was opened in 1933, Anne’s family fled to Holland to escape the growing anti-Jewish sentiment. On May 10th, 1940, Germany invaded the Netherlands. Two years later, Anne received her diary just months before her older sister Margot was called upon to serve in a labor camp. The next day, July 5th, 1942, ­­the Frank family fled to a secret Annex above Otto Frank’s offices at Opekta (a pectin and spice company), hidden by a bookcase that opened.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/385/1*hvPCYkuDhbQBA-y1TK39qA.png" /><figcaption><em>Figure A1: Anne Frank House (Center), Netherlands</em></figcaption></figure><p>The Frank family was aided by specific members of the Opekta company: Victor Kugler who was then Opekta’s owner since Jews couldn’t own businesses, Miep Gies who had been Otto’s assistant, Johannes Kleiman who was a worker at Opekta, and Bep Voskuijl who had worked for Otto. One week after going into hiding, the Frank family was joined by the van Pels family (which Anne gave the pseudonym of van Daan) who had also gone into hiding after being summoned for forced labor in the concentration camps. The van Pels family consisted of Peter, 15, who was the only child of Auguste (pseudonym Petronella) and Hermann. Peter and Anne had a brief but meaningful relationship heavily reflected in Anne’s diary over her time in the Annex. The final addition to the Annex was dentist Fritz Pfeffer (pseudonym Albert Dussel) on November 16th, 1942.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/1*z-Q-uSmo8alE1XQbWP_YxQ.png" /><figcaption><em>Figure A2: The Secret Annex</em></figcaption></figure><p>On August 4th, 1944, an anonymous tip-off resulted in a raid by police that inevitably arrested all those in hiding as well as the two men who were assisting the families. The Jewish families tragically did not survive the concentration camps except for Otto Frank who went on to remarry and publicize the contents of Anne’s diary given to him by Miep. Johannes was released from Amersfoort camp after just one week due to being in ill-health and returned to run Opekta. Victor escaped en route to a labor camp during Allied bombing and returned to business in Amsterdam followed by Canada. Today, the Anne Frank house has been restored as a biographical museum devoted to telling Anne’s story as well as educating about forms of discrimination.</p><p><strong>Why Network Analysis Matters</strong></p><p>Network analysis is necessary to analyze text such as Anne Frank’s diary since it reduces room for interpretation of Anne’s experiences during a critical time in human history. Network visualizations and metrics are important since they can take important texts in history and reduce ambiguity. For example, the prejudices that were facing the Frank family may not only repeat themselves politically but may retain auto-biographical signatures of someone experiencing prejudices in social media today. Refining network analysis tools in many ways serves to correlate metrics and sentiment output with what the author truly felt at the time the work was created.</p><p>The metrics at the end of this project will illustrate the primary concerns of Anne during her time in hiding, what concepts and people mattered to her most. Certain graphs will show the associations of those important to Anne’s survival, Opekta workers, and other associations will reflect more closely with a diary of a more typical 13-year-old girl, specifically Peter. Anne’s views on the negative sentiment between Christian &amp; Jew nodes are not unlike detecting co-occurrence of other natural language processing (NLP) use-cases on social media where enough negative sentiment should be, especially if geographically centered, cause for humanitarian aid.</p><p><strong>Data Analysis</strong></p><p>Figure B1 below shows how the different NLP tag counts changed throughout the diary entries. The Person tag is consistently the most prevalent NLP entity whereas Product is the most uncommon. Certain periods, such as the invasion of Sicily on July 10th, 1943 to September 10, 1943, when Anne gets news of Italy’s surrender, are marked by increased mention of overall tag count. The increased variance in all tags towards the final entries in the diary (and nearer the end of WWII) was also noticeable in Figure B1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*Dfm6sHlhwU4BqQhdMbFxBg.png" /><figcaption><em>Figure B1: NLP Entity Types by Date</em></figcaption></figure><p>For an additional look at the overall diary before diving into co-occurrence, Figure B2 displays a word cloud of the proper nouns for all tag types. As one might expect, the cohabitants of the Annex dominate the most prevalent terms in this figure. What also comes to light are words like Jewish, German, British, Dutch and other words that would likely fall into the NORP (nationalities or religious or political groups) NLP tag. This will be of interest later in this project when co-occurrent terms are clustered by the NLP tag type.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*3f3X15AF19u0go2XoMqUGg.png" /><figcaption><em>Figure B2: Wordcloud of Anne Frank’s Diary</em></figcaption></figure><p>Next, the overall graph was visualized for all terms shown in Figure B3 below. Here, node size represents the frequency of occurrence of each term in all diary entries and edge width represents the co-occurrence frequency of the connected terms within diary entries. Already it can be observed, like the word cloud, there are some terms that occur far more frequently than others. It can also be observed that certain terms in Figure B3 reside in certain clusters of more densely blue (higher degree) regions to other nodes in their respective cluster. One notable cluster, in the bottom right of Figure B3, contains nodes well-connected to each other but less connected to everything else. NLP as a community as well as other community types will be investigated during this project to see what terms can best be grouped together w/r/t different measures of clustering metrics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/1*39a0ZMMVITV-LVfzXEPfzw.png" /><figcaption><em>Figure B3: Overall Graph of Diary Co-Occurrence and Term Frequency</em></figcaption></figure><p>Taking a closer look at some of the most frequent terms in diary co-occurrence, Figure B4 shows a similar representation but now with labeled nodes. Due to the disparate frequency of co-occurrence, the edge width was taken as the log of the co-occurrent frequency while node size remains term frequency. Kitty, Anne’s pet to whom she addressed almost all her entries, appears as the most frequent term along with other members of the Annex and the NORP’s affecting the events surrounding the Annex. Edge connections between Jew &amp; Hitler and Jewish &amp; Margot tell a fuller story about the events detailed in Anne’s diary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*VY6H3AjiHBdz3Kr8T0gaYg.png" /><figcaption><em>Figure B4: Frequent Terms of Diary Co-Occurrence</em></figcaption></figure><p>To more clearly see how these terms relate, sentential co-occurrence was next used to specify edge connections as shown in Figure B5 below. The widths of these edges will now represent the log of co-occurrence in sentences and the color of these edges will represent the average sentiment when the connected terms co-occurred in sentences throughout the diary. The sentiment between Jew &amp; Christian is, on average, negative as is the sentiment when Jew and German co-occur in sentences. The frequency of Jew &amp; Christian co-occurrence can also be seen to be larger than most other edges involving NORP’s, comparable to the frequency of co-occurrence between Persons.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*zfF3Bn5mGDcoe7u-rWUNhw.png" /><figcaption><em>Figure B5: Sentiment of Sentential Co-Occurrence for Frequent Terms</em></figcaption></figure><p>To more clearly distinguish sentiment between NLP attributes, frequent terms were first grouped by term and the average sentiment and co-occurrence between them were visualized in Figure B6. The average sentiment between NLP attributes is generally neutral to positive, most notably so between Person &amp; Product and ORG &amp; NORP. The frequency of the people Anne lived with again dominates node size as reflected by the largest node being Person. On average, the sentential co-occurrence frequency was largest between Person &amp; ORG as reflected by the largest edge width. Here, Miep was detected as node fulfilling ORG rather than a Person using the Python library Spacy and its largest available ML trained library. This may in part be due to her diligent services to supply the Annex occupants with resources, “Miep is just like a pack mule, she fetches and carries so much. Almost every day she manages to get hold of some vegetables for us brings everything in shopping bags on her bicycle.” Annex is the node tagged as Product by Spacy since it is closest to the library’s definition of “objects, vehicles, foods, etc.” and does not fulfill a service that is often mentioned in the diary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/489/1*3GesYjbmldQIDd_i3JDldQ.png" /><figcaption><em>Figure B6: Sentiment of Sentential Co-Occurrence by NLP Attribute (Summarization)</em></figcaption></figure><p>A separate method of clustering was used using the Louvain Method (LM). The goal of this was to see what this picture would look like when the density of edges inside LM communities was maximized and the density of edges outside of these LM communities was minimized. Although clustering by NLP Attribute relies on a well-establish ML library in Spacy, it relies on text-based training that may be dissimilar from Anne Frank’s Diary and, also, does not factor in the co-occurrence of frequent terms. Figure B7 shows the Louvain communities with edge density as a factor. Figure B8 shows the membership of these communities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*ubNIwmDvT6kVyphAahAX6A.png" /><figcaption><em>Figure B7: Sentiment of Sentential Co-Occurrence by Louvain Community (Summarization)</em></figcaption></figure><p>Louvain Community 0 shows terms that were previously seen to be those both living in the Annex and frequently mentioned by Anne in her day-to-day life. These are well-connected nodes that are often centered near each other in the network graphs. Louvain Community 1 shows mostly what the Spacy referred to as NORP’s with the addition of Kitty. Louvain Community 2 shows those who lived in the Annex who were mentioned less often due to Anne’s relationship with them or, in the case of van D. &amp; Frank, the fact they were only mentioned formally when referring to a family. The membership of Figure B8 shows that Figure B7’s positive sentiment between Louvain Community 0 &amp; Louvain Community 1 such that the well-connected members of the Annex have positive sentiment between religions/nationalities. It also shows that their sentential co-occurrence, via edge width, is also the lowest. The average node size in Louvain Community 1 was shown as the largest in this case.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/1*ZGspJ_zuweiafxsG-llJNw.png" /><figcaption><em>Figure B8: Louvain Community Membership</em></figcaption></figure><p>To further inspect the interplay of sentiment between nodes in clusters, Figure B9 shows the graph of frequent sentential co-occurring terms clustered by the NLP attribute. More resolution is given to the sentiment between Person &amp; Product (the Annex) and why the summarization showed overall positive sentiment between the two. Kitty, the van D. family, and Bep can be seen to influence the sentiment with the Annex. The relationship between ORG &amp; NORP is seen to be solely influenced by Miep’s positive sentiment with Dutch, “You can see we’re never far from Miep’s thoughts, since she promptly noted their names and addresses in case anything should happen and we needed contacts with good Dutch people.” In contrast with the LM community detection approach, the nodes below are colored by Fluid Community. This community detection algorithm focuses less on edge density and treats each attempted community as a fluid in a non-homogeneous environment (Parés et al, 2017). Fluid Communities had a lower assortativity versus NLP Community. Community detection method metrics will be compared as part of this project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*zpnzBv9OPeMbqoFfqz8cKQ.png" /><figcaption><em>Figure B9: Sentential Co-Occurrence for Frequent Terms by NLP Attribute (Fluid Coloring)</em></figcaption></figure><p>Figure B10 shows sentential co-occurrences clustered by NLP Attribute; however, the nodes are now colored by the Louvain Community. All the NORP’s are now grouped together except for Kitty whereas the remaining Louvain Communities differ over Product/Person/ORG. The large width of edges between Louvain Community 0 &amp; Louvain Community 2 can now be shown to be driven by the edges between Dussel &amp; van D. as well as Peter &amp; van D. Peter was, of course, of the van Pels family (aka van Daan) who Anne was both fond of and mentioned frequently. Fritz Pfeffer (aka Dussel) had a mixed relationship with the van D.’s that seems, on average, positive which may have been influenced by an affair, “Mrs. van D. sprang out of bed and went downstairs to Dussel’ s room to seek the comfort she was unable to find with her spouse.” Unlike the fluid community method, the assortativity of the LM was better than that of the NLP attribute.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*u9r-mSe0t5dgwQGP1UYiCw.png" /><figcaption><em>Figure B10: Sentential Co-Occurrence for Frequent Terms by NLP Attribute (Louvain Coloring)</em></figcaption></figure><p>Performing the same graph except clustering by Louvain Community and coloring by NLP attribute is shown in Figure B11. Van D. can be seen more clearly responsible for the higher average edge width during summarization between the two Louvain communities comprised predominantly of Annex residents. In addition to Kitty not being a NORP, the degree Kitty has with differing Louvain communities also appears to stand out.</p><p>Additional complexity was removed from the graphs by next focusing on specific nodes (ego) and only those that shared edges with that specific node (alters). Figure B12 shows the ego-centric graph of Kitty. Reducing the cut-off for node frequency revealed negative sentiment between Kitty existed due to the use of the word ‘war’ in the following sentence, “Dearest Kitty, Mr. Bolkestein, the Cabinet Minister, speaking on the Dutch broadcast from London, said that after the war a collection would be made of diaries and letters dealing with the war.” In some ways, given that this is a diary written during wartime, it might in the future be prudent to try to find words to exclude from sentiment detection -or, possibly, a library that normalizes words of extreme sentiments given their frequency somehow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/518/1*XK0ZPD1E0Z59EFZ4KiVk7w.png" /><figcaption><em>Figure B11: Sentential Co-Occurrence for Frequent Terms by LM (NLP Coloring)</em></figcaption></figure><p>Figure B13 shows the egocentric network of Anne. Anne can be seen to have a generally positive sentiment with most connecting nodes. This is most notable for Kleiman &amp; Miep to whom (like all helpers) she was very grateful for, Peter to whom she had a relationship with, Margot her sister, and the Annex that kept her safe for two years in Amsterdam. ­­­Unlike the egocentric graph of Kitty, Anne’s egocentric graph has less variance in sentiment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/1*NNX9iH5I9rH_M126KhXY4g.png" /><figcaption><em>Figure B12: Sentential Co-Occurrence of Kitty’s Egocentric Network</em></figcaption></figure><p>Figure B14 shows the egocentric graph of Annex. Unlike the other terms, Annex has edge widths to the alters that are relatively smaller than the alters have to each other. ­­This is due to Anne writing more about the occupants of the Annex than of the Annex itself. The negative sentiment between Annex &amp; van Maaren refers­­ to Wilhelm van Maaren who became warehouse manager in the spring of 1943 but was unaware of the inhabitants of the annex (though, he later grew suspicious of intruders). Anne frequently suspected Wilhelm van Maaren would betray them and posed a direct threat to the safety of the Annex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*jg_RdI-9rqDo0jP6xOAL8w.png" /><figcaption><em>Figure B13: Sentential Co-Occurrence of Anne’s Egocentric Network</em></figcaption></figure><p>The Leiden community detection algorithm was then introduced to compare against Fluid and Louvain community detection methods. The Leiden algorithm differs from Louvain by randomly moving local nodes to nearby communities in order to find the best partition. Figure B15 shows the three different community detection methods side-by-side on the sentential co-occurring frequent terms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*n-e-r6wqQVsKiDWjkPuJew.png" /><figcaption><em>Figure B14: Sentential Co-Occurrence of the Annex’s Egocentric Network</em></figcaption></figure><p>Figure B16 below shows all communities via each detection method as nodes and the similarity of terms between each community as weighted edges. Three highly weighted edges can be observed between individual Leiden and Louvain communities. Since each Louvain community has a Leiden community it directly correlates to, it can be observed that both of these detection methods obtained the same membership. Fluid Communities have very low weights across all edges and share very little membership in common with Leiden and Louvain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L3gZ05WVPcmRwnzE0YUttQ.png" /><figcaption><em>Figure B15: Different Community Detection Methods, Clustered by NLP Attribute</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*KRa3cB4NVlTEHBlzz9lL_A.png" /><figcaption><em>Figure B16: Similarity-Weighted Communities for Fluid, Louvain, &amp; Leiden Methods</em></figcaption></figure><p>The metrics of these community detection methods are shown in Figure B17 below. Performance is the number of intra-community edges plus intra-community non-edges with the total number of potential edges. Modularity compares the number of edges inside a community to that if the network were randomly distributed (a bad assumption for very large networks). Coverage is the ratio of intra-community edges to the total number of edges in the graph. Fluid detection having negative modularity means the fluid communities had fewer edges between vertices of the same community than one would expect by chance. Since Leiden &amp; Louvain communities have the same membership, all three metrics were matched between these two detection algorithms. Fluid community detection had the highest coverage of all detection methods meaning it retained the most intra-community edges versus overall edges in the graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*qxgQefGQvU_U1Fox9G6ziA.png" /><figcaption><em>Figure B17: Community Metrics of the Fluid, Louvain, &amp; Leiden Methods</em></figcaption></figure><p>Certain community detection techniques (Multilevel/Eigenvector/FastGreedy) rely primarily on modularity maximization. The issue with these methods is that they can result in unrealistically large communities since a larger community mirrors that of increased samples from a distribution. More nodes captured in a community in this maximization technique is likely to increase the similarity of a community with random edges (Wagenseller, Wang, &amp; Wu, 2018). On the other hand, it is still important for modularity to be a factor in identifying the proper community detection algorithm since each individual community should mirror randomly distributed edges. Taking the metrics together, performance evaluates edges connecting intra-community vertices as well as the edges that do/don’t exist between vertices of differing communities (the lack thereof being desirable from the algorithm). Coverage is then, put another way, the ratio of edges that remain intra-community over those that are between communities. Since the Fluid community algorithm maximized the ratio of intra-community edges but failed to detect communities that matched those of randomly distributed edges, the Louvain detection method was chosen since it is one of the standard community detection algorithms in graph theory.</p><p>To further look at how the nodes naturally cluster together, cliques were inspected. Figure B18 shows the clique membership of nodes. Nodes that seemed more central to many of the graphs, namely Margot/Peter/van D./Kleiman, all have some of the largest degree for clique membership. Most cliques contain at least three or more nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*6pi9vH8fO25YSfTVTYo_BA.png" /><figcaption><em>Figure B18: Term Membership in Cliques</em></figcaption></figure><p>Figure B19 imagines what a similar graph would look like except as the intersection of these cliques as nodes and the membership of shared terms as edges. The shared terms between cliques 7 through 17 are even now more apparent as a cluster in the bottom right of this graph. Comparing the membership of which outlier cliques contain which terms, most of the NORP’s and those who assisted the Annex reside in these satellite cliques. This makes sense since they played less of a direct role in Anne’s day-to-day life and primarily affected the conditions inside and surrounding the Annex. Certain cliques outside the aforementioned cluster are still closer than most, most notably clique 1 perhaps in part due to the membership of the term German in that clique and the more direct relevance it had to the residents of the Annex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*5pftkSwEpkc2L6ZZWVTAyw.png" /><figcaption><em>Figure B19: Intersection of Cliques</em></figcaption></figure><p>To apply this knowledge of how cliques intersect to the original graphs of terms as nodes, Figure B20 shows a graph of how each term relates to each other with the clique intersection type as edges. Here, orange nodes represent leaves, red nodes represent bridges between cliques, cyan edges represent non-intersecting clique edges, and blue edges represent intersecting clique edges. French, English, and other NORP’s that had less direct influence on the residents of the Annex remain as the leaves with less connectedness within the cliques. German, Dutch, &amp; other NORP’s + helpers that played more of a direct influence act as bridges between the cliques while retaining a relatively low degree to as many cliques. Finally, Margot, Dussel, &amp; other residents of the Annex serve as both bridges between both intersecting &amp; non-intersecting cliques as well as display a high degree of edges with many of the cliques that exist.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/520/1*GWzYHkUj96_Facm3U0Lf1w.png" /><figcaption><em>Figure B20: Clique Bridging by Term</em></figcaption></figure><p>To better compare the centrality measures of each term, Figure B21 shows the rankings of each node for the overall centrality metrics. Margot, Peter, &amp; van D. all appear at the highest rank overall. Although previous analyses of the graphs discussed nodes/cliques that had a high measure of centrality regarding closeness to other nodes, this rank factors in nodes that served important relationships such as betweenness and load. In this way, a node that wasn’t one of the closest or part of Anne’s circle in the Annex could still be ranked highly due to its overall relationship to the characters of the diary (e.g. Dutch rank is 2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*tTIrfVeUIlGHAEKRsbjjtA.png" /><figcaption><em>Figure B21: Node Rank by Measures of Centrality</em></figcaption></figure><p>This aggregate centrality ranking is reimagined in Figure B22 where the difference between nodes like Dutch &amp; Annex can more readily be observed. Dutch not only is central to the members of the Annex but serves as a bridging node to specific NORP’s. This is because this term not only served to discuss the environment around the Annex but could also be used as an adjective for things inside the Annex that, after all, were Dutch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*mNTWBfiwj7mESnFe5d7eNQ.png" /><figcaption><em>Figure B22: Node Rank by Measures of Centrality</em></figcaption></figure><p>Figure B23 below shows the scatter matrix of the centrality rankings per node. The horizontal line across the matrix where nodes mix with themselves shows the distribution of rankings. Note how Kitty’s cell ­­­for this distribution is highly skewed to the left. This is because many of Kitty’s centrality rankings were very low (high measures of centrality). In the diary, Anne imagined Kitty as her best friend in lieu of having friends outside the Annex making Kitty central to the text in different metrics. Looking at the scatter matrix row for Anne &amp; Kitty or Christian &amp; Jew or German &amp; Dutch show some degree of linearity where the nodes are related in their centrality indexes. Previous graphs involving strong sentiment and high co-occurrence have given other aspects to the story behind the centrality metrics of these pairs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/1*eId2VuPSPbYRC9EM0kXeOA.png" /><figcaption><em>Figure B23: Scatter Matrix of Centrality Indices with Pearson Correlation Coefficients</em></figcaption></figure><p>One final figure to discuss centrality measure is Figure B24 showing the Katz centrality of each term. Here, node size and darkness represent terms with higher measures of Katz centrality. Katz centrality measures the number of neighbors to a node as well as distant neighbors via these neighbors. Distant neighbors are penalized w/r/t the Katz centrality metric such that the Katz centrality’s usefulness stems from its use to gauge the nearness of neighbors to a node. Van D., Anne, &amp; Peter all perform highly for this metric as one might expect. One node to notice that performed highly for this metric, interestingly, is Kleiman. Johannes Kleiman served as more than just an assistant to fetch supplies to the Annex, “When Mr. Kleiman enters the room, the sun begins to shine!” Kleiman might, in part, have a high Katz centrality measure due to becoming known to make cheering everyone up as his goal. With these actions and Anne noting them, the Kleiman node interacts with more neighbors than other assistants or NORP nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*3YCGgbED2kCqPkoK8MP-Gg.png" /><figcaption><em>Figure B24: Katz Centrality by Node</em></figcaption></figure><p>The density of a graph is also an interesting network statistic to understand how well-connected each node is with each other. The density of this fully connected graph involving the terms for sentential co-occurrence was 0.257, meaning about 26% of all edges that could have existed for a complete graph were present. This number does not factor in the weighting of edges. Weighted density is used to asses how much of the total graph was connected by edges up to the max weight that existed between any two nodes. The weighted density was 0.00243 which makes sense when considering the edge widths of the graphs in this project had to be displayed logarithmically due to the disparity in the number of co-occurrences between terms.</p><p>The graph was not found to be scale-free meaning that the distribution of degrees was not found to follow a power-law. Figure B25 shows the distribution of node degree as a degree histogram. If this graph followed a power-law distribution, the number of nodes would primarily decrease with an increasing degree. Instead, there are a significant number of nodes with higher degrees more like that of a random network. The same is true for the weighted degree distribution of G as shown in Figure B26.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/514/1*exJWsR-j3gGIdVK-hcV7Qw.png" /><figcaption><em>Figure B25: Degree Distribution of G</em></figcaption></figure><p>As a measure of how interconnected a graph is, network statistics typically refer to transitivity to measure the number of triangles in a graph or how close every node is to being part of a clique. The transitivity measure for this graph was 0.592 which implies this graph is neither fully interconnected nor relatively disconnected. The clustering coefficient is another measure of how close a node is to being surrounded by cliques. Figure B27 shows the weighted clustering coefficients for this graph. Note how the tertiary nodes that were only connected to the graph with one edge, like French, have a coefficient of 0 since they have no surrounding clique or interconnectedness. Dussel &amp; Boche­ have the highest clustering coefficients.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*Ju0IpU2YqkiVcYrBQVBNZA.png" /><figcaption><em>Figure B26: Weighted Degree Distribution of G</em></figcaption></figure><p>The Boche node experiences this because that node has only two edges that are involved in a clique, all with relatively high weight. The Dussel node, on the other hand, has a high degree of edges and all collectively have a relatively high weight on average. The average weighted clustering of all nodes below is 0.059.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/452/1*d1gXQIDROcoMY0LV0GmGGw.png" /><figcaption><em>Figure B27: Weighted Clustering Coefficients by Node</em></figcaption></figure><p>Another way to assess the interconnectedness of a graph is to calculate its diameter. Diameter is calculated by first finding the shortest paths between each pair of vertices. Diameter is finally the longest of any of these shortest paths. The diameter of this graph was 4. For reference in terms of the shortest paths used to find this figure, the average shortest path was calculated to be 2.08, far lower than the diameter. The weighted average shortest path was calculated to be 4.22, where the shortest path is then the path between any two vertices with minimum sum of edge weights along that path.</p><p>The last metric investigated for this graph was assortativity. Assortativity is the preference or likelihood for nodes to be attached to other like-nodes based upon a certain attribute. The degree assortativity of this graph, or the likelihood of nodes to be connected to other like-degree nodes, was -0.15. A negative assortativity value implies disassortativity based upon that attribute. Nodes with higher degree were more likely for this graph to be connected with nodes of lower degree and vice versa. The attribute assortativity coefficient using Louvain communities (rather than degree) is 0.33, showing intra-community nodes were more likely to have edges with other nodes in their respective community.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/1*8dEKWNHaOT8zuO8iIi0kGA.png" /><figcaption><em>Figure B28: Mixing Matrix by NLP Tag</em></figcaption></figure><p>The mixing matrix of the graph shows that most edges of the NLP tag Person and NORP are with other nodes of the same type. Product &amp; ORG, however, mix with nodes of other types. Aside from edges from Person to the same type, as was seen in this project, the Person most frequently connected to NORP followed by Product (Annex).</p><p><strong>Conclusions</strong></p><p>The findings of this network analysis reveal the interplay of characters &amp; relationships in Anne’s diary as well as illustrate the overall graph structure brought into focus by NetworkX. A surprising amount of overlap existed between the known personality of members of the Annex and their centrality metrics e.g. Kleiman &amp; Katz centrality. Sentiment in sentential co-occurrence between NLP types was also particularly revealing with respect to the ultimate themes behind WWII. Most notably, the iconic negative sentiment between Christian &amp; Jew underscored the influences behind other relationships such as the Frank family hiding in exile due to Margot’s call-up for forced labor. Maximizing the metric scores of community detection methods to choose the Louvain detection algorithm revealed the dynamic between the separate communities of the members of the Annex, the Annex’s assistants, and the nationalities surrounding the Annex. Proper nouns such as Dutch played a pivotal role in Anne’s diary, as shown by the aggregate centrality rank, not just as a nationality but also as a descriptor of daily life &amp; adjective for objects inside the Annex.</p><p>The overall graph was not found to be scale-free like many real-world networks. Partly this may be due to a lack of time duration since the network only grew over the course of the 2 years it was written. It also possible that this network is unlike scale-free social networks due to not only their growth over time but also the evolution of a network created from multiple authors rather than just one. Further research should be conducted regarding how quickly networks grow to be scale-free with the number of co-authors as a variable input (given a constant time duration for comparison purposes).</p><p>As the topic of network analysis pertains to text analysis, the strong alignment regarding the context of the diary and, separately, the findings of this analysis suggest network analysis is an adept tool at aggregating meaning from text documents. Finally, as the context of this project pertains, network analysis to detect authors that may be facing oppression or signs of duress should continue as a legitimate form of early, preventative detection. By comparing test cases of diaries, social media, and other forms of print, it is possible that certain aggregate patterns regarding sentiment summarization between Person &amp; NORP might exist (or the assortativity by community attribute therein).</p><p><strong>References</strong></p><p>Parés, F., Gasulla, D. G., Vilalta, A., Moreno, J., Ayguadé, E., Labarta, J., Suzumura, T. (2017). Fluid Communities: A Competitive, Scalable and Diverse Community Detection Algorithm. Studies in Computational Intelligence Complex Networks &amp; Their Applications VI, 229–240. doi: 10.1007/978–3–319–72150–7_19</p><p>Wagenseller, P., Wang, F., &amp; Wu, W. (2018). Size Matters: A Comparative Analysis of Community Detection Algorithms. IEEE Transactions on Computational Social Systems, 5(4), 951–960. doi: 10.1109/tcss.2018.2875626</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f2ef42779d41" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mapping the Networks of Voltaire’s Philosophical Dictionary]]></title>
            <link>https://medium.com/@mosabou/mapping-the-networks-of-voltaires-philosophical-dictionary-e7d1ba865287?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7d1ba865287</guid>
            <category><![CDATA[social-network-analysis]]></category>
            <category><![CDATA[voltaire]]></category>
            <category><![CDATA[digital-humanities]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Sun, 08 Dec 2019 18:39:43 GMT</pubDate>
            <atom:updated>2019-12-08T18:39:43.181Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/304/1*4A81SRbVSf-PZxJPhyfM0g.png" /></figure><p>During the 2019 Fall semester, I’ve been working this project with three students <em>Sashank Silwal, Yashaswi Malla </em>and<em> Hamad Bin Fahad</em> in my Networks course at New York University Abu Dhabi. The text data of our project was the famous <strong><em>Philosophical Dictionary</em></strong> (<strong><em>Dictionnaire philosophique</em></strong>),<strong><em> </em></strong>an encyclopedic dictionary originally published by <strong>Voltaire </strong>in 1764. Here, we were based on the version produced by Juliet Sutherland, Lisa Reigel released in 2006 and provided online by <a href="https://www.gutenberg.org/ebooks/18569">Project Gutenberg</a>.</p><p><strong>I. The directed graph of references among lexical items of Voltaire’s Philosophical Dictionary</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/268/1*6S9lkqv4ivDuQYc1YmwT5A.png" /></figure><p>Voltaire’s Philosophical Dictionary (<strong>VPD</strong>, from now on) consists of 99 <strong>lexical items</strong>, each one referencing some other lexical items (or none), thus, producing a <em>directed graph</em> consisting of 99 nodes (lexical items) and 1136 edges (references among lexical items). This graph is not strongly connected and it has 21 strongly connected components. However, the VPD graph is weakly connected graph and hence, it has no isolates. The density, transitivity and reciprocity of the VPD graph are 0.117, 0.322 and 0.146, respectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*52iDlzrZ6SRYyOuo5Oayfw.png" /></figure><p>The largest strongly connected component of the VPD graph has 79 nodes (lexical items) and 930 edges (references among lexical items). Its density, transitivity and reciprocity are 0.151, 0.324 and 0.178, respectively, while its diameter is equal to 5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x0tLQjxhwKGqnGiGFOzWUg.png" /></figure><p><strong>II. The undirected graph of co-referenced and co-referencing lexical items of Voltaire’s Philosophical Dictionary</strong></p><p>The VPD graph is a directed graph similar to the <strong>citation</strong> graph in bibliometrics. Thus, as one typically derives from a citation graph the two undirected graphs of <strong>co-citation</strong> and <strong>bibliographic coupling</strong>, similarly, we are going to get two undirected graphs from the VPD graph of references among lexical items: the co-referenced graph of lexical items and the co-referencing graph of referencing lexical items.</p><p>The <strong>co-referenced graph of lexical items</strong> is a weighted undirected graph of lexical items, in which two lexical items are adjacent as far as in there exist other lexical items referencing them both in the VPD graph (the link weight being the number of such referencing items). Thus, the VPD co-referenced graph contains 82 nodes (lexical items) and 1865 edges (in-going co-references). Its density, normalized density and transitivity are 2.409, 0.055 and 0.752, respectively. Below is a plot of this graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QlVuwGqBX4KkQEyqsX35kw.png" /></figure><p>Moreover, the co-referenced graph of lexical items is partitioned in 3 <strong>communities</strong>, displayed in the following plot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XNKEguV8cHh6Yu5aqawwgA.png" /></figure><p>In addition the <strong>co-referencing graph of lexical items</strong> is a weighted undirected graph of lexical items, in which two lexical items are adjacent as far as they are both referencing other lexical items in the VPD graph (the link weight being the number of such referenced items). Thus, the VPD co-referencing graph contains 99 nodes (lexical items) and 4442 edges (out-going co-references). Its density, normalized density and transitivity are 3.388, 0.161 and 0.947, respectively. Below is a plot of this graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5reAE0ZA7gpxISPqSzx0EQ.png" /></figure><p>The 3 communities of the co-referencing graph of lexical items are showing in the following plot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zauycrigNKlqF1yct-ZsAg.png" /></figure><p><strong>III. The vocabulary of words in Voltaire’s Philosophical Dictionary</strong></p><p>Besides the above 99 lexical items, using standard NLP techniques of POS tagging, we were able to locate two more categories of words in VPD: names of characters and geographical locations. Thus, the total VPD vocabulary words (which will be mentioned here as “dictionary terms”) included:</p><ul><li>370 names of <strong>character names</strong>,</li><li>242 <strong>locations</strong> and</li><li>99 <strong>lexical items</strong> (or <strong>dictionary terms</strong>).</li></ul><p>Here are the top character names with their frequencies of occurrences in VDP and the wordcloud that they altogether form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/329/1*7TfewbVLHWXu7H2aOSmAGQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*95JVY9NHSfLSi3r1Ua7Xbw.png" /></figure><p>Next are the top locations with their frequencies of occurrences in VDP and the wordcloud that they altogether form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/318/1*Uf6BlDxdLTGWWTaCylpI-g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j7Llu2OWcVrmKkSx2MQobA.png" /></figure><p>Finally, these are the top dictionary terms (lexical items) with their frequencies of occurrences in VDP and the wordcloud that they altogether form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/310/1*RCQfXAtUMA5LPEcPtOCGhQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JzJuPUCj2APd65ve7_YFmw.png" /></figure><p><strong>IV. The graph of co-occurring dictionary terms in the content of items of Voltaire’s Philosophical Dictionary</strong></p><p>In this part of our work, we were interested in constructing the <strong>graph of co-occurring dictionary terms</strong> in VPD. Thus, we found a connected weighted undirected graph of 696 nodes (the dictionary terms) and 25561 edges (the co-occurrences of pairs of dictionary terms in the content of items of VPD). The nodes of this graph were: 370 names of characters, 242 locations and 84 dictionary terms. What follows is a first plot of this graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VN-hfg3kaOM79zboGd-04g.png" /></figure><p>The following is a second visualization of the graph of co-occurring dictionary terms of VPD, in which nodes of the same type are positioned together in different concentric areas:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HKenN16z0gArKR1xuyRiRg.png" /></figure><p>In particular, the subgraph of co-occurring character names in VPD contains 370 nodes (characters) and 3916 edges (co-occurrences of pairs of characters in the content of items of VPD). This graph is disconnected and it includes 11 connected components. The largest connected component, which contains 352 nodes (characters) and 3905 edges (co-occurrences), is plotted below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HPSqE1XDqSw1m8fQpjs9MA.png" /></figure><p>In addition, the subgraph of co-occurring locations in VPD contains 242 nodes (locations) and 2766 edges (co-occurrences of pairs of locations in the content of items of VPD). This graph is disconnected and it includes 5 connected components. A plot of the largest connected component, which contains 238 nodes (locations) and 2766 edges (co-occurrences), is shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gAZwqiL4ATjc0k1wS1_ECw.png" /></figure><p>Moreover, the subgraph of co-occurring dictionary terms in VPD contains 84 nodes (terms or lexical items) and 1858 edges (co-occurrences of pairs of terms in the content of items of VPD). This graph is connected and it is plotted below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NHG9rCTSib6y4nIMn5DwEA.png" /></figure><p>Finally, the subgraph of co-occurring dictionary terms in VPD contains 4 communities of dictionary terms, which are shown in the next plot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LAybtGLUC9dSs5B7ueiK1g.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7d1ba865287" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Networks of Co-Occurring Celebrities in The Andy Warhol’s Diaries edited by Pat Hackett]]></title>
            <link>https://medium.com/@mosabou/the-networks-of-co-occurring-celebrities-in-the-andy-warhols-diaries-edited-by-pat-hackett-b5aa08461e1a?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5aa08461e1a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[art]]></category>
            <category><![CDATA[network-analysis]]></category>
            <category><![CDATA[digital-humanities]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Tue, 03 Dec 2019 18:21:52 GMT</pubDate>
            <atom:updated>2019-12-03T18:21:52.173Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/435/1*QN6_aNj-6WENw5PPnqe6FQ.jpeg" /></figure><p>Last summer, even before my arrival at the New York University Abu Dhabi, <a href="https://nyuad.nyu.edu/en/academics/divisions/arts-and-humanities/faculty/bryan-waterman.html">Brian Waterman</a> was one of the first colleagues that I had the pleasure and honor to meet and also to interact with (in the context of his role as Associate Vice Provost for Undergraduate Academic Development and my involvement in teaching the Networks course in the Core Curriculum).Talking about Digital Humanities and Networks provided the opportunity to Bryan to pitch me into the idea of working with a text-data-network analysis of the volume “The Andy Warhol Diaries” that has been edited by Pat Hackett. Subsequently, during the current Fall semester at NYUAD, one of the projects that I have given to students (in the Networks course that I was teaching) was on doing data and network analysis on the text of this volume. Thus, I am presenting here a few first findings of our project on Warhol’s diaries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/267/1*Lq-YEINcqN7GX8J84_ptLA.jpeg" /></figure><p>Let me start mentioning that the text of “The Andy Warhol Diaries” includes 450333 words and 2401603 characters/symbols. It contains 2024 Diaries from November 24, 1976, to February 17, 1987. In each of these 12 years, there appears the following distribution of numbers of diaries (each diary unfolding on a specific date):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/163/1*4sHP9hjUto6hvigoqORCKg.png" /></figure><p>Using the standard Natural Languages Processing technique of POS tagging for name-recognition in textual corpora, together with my students Nas Gopee, Rayann Cherrat and Ons Taktak, we were able to detect automatically 2423 names of celebrities (or other persons) included in all the diaries. For the sake of brevity, these names are just going to be called “characters” from now on. Most of them were 2-grams (first name, surname) but some of them were 3-grams too(either with a middle name or a composite surname). The top 12 (in frequency of occurrence in the diaries) are the following characters:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*TFT7VDHO32dT2ilAAax5dw.png" /></figure><p>All of these characters generate the following wordcloud:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*Wm9vHe3jBeREAvkz4V3Z4g.png" /></figure><p>However, in each diary there may appear one or more characters (or none). When <em>k</em> characters appear in a diary, from the relational (network or graph-theoretic point of view) they generate a <em>k</em>-<strong>clique </strong>of characters (corresponding to that diary). Of course, different cliques of characters generated by different diaries may intersect (or overlap) in subcliques of various sizes. Thus, getting all possibly existing cliques of characters in all diaries and superimposing them, what one finds is the <strong>(multi)graph of co-occurring characters</strong> in the corpora of the diaries. This comes from the fact that in any nontrivial clique (of size at least two, i.e., implicating at least two characters), every pair of characters adds up to a link (or tie) between the two characters (i.e., nodes of the graph or actors of the network) which co-occur in at least one diary. In particular, sorting co-occurrences of characters in the diaries, one obtains the following 12 co-occurrences:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/1*2LTMj6ohHvcva7oYKlfbVQ.png" /></figure><p>Moreover, as every co-occurrence implicates the relation between two characters, each character (node in the graph of co-occurring characters) acquires a degree (which amounts to the number of co-occurrences with other character that the former is involved). Thus, sorting characters according to their degrees in the co-occurrent graph, the following wordcloud emerges:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*MA4nZNtMYakyXSsm_1VAnQ.png" /></figure><p>In any case, the total graph of co-occurrent characters involves 2345 nodes (characters), i.e., the 97% of all characters in the diaries, along 21123 (multiple) edges (co-occurrences), thus, forming a sufficiently large (connected weighted undirected) graph which is very hard to be visualized in such a way that the labels of nodes (names of characters) would be legible. This is why, instead of the global graph, I am displaying below four ego-centric subraphs around four important figures in Andy Warhol’s diaries: Thomas Ammann, Mick Jagger, Bianca Jagger and Truman Capote.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*srxkFveBb_wCYLTUimzJng.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*StOFiMeErFq6Z0ksrv-X2g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B1_ppdd61eN0_o9rD4nhHw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_Osx8DJCd5Neon4Shb5yuw.png" /></figure><p>Notice that the graph of co-occurring characters in Andy Warhol’s Diaries is a temporal graph (or longitudinal network) as every co-occurrence arises in a number of diaries, meaning that it unfold in particular time periods. For example, the co-occurring graphs for 1976 and 1987 implicate the characters in the following two plots:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WtZ3tui6nBNNafXU8Ju-ww.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nr5KfvLfEQPdIqAZ9SGNiw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5aa08461e1a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Network of Climate-Weather related Wikipedia pages]]></title>
            <link>https://medium.com/@mosabou/the-network-of-climate-weather-related-wikipedia-pages-bc84acfde476?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc84acfde476</guid>
            <category><![CDATA[network-science]]></category>
            <category><![CDATA[wikipedia]]></category>
            <category><![CDATA[climate-change]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Sun, 24 Nov 2019 13:19:44 GMT</pubDate>
            <atom:updated>2019-11-24T13:26:49.982Z</atom:updated>
            <content:encoded><![CDATA[<p>In one of the projects at the Networks course that I was teaching at New York University Abu Dhabi in Fall 2019, my students have explored the network of Climate-Weather related Wikipedia pages.</p><p>Starting from the following three Wikipedia pages:</p><ul><li><a href="https://en.wikipedia.org/wiki/Global_warming">https://en.wikipedia.org/wiki/Global_warming</a></li><li><a href="https://en.wikipedia.org/wiki/Carbon_dioxide#cite_note-71">https://en.wikipedia.org/wiki/Carbon_dioxide</a></li><li><a href="https://en.wikipedia.org/wiki/Weather">https://en.wikipedia.org/wiki/Weather</a></li></ul><p>and using the Python module <em>wikipedia 1.4.0</em>, we have collected Wikipedia pages originating from the above three Climate-Weather Wikipedia pages:</p><ul><li>495 pages from ‘Global Warming’ (redirected from ‘Climate Change’),</li><li>1140 pages from ‘Carbon Dioxide’, and</li><li>290 pages from ‘Weather’.</li></ul><p>The number of common pages hyperlinked by ‘Global Warming’ and ‘Weather’ was 41. The number of common pages hyperlinked by ‘Carbon dioxide’ and ‘Weather’ was 24. The number of common pages hyperlinked by ‘Global warming’, ‘Carbon dioxide’ and ‘Weather’ was 16. The number of common pages hyperlinked by ‘Global Warming’ and ‘Carbon dioxide’ was 207.</p><p>Moreover, there exist 179,104 hyperlinks among the pages starting from all the Climate-Weather originating Wikipedia pages. Therefore, this network has 1,669 nodes (webpages) and 179,104 edges (hyperlinks).</p><p>The Climate-Weather graph is a simple, unweighted directed graph (without any isolates), it is not strongly connected, it has 32 strongly connected components, and it is weakly connected. Its density is 0.064, its transitivity is 0.543, and its reciprocity is 0.615. What the three last numbers indicate is that the Climate-Weather graph is a sparse graph, with a moderate number of triangles, and rather high percentage of reciprocating links.</p><p>The following is a plot of the Climate-Weather graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nEmxl3Qbd5Aymq6B" /></figure><p>The largest strongly connected component of the Global Warming &amp; Weather graph has 1637 nodes and 177095 edges. Its density is 0.066, its transitivity is 0.551, and its reciprocity is 0.622. The percentage of pages from the whole graph included in the largest strongly connected component is 0.981, and the percentage of links from the whole graph included in the largest strongly connected component is 0.989. This means that the biggest part of the Global Warming &amp; Weather graph is included in its largest connected component. The diameter of this graph is 6. This means that the maximum geodesic distance between any two webpages in the graph is 6.</p><p>The following is the plot of the largest strongly connected component of the Global Warming and Weather graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kKmqwDC0qopu98wE" /></figure><p>Coloring nodes (pages) according to where they originate from, we get the following four categories:</p><ul><li>Pages hyperlinked only from the “Global Warming” starting page (dark sea green)</li><li>Pages hyperlinked only from the “Carbon Dioxide” starting page (salmon)</li><li>Pages hyperlinked only from the “Weather” starting page (pink)</li><li>Pages hyperlinked by at least two starting pages (pale turquoise)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MKqOJD-XL1oKPqmq" /></figure><p>The assortativity coefficient of the Climate-Weather network is equal to 0.689743, with regards to the attribute of the origin of nodes (pages) from the above four categories. Apparently, the assortativity coefficient is sufficiently high meaning that pages from a category tend to be hyperlinked to other pages from the same category.</p><p>If we group the four categories of nodes (pages) inside separate blocks, the plot of the Climate-Weather network is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*l7-upwDa0_TDfcmc" /></figure><p>Finally, we are considering four separate egocentric subnetworks of Climate-Weather around geothermal, acid, bacteria, and volcano. Notice that the color of nodes (pages) corresponds to one of the above four categories of origin and the size is proportional to the in-degree of the node (page).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R4-K-9WA85Aj7pBs" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DI_aJFHEGUPFDTjR" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*erN6GZYfsQX2p9lD" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TzoCLO-liKVTM2d7" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc84acfde476" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The graph of Raymond Williams’ “Keywords”]]></title>
            <link>https://medium.com/@mosabou/the-graph-of-raymond-williams-keywords-de7bb0e0a9f8?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/de7bb0e0a9f8</guid>
            <category><![CDATA[wordnet]]></category>
            <category><![CDATA[graph-theory]]></category>
            <category><![CDATA[social-network-analysis]]></category>
            <category><![CDATA[t]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Thu, 13 Sep 2018 07:43:42 GMT</pubDate>
            <atom:updated>2018-09-13T19:31:13.798Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*aOEfp0ilsGdlZXbx4_Q_7A.jpeg" /></figure><p>The 1983 revised edition of Raymond Williams “Keywords: A Vocabulary of Culture and Society” (published by Fontana, while the first edition was published in 1976 by Croom Helm) lists 131 familiar but widely contested words in Culture and Society. Examples of these words are: <em>Aesthetic, Bourgeois, Culture, Hegemony, Isms, Organic, Romantic, Status, Violence, Work, Anarchism, Ecology, Liberation</em> and <em>Sex</em>. Each of the words is commented in short essays and notes, in which Williams’ main intention was not so much to cast the etymological or historical origins of these keywords, but to understand how society gives them new meanings and how such semantic transformations are shaped by various political and cultural societal processes.</p><p>My aim here is to extract and analyze the network of the 131 Williams’ keywords. Since, in Williams’ short essays (and notes) following each of these keywords, there are also other keywords which are mentioned (and thoroughly discussed in their own short essays), clearly, the resulting formal (mathematical) representation is that of a directed graph composed of 131 nodes (i.e., Williams’ keywords). Whenever in the short essay of keyword A, keyword B is also <em>mentioned</em> or <em>cited</em> (internally to Williams’ book), a link directed from A to B is added in the formed graph. For obvious reasons, I will be referring to such links (among nodes of the graph) as <strong>mentions</strong> or <strong>internal citations </strong>(among keywords). As a matter of fact, Williams’ book contains 2146 such mentions (among 131 keywords) as the edges of the keyword-net of the book.</p><p>Notice that all the computations that I have done were implemented in an <a href="https://zeppelin.apache.org/">Apache Zeppelin Notebook</a>, which may produce fully interactive dynamic visualizations (in javascript). However, what follows below is a fixed (static) screen capture of such a visualization of Williams’ keyword-net:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ya_G5mW_SYSJ5-sCM7d09Q.png" /></figure><p>To display an example of a small part of this network (what in <em>Graph Theory</em> is called an “<strong>induced subgraph</strong>” of a certain number of nodes), let us consider the following STS-related 10 keywords [<em>Anthropology, Consumer, Ecology, Media, Nature, Philosophy, Science, Society, Sociology, Technology</em>] and see how they are connected in their subgraph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*MyK-TvV7KB39SuGUijnsOQ.png" /></figure><p>The density of Williams’ keyword-net is 0.126, while the following table shows the in-degrees and out-degrees of the top 12 keywords (ordered by in_degree):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HFnqskzdcWYLXremgTx8aA.png" /></figure><p>The following plot is the scatter plot of in- and out-degrees is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GZmpBYSvQuDVdV2CK-Tqxg.png" /></figure><p>Moreover, the histograms of in- and out-degrees are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eVjGrepHaLg21I7Xic1yxQ.png" /></figure><p>Furthermore, five main <strong>centrality coefficients</strong> of the top 12 keywords (ordered according to their <em>PageRank</em>) are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XPoQ_lQC9gUh7fkt6vLvsw.png" /></figure><p>At this point, let us notice that the Williams keyword-net is <strong>weakly connected</strong>, though it is neither <strong>strongly connected</strong> nor acyclic. Saying that it is weakly connected means that any two nodes (keywords) are connected to each other through a chain of links (consecutive keyword mentions among/inside the corresponding explaining essays for each keyword), all of which do not necessarily point at the same direction (if that was the case, then the graph would have been called strongly connected). Practically speaking, this means that only along a certain (small) number of keywords one may navigate in the graph by following a directed path of consecutive mentions from the one keyword to the other. Similarly, the fact that Williams’ keyword-net is not <strong>acyclic</strong> implies that there are cyclic directed paths of consecutive mentions from a certain keyword returning to themselves, which are either reciprocating mentions between two keywords or they are passing through a number of at least two other keywords before they return to the keywords form where they have originated. In particular, the <strong>reciprocity coefficient</strong> of Williams’ keywords-net is computed to be equal to 0.2675, meaning that a percentage of 26.75% of pairs of keywords possess mutually reciprocating mentions.</p><p>Now, collecting the maximum subgraph of strongly connected keywords of Williams’ keyword-net, one gets what in <em>Graph Theory</em> is called the “<strong>largest strongly connected component</strong>” of the graph, which is composed of 126 keywords and 2054 mentions. (The five keywords lying outside the largest strongly connected component of Williams’ keyword-net are [<em>Career, Commercialism, Existential, Monopoly, Work</em>].) Notice that in the largest strongly connected component of the graph every two keywords are joined to each other through paths of consecutive mentions which always follow the same direction from the source to the target keyword.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ax6wlRFfjWml6k5VME1iXA.png" /></figure><p>Having the strongly connected component of a graph, there are several efficient algorithms to derive what in <em>Graph Theory</em> is called a “<strong>maximum spanning arborescence</strong>” of the strongly connected component of the graph. This is the maximal rooted tree extended all over the graph such that there is a root (node) pointing towards all other nodes (now nodes being keywords) along directed paths of minimum length (<strong>geodesic paths</strong>).</p><p>Actually, the computation of the maximum spanning arborescence of a strongly connected directed graph answers a practical question. Since in such a graph, starting from any node (as a source), all other nodes are accessible (as targets) along directed paths, the question is to find the source starting from which all targets can be accessed “as fast as possible,” in the sense of transversing the minimal number of intermediate nodes between source and target. Clearly, the answer is that the root of the maximum spanning arborescence, by construction, satisfies this requirement. In Williams’ keyword-net it turns out that such root-source is the keyword <em>Improve</em>. Therefore, if one starts reading the essay of the keyword <em>Improve</em> and subsequently follows the mentions-links of the maximum spanning arborescence, one is going to read (span) all the essays of the 126 keywords of the largest strongly connected component of Williams’ keyword-net in the most economizing way. A partial detail of the maximum spanning arborescence of Williams’ keyword-net is shown below (where the root <em>Improve</em> is located in the upper left corner):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KmVGZKybxZoXf1-zhnTb7A.png" /></figure><p>Similarly, for any node of Williams’ keyword-net, one may construct the <strong>Breadth First Search tree</strong> (or <strong>BFS-tree</strong>) with root this node, which reaches other keywords (but not all, unless it is the root) along geodesic paths. Notice that since the diameter of the largest strongly connected connected component of Williams’ keyword-net is 5, all the geodesic distances are less or equal to 5. The following is a partial view of the BFS-tree from the keyword <em>Culture</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-QC8Gl4luxuVBFdIZu442g.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=de7bb0e0a9f8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cumulative Rank Aggregation of a Family of Network Centrality Indices]]></title>
            <link>https://medium.com/@mosabou/cumulative-rank-aggregation-of-a-family-of-network-centrality-indices-e625a76bf7e4?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/e625a76bf7e4</guid>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Sun, 01 Apr 2018 21:02:24 GMT</pubDate>
            <atom:updated>2018-04-01T21:02:24.010Z</atom:updated>
            <content:encoded><![CDATA[<p>A growing number of centrality indices are used today in social network analysis. The purpose of using all these network centrality measures is that through them one might be able to identify the most important nodes according to a variety of structural criteria (like nodal degree, closeness, betweenness, eigenvector, PageRank etc.). Moreover, computations (in Python, R etc. or standalone applications) may very easily derive the tables of various centrality indices of network nodes. Therefore, knowing a good deal of network nodal centralities, the crucial question would be how to make sense for all such indices in a illuminating way that would account for the structural features that an empirical network exhibits. What I am proposing here is a methodology for a cumulative ranking of network nodes according to the scores that each node possesses, not on a single centrality measure, but on a whole group (a family) of centrality measures. In particular, the family of centrality indices that I am considering here includes:</p><ul><li>for undirected graphs, the following 10 nodal measures: degree, closeness, betweenness, eigenvector, HITS, Katz, PageRank, load, communicability, and current flow, while,</li><li>for directed graphs, the following 10 nodal measures: out degree, in degree, closeness, betweenness, eigenvector, HITS hubs, HITS auths, Katz, PageRank, and load.</li></ul><p>The first step of the procedure that I am following requires the conversion of the sequence of numerical values (typically, scalars from 0 to 1), which a centrality measure takes on network nodes, to a corresponding ordinal sequence of integers, which represent the existing ranking (ordering) among the numerical values of nodal centrality indices. For instance, if a certain centrality index takes the following values on the 15 nodes of a network:</p><pre>[0.3684210526315789, 0.4827586206896552, 0.4375, 0.4, 0.3888888888888889, 0.3333333333333333, 0.4666666666666667, 0.32558139534883723, 0.56, 0.2857142857142857, 0.3684210526315789, 0.5, 0.3888888888888889, 0.4375, 0.4827586206896552]</pre><p>then the corresponding (sequence of) ranks of the 15 nodes is the following:</p><pre>[5, 6, 5, 3, 4, 2, 10, 8, 3, 8, 7, 7, 11, 9, 1]</pre><p>Notice that a sequence of ranks is always given in decreasing order, starting with the rank of the largest element of the sequence of numerical values, the rank of which is set equal to 1. Apparently, the maximum rank can be less or equal to the total number of nodes, as it happens in this examples where there exist 4 “ties,” the maximum rank being 11.</p><p>The point is that, when concerned with nodal rankings, it would be motivational to think of a network as an electoral system that uses preferential ballots. In this sense, each node can be considered as a candidate in a hypothetical election with the centrality indices being the voters, “who” instead of voting for a single candidate/node, they are placing their votes as preferences in a hierarchy on the ordinal scale. For instance, in the previous example, the shown sequence of ranks can be considered as the vote of a single voter/centrality-measure, who casts the last node/candidate on top of everybody else in her preferences (1st rank), putting the first node somewhere in the middle (5th rank) and so on, attributing rank-choices for all nodes/candidates of the network/electoral-system.</p><p>Then the question is, after all voters/centrality-indices cast their vote in the form of a ranked choice over candidates/nodes, who is the winner among the latter? Actually the final outcome of a ranked voting procedure has to be a rank itself (like the previous example’s sequence of ranks), in which all candidates/nodes are placed in a resulting hierarchical order according to the preferences of voters/centrality-indices. Nevertheless, one should add the caveat that such electoral systems are not always conclusive, although it is not the case here to be concerned with such paradoxes. In any case, what I have described metaphorically is exactly the general idea of what might be called the outcome of a rank aggregation of a family of network centrality indices (seen as voters), essentially, ordering in an ordinal scale hierarchy all the network nodes (seen as candidates) in order to be able to pronounce the relative “collective” or “cumulative” importance of all the network nodes in terms of a multi-sided totality (a family) of network centrality indices (and not the one-sidedness of separate structural criteria).</p><p>Technically speaking, the computational methodology to implement such a rank aggregation is informed by the advances in the Theories of Social Choice and Voting. However, given the computational complexity of the problem, especially for large networks (as the problem is NP-hard even for only four voters/centrality-indices, because of which one is necessitated to resort to approximate solutions, usually provided by a constrained integer programming formulation of the problem), the only thing I can do here is to talk about the heuristics of my approach of how to come to grips with the problem of rank aggregation of scores coming from a whole range of network centrality measures.</p><p>Actually, my idea resembles the pairwise comparisons of the Kemeny-Young method of rank aggregation, although what I am doing is something simpler and it proceeds in a traversal direction. Instead of considering the ranks in a centrality index that nodes possess, I am focusing on the ranks of single nodes for all the centrality indices (of a given family). To explain what I mean, it would be more instructional to start with examples. Say that a certain node (let us say node i) possesses the following “profile” of scores on 10 centrality indices:</p><pre>[3, 2, 5, 3, 3, 4, 6, 5, 4, 4]</pre><p>This means that node i is ranked 3rd with regards to the first centrality index, 2nd the second, 5th the third and so on. Let us pick up the minimum of all the ranks of i in this profile, which is 2 here, and then ask the question: If we take a second node j, are all the ranks in j’s profile bigger or equal than 2, the minimum of the ranks in i’s profile? If they are, we are going to say that “node i beats node j” and the rationale of this expression comes from the fact, for a node, the higher a particular rank in a certain centrality happens to be, the lower the importance of this node with regards to that centrality turns out to be. For example, if the node j had the centrality profile:</p><pre>[5, 9, 12, 14, 14, 14, 13, 12, 14, 14]</pre><p>then it would be the case of i beating j. However, for a node k with profile:</p><pre>[1, 4, 2, 5, 5, 1, 2, 1, 2, 2]</pre><p>it would not be true that i would be beating k, but exactly the opposite that k would be beating i. Notice that for a node p with profile:</p><pre>[1, 2, 3, 8, 9, 10, 11, 12, 13, 14]</pre><p>neither i beats p nor p beats i.</p><p>However, given the fact that the so defined relationship of “beating” is reflexive (i.e., i beats herself, for every node i!), it makes sense to count how many are the nodes j that i beats. I would like to call this number of nodes that a given node i beats as the “rung” of i, because the more nodes i beats, the higher to the rank hierarchy i ascends. For instance, if the centrality profile of node i was such that i was ranked first in every centrality (in which case the minimum rank of i would be1), then trivially i would beat all network nodes and the rung of i would be equal to the total number of nodes in the network. On the other side, if there was a single node q ranked last in every centrality, while all the other nodes were beating k, then the rung of k would be obviously taken equal to 1. Typically, several (if not most) nodes would be ranked in between the previous two extremes.</p><p>Therefore, the final rank of a node could result by subtracting her rung from the total number of nodes of the network. Obviously, knowing the rung of each node amounts to knowing her “cumulative” or aggregate ranking in accordance to the whole set of hierarchical placements that the entire family of centrality measures imposes on every network node.</p><p>In another post (<a href="https://medium.com/@mosabou/the-twitter-network-of-the-february-march-2018-ucu-mobilization-7c81e57a2874">https://medium.com/@mosabou/the-twitter-network-of-the-february-march-2018-ucu-mobilization-7c81e57a2874</a>), I have applied this methodology of aggregate network ranking to a number of relatively large Twitter networks (of hashtags and mentions). In the sequel, here, I am going to give three simple examples of small networks.</p><p>The first example is the network of the renowned 15 Florentine Families with with 20 ties (marriages etc.):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/1*v3Dm2lT4zBpVGgUqCwSDHw.png" /></figure><p>The 15 nodes (families) of the Florentine network take the following values with regards to the following 10 centrality indices:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/1*HVtg4YK2rOZrk5RAxajAmw.png" /></figure><p>Furthermore, the corresponding individual nodal ranks on each centrality are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/1*BF5cxIYYInpfOZAVUgWjjA.png" /></figure><p>Applying the rank aggregation methodology that I have described above gives the following “cumulative” network ranks of the 15 Florentine families:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/205/1*KN3fb2CoB08bY45i6kbDxA.png" /></figure><p>As one may observe, the Medici have the highest total rank (in fact, the Medici beat every other family/node in the Florentine network). Second in the overall ranking come 3 families (Strozzi, Guadagni and Ridolfi), who stand for as a group of highly frequented hubs. Third, fourth and fifth in rank are 7 families lying at the periphery of the Medici and the main hubs. At the very end, sixth in rank come 4 families who are what in graph theory are called “leaves,” i.e., nodes with degree one, which are hanging upon other (better embedded) nodes. Apparently, the structural classification given by this methodology of network centrality aggregation makes also perfect sense in terms of the historical importance and political salience (power) of the 15 Florentine Families.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/1*40P5cZj4MKoRsdRClO4W4g.png" /></figure><p>The second example is David Krackhardt’s “strong trust network at the Pentagon” (that I’ll be simply calling “Krackhardt’s network” from now on), which is composed of 17 nodes/persons and 41 edges/ties.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/1*jaWpTe2xb5NcI0oKBPTbiw.png" /></figure><p>The table of centralities is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*tI_yYGXbnzxAiDeZRWjxTg.png" /></figure><p>and the individual centrality rankings:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*3EvQ9OfOw6P1gZ8Gp_XGKQ.png" /></figure><p>resulting the following cumulative nodal ranks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/199/1*xMiwxr_XWJjpU-jbQXPLFw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*Hfl_4tAt0rg2iRKu78_C0Q.png" /></figure><p>The last example is an “artificial” directed Gnm random network composed of 50 nodes and 150 edges:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/1*0EkrWigJKu-_8TZEeOAolw.png" /></figure><p>Nodal centralities (only of a selection of 15 nodes):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*zrpb0y3wZplZnGZSfkzhxA.png" /></figure><p>Nodal centrality ranks (only of a selection of 15 nodes):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*o2iS4jlZ5pKvJkEadusbsw.png" /></figure><p>Nodal ranks resulting cumulatively from all 10 centralities (shown are only nodes up to the 4th rank):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/159/1*xpd0wyHuJPgVnx-GCVFVeg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*-x7x0g9e--4-h9kZfLVmHw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e625a76bf7e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Twitter Network of the February-March 2018 UCU Mobilization]]></title>
            <link>https://medium.com/@mosabou/the-twitter-network-of-the-february-march-2018-ucu-mobilization-7c81e57a2874?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c81e57a2874</guid>
            <category><![CDATA[big-data]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Wed, 28 Mar 2018 12:15:09 GMT</pubDate>
            <atom:updated>2018-03-29T18:10:38.591Z</atom:updated>
            <content:encoded><![CDATA[<p>This is a report of the first findings of the Twitter network analysis that I am doing with Sergios Lenis on a (sample) dataset of tweets related to the February-March 2018 UCU Mobilization. Actually, this research is a part of a project on Union Digicomm that we are currently conducting with Athina Karatzogianni, Ioanna Ferra, Torsten Geelan, Anastasia Veneti, Lina Dencik, Alessandro Gandini and Maria Rovisco.</p><p>Our dataset was composed of 49,019 tweets which were mined from the Twitter API in the time period from 19 February to 17 March 2018 and on the basis of the following five search terms (hashtags):</p><p>#nocapitulation<br>#livunistrike<br>#strikeforuss<br>#ucustrike<br>#ussstrikes</p><p>What follows below is a brief presentations of three Twitter data/network-analytic issues: (1) Time Series, (2) Hashtags Networks and (3) Mentions Networks.</p><h3>1. Time Series</h3><p>The collected 49,019 tweets have been written by 10,815 users or tweeple (as they are going to be mentioned from now on). Needless to say that, though each one of these tweeple might correspond to a Twitter account of a person or an organization (or possibly even a bot), we are using the term “tweeple” to refer to all such cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qrx-w8EBnDL0u-Np-Ufp8A.jpeg" /><figcaption>The time series of the count of tweets and tweeple per day.</figcaption></figure><p>According to the bulk of tweets they have contributed, the following two tables display those tweeple, each one having sent more than 200 tweets in the whole period and more than 50 tweets per day:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iUkwANlqnhnWnfKDSYx2ww.png" /><figcaption>Top tweeple, each one having contributed more than 200 tweets in the whole period.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ETvnA5rC-Iw4kFFC3AWucA.png" /><figcaption>Top tweeple, each one having contributed more than 50 tweets per day.</figcaption></figure><p>These tweets were written in 35 languages totally, though 90% (44,256 tweets) in English and 9% (4,229) in an undefined language (according to the language classification of the Twitter API). Other languages are: Spanish (153), Cypriot and Greek (64), French (60), Indian (47), German (34), Italian (30) etc.</p><p>There were embedded media inside the contents of the collected tweets: 36,404 photos or URLs to photos and 113 videos. The time distribution of these embedded media per day is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mZ7B1hJKg_MugUodkjSS6w.jpeg" /><figcaption>The time series of 36,517 embedded photos.</figcaption></figure><p>In many tweets, there were tweeple mentioned in their content. The time series in the daily distribution of the number of tweets is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k5JUddqVOiM52daLRXrBAA.jpeg" /><figcaption>The time series of mentions.</figcaption></figure><p>In all these tweets there are 100 hashtags (including the ones of the queries in the Twitter API), which are distributed per day as in the following time series:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lPYxT3L5WqSQ1GdzstJAMA.jpeg" /><figcaption>The time series of the 100 hashtags.</figcaption></figure><p>The top hashtags (according to the frequency of occurrences in tweets) are the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/434/1*THVBrTPg5vzoNg2_cBs20g.png" /></figure><p>The time distribution of the top 20 hashtags is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eBb175S_2skhmk6IUPVm6w.jpeg" /><figcaption>The time series of the 20 top hashtags.</figcaption></figure><p>A sentiment analysis of the content of these tweets has revealed the following scores (according to the dimensions of polarity and subjectivity) when the tweets were grouped in the 20 top hashtags they were including:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fHBEyAGP6Ynk90Hgo9sKtQ.jpeg" /><figcaption>The polarity score of the sentiment analysis of tweets grouped in the 20 top hashtags.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D8QVJe8AH7p_uEj2i4HSxA.jpeg" /><figcaption>The subjectivity score of the sentiment analysis of tweets grouped in the top 20 hashtags.</figcaption></figure><h3>2. Hashtags Networks</h3><p>The graph of co-occurrent hashtags is composed of 3,619 nodes (hashtags) and 11,442 edges (co-occurrences of hashtags in tweets). The biggest connected component of the graph of hashtags contains 3,430 nodes and 11,136. Moreover, the hashtags of the biggest connected component are divided in 129 communities. What follows is a visualization of this graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tbLoihw8oBhNJHcZMSIjCA.png" /><figcaption>The biggest connected component of the graph of co-occurrent hashtags.</figcaption></figure><p>Of course, this is a weighted graph, in which the weight of an edge signifies the total number of co-occurrences among the two hashtags at the end points of the edge (co-occurrence). By filtering the weights of the biggest connected component of the graph of hashtags so that only weights larger than 10 co-occurrences are retained, what remains is a graph composed of 141 hashtags and 1,363 co-occurrences. Moreover, the nodes (hashtags) of this graph are partitioned in 4 communities. The thus filtered hashtags graph is plotted below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZDot-xp7rZiFLvpKhQ1IGA.png" /><figcaption>The filtered biggest connected component of the graph of co-occurrent hashtags for weights &gt; 10.</figcaption></figure><p>All these computations were implemented in a Zeppelin Notebook, where it is possibly to proceed interactively. For instance, one can focus on the ego-centric subgraphs of the hashtags graph by picking up a certain hashtag (called ego) and the hashtags with which the ego co-occurs (called alters) together with the existing co-occurrences among alters. Here, we are showing the ego-centric hashtags graph for the ego “#pension” composed of 51 nodes (hashtags) and 367 edges (co-occurrences). Notice that the ego and the edges-co-occurences between ego-alters are coloured yellow, while the alters and the edges-mentions alter-alter are coloured blue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Or9kFASgKz56At-_4COO6A.png" /><figcaption>The ego-centric hashtags graph around a single hashtag (“#pension”).</figcaption></figure><p>Similarly, by considering several egos instead of a single one, what results is a multi-ego-centric subgraph. For instance, the multi-ego-centric hashtags graph with three hashtags (“#leicester”, “#sheffield” and “#edinburgh”) is composed of 51 nodes (hashtags) and 277 edges (co-occurrences). This multi-ego-centric hastags graph is plotted below. Notice that each ego together its edges (co-occurrences) takes a distinct colour, which is different from the colour of the alters and their edges (co-occurrences).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hf5nGobNoZfB_Ak-LJ63Ww.png" /><figcaption>The multi-ego-centric hashtags graph around a three hashtags (“#leicester”, “sheffield” and “#edinburgh”).</figcaption></figure><p>As it is known from Social Network Analysis (SNA), one way to assess the importance of nodes in a network is though the indices of their centralities. In particular, the following table exhibits all the (common) centralities of the top 20 nodes (hashtags) of the connected component of the hashtags graph (“top” according to their sorting in the degree centrality):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*NL6_n3bk6EmwqRYncuBAuQ.png" /><figcaption>The 8 commonest centrality indices of the 20 top (in degree centrality) nodes (hashtags) of the connected component of the hashtags graph.</figcaption></figure><p>Thus, aggregating all the above centrality indices, we obtain the following ranking of the 20 top hashtags (in the connected component of the hashtags graph):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*-d_vRid6Dxil7a4CSdwbpw.png" /><figcaption>The 20 top (by aggregating all the previous centrality indices) nodes (hashtags) of the connected component of the hashtags graph.</figcaption></figure><h3>3. Mentions Networks</h3><p>Finally, the graph of mentions is a directed graph, the nodes of which are 9,507 tweeple and the (directed) edges or arcs are 19,882 mentions among tweeple. The biggest weakly connected component of the mentions graph is composed of 8,836 nodes (tweeple) and 19,487 edges (mentions). Again, this is a weighted graph, in which the weight of an edge signifies the total number of mentions among the two tweeple at the end points of the edge. By filtering the weights of the biggest weakly connected component of the graph of mentions so that only weights larger than 25 mentions are retained, what remains is a graph composed of 50 tweeple and 158 mentions. This filtered mentions graph is plotted below (where the colours of nodes-tweeple are yellow, whenever the out-degree of a node is larger than its in-degree, they are blue, when in-degree &gt; out-degree, and they are red, when in-degree=out-degree):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XiEYTBaVIhIT51BdkF5qEQ.png" /><figcaption>The filtered biggest weakly connected component of the graph of mentions for weights &gt; 25</figcaption></figure><p>Moreover, what we are showing below is the ego-centric mentions graph for the ego “AlisonGilmour” composed of 63 nodes (tweeple) and 240 edges (mentions). Notice that the ego and the edges-mentions ego-alters are coloured yellow, while the alters and the edges-mentions alter-alter are coloured blue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rbgzQ4aUgUWpoXj93eEasg.png" /><figcaption>The ego-centric mentions graph around a single tweeple (“AllisonGilmour”).</figcaption></figure><p>Similarly, we are giving one example of a multi-ego-centric subgraph around four egos (“CBanthro”, “durani_nauren”, “HelenCFinch” and “GraemeLaurie”), which is composed of 95 nodes (tweeple) and 292 edges (mentions). This multi-ego-centric mentions graph is plotted below. Notice that each ego together her edges (mentions) takes a distinct colour, which is different from the colour of the alters and their edges (mentions).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ilJa_Fi1we7Rl62UhTEY6w.png" /><figcaption>The multi-ego-centric mentions graph around a four tweeple (“CBanthro”, “durani_nauren”, “HelenCFinch” and “GraemeLaurie”).</figcaption></figure><p>Returning to the whole weakly connected component of the mentions graph, the following table exhibits all the (common) centralities of the top 20 nodes (tweeple) of this graph (“top” according to their sorting in the out-degree centrality):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RpR0bqyyK_P6-s5jlPfgyg.png" /><figcaption>The 10 commonest centrality indices of the 20 top (in out- degree centrality) nodes (tweeple) of the weakly connected component of the mentions graph.</figcaption></figure><p>Finally, aggregating all the above centrality indices, we obtain the following ranking of the 20 top tweeple (in the weakly connected component of the mentions graph):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*_-BbOw-AwVCUTJQoj3fNvw.png" /><figcaption>The 20 top (by aggregating all the previous centrality indices) nodes (tweeple) of the weakly connected component of the mentions graph.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c81e57a2874" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Lost in Translation” inside a Multipartite Network]]></title>
            <link>https://medium.com/@mosabou/lost-in-translation-inside-a-multipartite-network-60650747c175?source=rss-b3c587159c74------2</link>
            <guid isPermaLink="false">https://medium.com/p/60650747c175</guid>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Moses Boudourides]]></dc:creator>
            <pubDate>Fri, 15 Sep 2017 23:35:03 GMT</pubDate>
            <atom:updated>2017-09-15T23:35:03.303Z</atom:updated>
            <content:encoded><![CDATA[<p>It is a commonplace that to make a good translation one needs to apprehend the double context before and after the translation, i.e., the context of both what is to be translated and the context of that to which it has been translated. To this purpose, my main claim is the suggestion that one might need to check a good number of synonyms of a word in both languages before and after the word is translated in order to double-check the right translation. Moreover, I am claiming that the selection of a proper translation might be implemented algorithmically, formally inside a multipartite network structure encompassing the bundle of translinguistic renditions that need to set in motion for this purpose.</p><p>The idea is very simple, but it would require a good deal of abstract formal definitions to be described (especially for the incognizants of certain mathematical technicalities). Before doing this, I would rather attempt to expressing it in an informal way, which might sound to be unnecessarily circumlocutory. Say <em>x</em> is a word in a language <em>L</em> to be translated into a second language <em>M. </em>Instead of translating <em>x</em> directly from <em>L</em> to <em>M</em>, one might want to follow the detour of first gathering a whole set of <em>L</em>-synonyms of <em>x</em> and subsequently translating all these synonyms in <em>M</em>. Intuitively, one would expect that the more relevant are the collected <em>L</em>-synonyms (of <em>x</em>), the more plausible it would be that the right (or “good”) <em>M</em>-translations of <em>x</em> could be enclosed inside them. However, at this point, there is no ad hoc valid criterion of how to select the proper translations. My suggestion is that to be able to detect efficaciously the “good” translations, one would need to proceed further ahead and then to reverse the process backwards. What I mean is that one would need to embrace two further steps leading, more or less, to where one has started, but in an excessive way. The first step would be to gather all the <em>M</em>-synonyms of all the <em>M</em>-translations of all the <em>L</em>-synonyms of the <em>L</em>-word <em>x</em> and the second step would be to try to translate back in <em>L</em> what has been thus gathered in the previous step. Therefore, starting from the <em>L</em>-word <em>x</em>, one will eventually derive an excessive abundance of <em>L</em>-words into which <em>x</em>, in a sense, proliferates.</p><p>However, it cannot be taken for granted that <em>x</em> survives, appears or re-emerges inside the derived translinguistic proliferation of the <em>L</em>-word <em>x</em>. If this was the case, then it would be easy to locate the proper translations of <em>x</em> by reversing the last two steps and going backwards from the found proliferated gathering of <em>L</em>-words to a proper subdomain of the generating translations from where the latter <em>L</em>-words have originated. Actually, this subdomain of translations is simply the intersection of the first forward translations of the synonyms of <em>x</em> with the outcome of a reverse traversing of the last two steps.</p><p>Now, if the starting <em>L</em>-word <em>x</em> is not included in the final dilated gathering of <em>L</em>-words, it turns out that one may use some elementary graph theory in order to assess the depth at which <em>x</em> was lost (“in translation”). This can be done by a simple algorithm that I am going to describe at the last part of my analysis. As I am going to show there, depending on how deeply <em>x</em> was “lost in translation,” one might have some chances to retrieve its proper (“good”) translations.</p><p>Next, let me come to a rather formal description of the above methodology. Without any loss of generality, instead of starting from a single word x, let me assume that we are given a set <em>X</em> of words in a language <em>L</em> to be translated into a second language <em>M.</em> Then what I was informally describing above amounts to a multilayer structure (as a matter of fact, a multipartite graph) consisting of the following five layers:</p><p><strong>Layer 0</strong>: This layer contains all the words of the set <em>X</em> in language <em>L</em> to be translated into language <em>M</em>.</p><p><strong>Layer 1</strong>: For each word <em>x</em> in the set <em>X</em>, one needs to find a good number of synonyms of <em>x</em> in <em>L </em>denoted as<em> S</em>(<em>x</em>)<em>. </em>This means that one is considering a set-valued (or multi-valued) mapping <em>S</em> from individual words of <em>L</em> to subsets of synonymous words in <em>L</em>;<em> </em>here,<em> </em>the mapping<em> </em>$<em>S = S</em>_{<em>L</em>}$<em> </em>will be called <strong><em>L</em>-synonymization mapping</strong>. Thus, exhausting all the words in <em>X</em>, one obtains a set <em>S</em> = <em>S</em>(<em>X</em>) of all the synonyms of the words of <em>X</em> in <em>L</em> (provided that the latter comply with the context and the grammatical type of the former in <em>L</em>).</p><p><strong>Layer 2</strong>: Next, one needs to translate each word <em>s</em> of the set <em>S</em> into one or more words in language <em>M,</em> thus, getting, in general, a subset <em>T</em>(<em>s</em>) of words of <em>M</em>. In other words, $<em>T = T</em>_{<em>L,M</em>}$ is a set-valued mapping from words of <em>L</em> to subsets of translated words in <em>M</em> which here is going to be referred to as a <strong><em>L</em>-to-<em>M</em>-translation mapping</strong>. The image of $<em>S</em>$ under this mapping is a set <em>T</em> = <em>T</em>(<em>S</em>) of the translated synonyms <em>S</em> of all the words of <em>X</em> from language<em> L </em>to language<em> M.</em></p><p>Notice that Layer 2 contains only the translations of the synonyms of the words of set <em>X</em>, not necessarily actual translations of the words of set <em>X </em>(although the latter might be indirectly produced through some synonyms)<em>. </em>The idea is that the proper translations of the words of set <em>X</em> are going to be selected by certain “projections” of the initial Layer 0 and the final Layer 4 onto Layer 2. Needless to say that the bigger the set <em>S</em> of synonyms of the words of <em>X</em>, the more accurate is the selection of the sought proper translations (although this might cost to increasing the computational time and complexity of the proposed algorithm).</p><p><strong>Layer 3</strong>: In addition, synonymizing each word <em>t</em> in <em>T </em>in language <em>M</em>, one needs to find its synonyms in language <em>M </em>denoted as <em>U</em>(<em>t</em>). Apparently, this is done through the action of a set-valued <strong><em>M</em>-synonymization mapping </strong>$<em>U</em> = <em>S</em>_{<em>M</em>}$ from words of <em>M</em> to subsets of synonymous words in <em>M</em>. Thus, exhausting all the words in <em>T</em>, one obtains a set <em>U</em> = <em>U</em>(<em>T</em>) of all their synonyms (as far as the latter comply with the context and the grammatical type of the former in <em>M</em>).</p><p><strong>Layer 4</strong>: Finally, one needs to translate each word <em>u</em> of <em>U</em> in language <em>M</em> back to language <em>L </em>getting a subset<em> of </em>words (of<em> L</em>) denoted as<em> V</em>(<em>u</em>). Apparently, this is done through the action of a set-valued <strong><em>M</em>-to-<em>L</em>-translation mapping </strong><em>V</em> from words of <em>M</em> to subsets of translated words in <em>L</em>, denoting $<em>V</em> = <em>T</em>_{<em>M,L</em>}$. The image of $<em>U</em>$ under this mapping is a set <em>Y</em> = <em>V</em>(<em>U</em>) of the translated synonyms <em>U</em> of all the words of <em>T</em> from language<em> M </em>to language<em> L.</em></p><p>In other words, the above steps can be illustrated by the following chain of transformations (alternating synonymization and translation) from set <em>X</em> in Layer 0 to set <em>Y</em> in Layer 4 (both <em>X, Y</em> being subsets of words of <em>L</em>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*YUjnbb9hmxSb4ds56OXbwg.png" /></figure><p>Notice that the set <em>Y</em> is the outcome of the composition of four (set-valued) mappings acting on set <em>X</em>, in the sense that, for any <em>L</em>-word <em>x</em> in <em>X</em>, the following formula:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/310/1*r8DBsyg9zJryIuQDGcSzCw.png" /></figure><p>returns a subset <em>y</em> of words in <em>L</em>. In this way, <em>y</em> = <em>F</em>(<em>x</em>), where the set-valued mapping <em>F</em> maps words of <em>L</em> to subsets of words of <em>L,</em> is the composition of the above mappings of translation and synonymization:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/266/1*jev7EOPSzjGlgku7bnR3xQ.png" /></figure><p>Apparently, one might follow the above steps in the opposite direction too, through the following chain of transformations (alternating translation and synonymization) from set <em>Y</em> to set <em>X</em> (both subsets of words of <em>L</em>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/1*YnXJpWbZEpaXhHLTBWeecw.png" /></figure><p>Again, notice that the set <em>X</em> is the outcome of the composition of four (set-valued) mappings acting on the set <em>Y</em>, in the sense that, for any <em>L</em>-word <em>y</em> in <em>Y</em>, the following formula:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/307/1*YEwnvxlQy-JDMR-ccq_sJA.png" /></figure><p>returns a subset <em>x </em>of words in <em>L</em>. In this way, $<em>x</em> = <em>F^{-1}</em>(<em>x</em>)$, where the set-valued mapping $<em>F^{-1}</em>$ maps words of <em>L</em> to subsets of words of <em>L,</em> is the composition of the above mappings of translation and synonymization:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*_h82QzLbYsy0pvGRwGhdLA.png" /></figure><p>At this point, let me resort to a very elementary fact that is going to help us shift the focus of our analysis (and computations) from set-valued mappings to graphs. This is the following remark: Given two finite (disjoint) sets <em>A</em> and <em>B</em>, any set-valued mapping <em>Φ</em> from elements of <em>A</em> to subsets of <em>B</em> corresponds to a <strong>bipartite graph</strong> <em>G</em> with bipartition (<em>A,B</em>) (which is a undirected graph with its nodes being all the elements of both sets <em>A</em> and <em>B</em> and its edges being the associations of any <em>x</em> in <em>A</em> to all the elements of the subset <em>Φ</em>(<em>x</em>) of <em>B</em>). Conversely, any bipartite graph <em>G</em> with bipartition (<em>A,B</em>) corresponds to a set-valued mapping <em>Φ</em> from elements of <em>A</em> to subsets of <em>B</em> (such that <em>Φ</em>(<em>x</em>) is composed of all nodes in <em>B </em>with which node <em>x</em> is linked in the graph).</p><p>Therefore, at each layer (0–4), the following correspondences between set-valued mappings and bipartite graphs can be considered too:</p><ol><li>The <em>L</em>-synonymization mapping $<em>S</em>_{<em>L</em>}$ corresponds to a bipartite graph (still denoted by) $<em>S</em>_{<em>L</em>}$ with bipartition (<em>X,S</em>), where <em>X,</em> <em>S</em> are subsets (of words) in the set (language) <em>L</em>.</li><li>The <em>L</em>-to-<em>M</em> translation mapping $<em>T</em>_{<em>L,M</em>}$ corresponds to a bipartite graph (still denoted by) $<em>T</em>_{<em>L,M</em>}$ with bipartition (<em>S,T</em>), where <em>S,</em> <em>T</em> are subsets (of words) in the sets (languages) <em>L, M, </em>respectively.</li><li>The <em>M</em>-synonymization mapping $<em>S</em>_{<em>M</em>}$ corresponds to a bipartite graph (still denoted by) $<em>S</em>_{<em>M</em>}$ with bipartition (<em>T,U</em>), where <em>T,</em> <em>U</em> are subsets (of words) in the set (language) <em>M</em>.</li><li>The <em>M</em>-to-<em>L</em> translation mapping $<em>T</em>_{<em>M,L</em>}$ corresponds to a bipartite graph (still denoted by) $<em>T</em>_{<em>M,L</em>}$ with bipartition (<em>U,Y</em>), where <em>U, Y </em>are subsets (of words) in the sets (languages) <em>M, L, </em>respectively.</li></ol><p>Now, when the above set-valued mappings are sutured together, one gets a 5-partite graph with 5-partition (<em>X,S,T,U,Y</em>). This 5-partite graph will be denoted here as <em>G</em> and it will be referred to as the <strong>full synonymization-translation graph</strong>. Of course, retaining only the starting and the final sets of words (in <em>L</em>) associated to each other through the set-valued mapping of compositions <em>F</em>, one gets the <strong>concise synonymization-translation graph</strong>, denoted here by <em>H</em>, which is the bipartite graph with bipartition (<em>X,Y</em>) (that corresponds to the set-valued mapping <em>F</em>). Needless to say that <em>H</em> is isomorphic (and, as such, identical) to the bipartite graph that corresponds to the set-valued mapping $<em>F^{-1}</em>$ (which has bipartion (<em>Y,X</em>)).</p><p>Before addressing the issue of selecting the proper translations of words in <em>X</em> from language <em>L</em> to language <em>M,</em> I am going to give a concrete example on which my proposed methodology is going to be applied. In this example, the source language <em>L</em> is <strong>English</strong> and the target language <em>M</em> is <strong>Greek</strong>. In particular, the dataset of this example is composed of:</p><ul><li>59 unique words in English (in the set <em>X</em>, Layer 0),</li><li>synonymized into 69 unique English words (in the set <em>S</em>, Layer 1),</li><li>translated into 50 unique Greek words (in the set <em>T</em>, Layer 2),</li><li>synonymized into 71 unique Greek words (in the set <em>U</em>, Layer 3),</li><li>translated into 58 unique English words (in the set <em>Y</em>, Layer 4).</li></ul><p>Totally, the dataset of this example consists of a 294x5 matrix (accessible at <a href="https://docs.google.com/spreadsheets/d/1zj7-3Lyz33TaPz9CtVJ7IdepHpw2L1fwrH1KkwtoL6o/edit?usp=sharing">https://docs.google.com/spreadsheets/d/1zj7-3Lyz33TaPz9CtVJ7IdepHpw2L1fwrH1KkwtoL6o/edit?usp=sharing</a>). For instance, the top (sorted alphabetically in the first column) 7 rows of this matrix are (unlabelled):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*osMRUJmmElWgYptHpz3uJw.png" /></figure><p>Notice that certain columns (layers) of this dataset involve more or less the same English or Greek words. To be able to distinguish them, I am going to use the the notation of the following labels:</p><ul><li><em>w</em>_f for words in the set <em>Y</em> (Layer 4) and</li><li><em>w</em>_s for words in the sets <em>S</em> and <em>U</em> (Layers 1 and 3).</li></ul><p>Moreover, given the words <em>x, y, z, u </em>and<em> v</em> in the sets <em>X, S, T, U</em> and <em>Y</em>, respectively, let me define three equivalence relations. Actually, I am going to write:</p><ul><li><em>x ~ v</em>, whenever <em>v = x</em>_f,</li><li><em>x ~ y</em>, whenever <em>y = x</em>_s,</li><li><em>z ~ u</em>, whenever <em>u = z</em>_s.</li></ul><p>All the above pairs of words are said to be pairs of <strong>identical words</strong> (of course, when their labels are discarded). In particular, in the dataset of the example, we have:</p><ul><li>57 identical words in sets <em>X</em> and <em>Y</em> (with the only words of <em>X</em> which are not identical to any word in <em>Y</em> being the two words [fury, option] and the only word of <em>Y</em> which is not identical to any word in <em>X</em> being the single word [sensitivity_f]);</li><li>31 identical words in sets <em>X</em> and <em>S</em> (with 38 words of <em>X</em> which are not identical to any word in <em>S</em> and 28 words of <em>S</em> which are not identical to any word in <em>X</em> );</li><li>32 identical words in sets <em>T</em> and <em>U</em> (with 39 words of <em>T</em> which are not identical to any word in <em>U</em> and 18 words of <em>U</em> which are not identical to any word in <em>T</em> ).</li></ul><p>Considering now the correspondence between set-valued mappings and graphs, we remark that this example generates a 5-partite (full synonymixation-translation) graph <em>G</em> with 307 nodes and 388 edges. Actually, each column (layer) of the dataset becomes one of the five sets (modes) of the 5-partition of <em>G</em>. A visualization of the 5-partite graph<em> G</em> of this example is given below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d45XmgAxaGu8xdyKpiJvAQ.png" /><figcaption>The 5-partite graph G (with the edges of the smallest connected component colored red).</figcaption></figure><p>The corresponding concise synonymization-translation graph is a bipartite graph <em>H</em> with 117 nodes (the set <em>X</em> being the first mode of <em>H</em> and the set <em>Y</em> the second mode) and 231 edges:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v5TxzqDF6wUsXjw9AfyhjA.png" /><figcaption>The bipartite graph H (with the edges of the smallest connected component colored red).</figcaption></figure><p>Let me now remind the standard notion of <strong>connectedness</strong> (for bipartite graphs) from Graph Theory. Without any loss of generality, let me consider only (alternating)<strong> walks</strong> of words between <em>X</em> and <em>Y </em>in<em> </em>the bipartite graph<em> H, </em>which (walks) have <strong>odd length</strong><em>. </em>In general, any such a walk <em>W</em> is a family of nodes (words) <em>W</em> = {<em>x_0, y_1, x_1, y_2</em>, …, <em>x_{k-1}</em>, <em>y_k</em>}, for some positive integer <em>k</em>, where <em>x_0, x_1, …, x_{k-1}</em> are words in <em>X</em>, <em>y_1, y_2, …, y_k</em> are words in <em>Y </em>and <em>y_j</em> belongs to <em>F(x_j)</em>, for all integers <em>j </em>between <em>0</em> and <em>k</em>. Point <em>x_0 </em>is called the <strong>starting word</strong> of walk <em>W</em> (in<em> X</em>)<em> </em>and point<em> y_k </em>is called the <strong>terminal word</strong> of <em>W</em> (in<em> Y</em>). Moreover, given two points (words) <em>x</em> and <em>y</em> in <em>H</em>, with <em>x</em> in <em>X</em> and <em>y</em> in <em>Y</em>, one says that points (words) <em>x</em> and <em>y</em> are mutually <em>k</em>-<strong>accessible</strong> if there exists a walk <em>W</em> in <em>H</em> of length <em>k</em> with starting point <em>x</em> and terminal point <em>y</em>. Often the prefix <em>k</em> will drop and one would say just (mutually) accessible instead of k-accessible.</p><p>However, saying that points (words) <em>x</em> and <em>y</em> are mutually <em>k</em>-accessible, it is meant here that <em>k</em> is the minimum walk length among all possible walk lengths for all walks from <em>x</em> to <em>y</em>: such walks are often called <strong>geodesic</strong>. In addition, for any two two points (words) <em>x</em> and <em>y</em> in <em>H</em>, the (geodesic) <strong>distance</strong> between <em>x</em> and <em>y</em> (denoted as <em>d</em>(<em>x,y</em>))is defined as the (minimum) length of walk connecting <em>x</em> and <em>y</em>, if such a walk exists (and <em>d</em>(<em>x,y</em>) is considered to become infinity, if such a walk does not exist, i.e., if <em>x</em> and <em>y</em> are not mutually accessible). Of course in a bipartite graph, the values of such a distance <em>d</em>(<em>x,y</em>) might be odd number, 1, 3, 5, … (or infinity).</p><p>Furthermore, a (bipartite) graph <em>H</em> with bipartition (<em>X,Y</em>) is called <strong>connected</strong> if any pair of points <em>x, y</em>, with <em>x</em> in <em>X</em> and <em>y</em> in <em>Y</em>, is a pair of mutually accessible points (words). If a graph <em>H</em> is not connected, the points of <em>H</em> are partitioned in a number of <em>connected components</em>. Actually, a <strong>connected component</strong> of <em>H</em> is defined as a <em>maximal </em>set of points of <em>H</em> such that all pairs of points in this set are mutually accessible. Similar definitions hold for the (multipartite) graph <em>G</em>, the only difference being that distances in <em>G</em> are distances in <em>H</em> times 4.</p><p>In our example, the 5-partite graph <em>G</em> is not connected and it contains two connected components. The biggest (giant) connected component of <em>G</em> includes 302 points (words) and the second (the smallest) connected component of <em>G</em> includes 5 points (which are the following words: [momentum, drive_s, θέληση, βούληση_s, will_f]).</p><p>In addition, the bipartite graph <em>H</em> of our example is not connected too and it also contains two connected components. The biggest (giant) connected component of <em>H</em> includes 115 points (words) and the second (smallest) connected component of <em>H</em> includes 2 points (i.e., [momentum, will_f]).</p><p>According to what we have been already discussing in the above informal introduction of the present methodology, what really matters for the solution of the problem of detecting the proper translations of a word <em>x</em> in <em>X</em> is to compute the distance (in the bipartite graph <em>H</em>) <em>d</em>(<em>x,y</em>) of <em>x</em> from the word <em>y</em> in <em>Y,</em> which is identical to <em>x</em>, i.e., <em>y</em> = <em>x</em>_f (in which case <em>x ~ y</em>). As a matter of fact, we are going to distinguishes three cases:</p><ul><li><em>d</em>(<em>x,x</em>_f) = 1;</li><li><em>d</em>(<em>x,x</em>_f) is a finite odd number larger that 1 (i.e., <em>d</em>(<em>x,x</em>_f) = 3, 5, …);</li><li><em>d</em>(<em>x,x</em>_f) is infinity or undefined.</li></ul><p>The case <em>d</em>(<em>x,x</em>_f) = 1 is equivalent to the inclusion of the point <em>x</em>_f into the image <em>F</em>(<em>x</em>) in <em>Y</em> (which is the image of <em>x</em> under the composition <em>F</em> of the above mappings of translation and synonymization). In other words, <em>d</em>(<em>x,x</em>_f) = 1 if there is an edge in the bipartite graph <em>H</em> joining the identical words <em>x</em> and <em>x</em>_f (and a (geodesic) walk of length 4 in the 5-partite graph <em>G</em> joining the identical words <em>x</em> and <em>x</em>_f).</p><p>In our example, there are 33 words <em>x</em> in <em>X</em> such that <em>d</em>(<em>x,x</em>_f) = 1 (this amounts to the 56% of the total words in the layer-column X of the example). This are the following words: [love, consciousness, idea, lust, sympathy, fulfillment, anger, liking, feat, knowledge, implementation, acceptance, stupidity, tendency, awareness, perception, susceptibility, view, prudence, satisfaction, jubilation, preference, reverence, understanding, achievement, desire, conception, predisposition, zeal, fun, opinion, feeling, election]. The edges among such pairs of words are colored <em>red</em> in the following two visualizations of the bipartite graph <em>H</em> and the 5-partite graph <em>G</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1evwRNjHKLzYO5kyP5bWtA.png" /><figcaption>Identical words at distance 1 in the bipartite graph H.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lBxjPN2VzeMEcXSVRdTJnQ.png" /><figcaption>Identical words at distance 4 in the 5-partite graph G.</figcaption></figure><p>The case <em>d</em>(<em>x,x</em>_f) = 3 is equivalent to the fact that the point <em>x</em>_f does not belong into the image <em>F</em>(<em>x</em>) in <em>Y</em>. Obviously, <em>d</em>(<em>x,x</em>_f) = 3 if and only if there is a (geodesic) walk of length 3 in the bipartite graph <em>H</em> joining the identical words <em>x</em> and <em>x</em>_f (and a (geodesic) walk of length 12 in the 5-partite graph <em>G</em> joining the identical words <em>x</em> and <em>x</em>_f).</p><p>In our example, there are 18 words <em>x</em> in <em>X</em> such that <em>d</em>(<em>x,x</em>_f) = 3 (this amounts to the 30% of the total words in the layer-column X of the example). This are the following words: [emotion, impression, sense, craze, absurdity, trend, fervor, furor, compassion, nonsense, delight, pleasure, longing, wisdom, wish, excitement, prejudice, sensibility]. The edges among such pairs of words are colored <em>green</em> in the following two visualizations of the bipartite graph <em>H</em> and the 5-partite graph <em>G, </em>where the latter depicts only the walk between the identical words <em>x</em>=pleasure and <em>y</em>=pleasure_f :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vi4hoE7luzFtuHlHheR5rg.png" /><figcaption>Identical words at distance 3 in the bipartite graph H.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QFVZyrwefdweNsLO7nX2Lg.png" /><figcaption>The pair of identical words x=pleasure and y=pleasure_f at distance 12 in the 5-partite graph G. The 12-walk among these words is [pleasure, joy_s, χαρά, ευαρέσκεια_s, fun_f, ευαρέσκεια_s, ευχαρίστηση, pleasure_s, fun, joy_s, χαρά, ευαρέσκεια_s, pleasure_f].</figcaption></figure><p>The case <em>d</em>(<em>x,x</em>_f) = 5 is clearly equivalent to the fact that the point <em>x</em>_f does not belong into the image <em>F</em>(<em>x</em>) in <em>Y</em>. Obviously, <em>d</em>(<em>x,x</em>_f) = 5 if and only if there is a (geodesic) walk of length 5 in the bipartite graph <em>H</em> joining the identical words <em>x</em> and <em>x</em>_f (and a (geodesic) walk of length 20 in the 5-partite graph <em>G</em> joining the identical words <em>x</em> and <em>x</em>_f).</p><p>In our example, there are 4 words <em>x</em> in <em>X</em> such that <em>d</em>(<em>x,x</em>_f) = 5 (this amounts to the 6.8% of the total words in the layer-column X of the example). This are the following words: [selection, prosperity, joy, luck]. The edges among such pairs of words are colored <em>magenta</em> in the following two visualizations of the bipartite graph <em>H</em> and the 5-partite graph <em>G, </em>where the latter depicts only the walk between the identical words <em>x</em>=pleasure and <em>y</em>=pleasure_f :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BbpQ9OeYdKkDgZWuDsQMhw.png" /><figcaption>Identical words at distance 5 in the bipartite graph H.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M-BZLBwHFuqOyIFFqb-yOQ.png" /><figcaption>The pair of identical words x=luck and y=luck_f at distance 20 in the 5-partite graph G. The 20-walk among these words is [luck, success_s, επιτυχία, ικανοποίηση_s, satisfaction_f, ικανοποίηση_s, ευχαρίστηση, pleasure_s, fun, joy_s, χαρά, κέφι_s, fun_f, κεφι_s, χαρά, joy_s, pleasure, happiness_s, ευτυχία, τύχη_s, luck_f].</figcaption></figure><p>Notice that in the bipartite graph of the example, there are no words <em>x</em> in <em>X</em> with distance <em>d</em>(<em>x,x</em>_f) larger than 5. However, if we sum up the number of words <em>x</em> in <em>X</em> with distances <em>d</em>(<em>x,x</em>_f) equal to 1, 3 and 5, we find 57 words. Since the <em>X</em> contains 59 words, there remain two words [furor, option], which do not possess their identical words in the set <em>Y</em> and, thus, for which the distance <em>d</em>(<em>x,x</em>_f) is undefined.</p><p>Let now describe, how according to this methodology, one would detect a subset <em>T</em>*(<em>x</em>) of the set <em>T</em>(<em>S</em>(<em>x</em>)) consisting of the proper translations of the word <em>x</em> in <em>X</em>. Again, one has to consider the three cases:</p><ul><li><em>d</em>(<em>x,x</em>_f) = 1;</li><li><em>d</em>(<em>x,x</em>_f) is a finite odd number larger that 1 (i.e., <em>d</em>(<em>x,x</em>_f) = 3, 5, …);</li><li><em>d</em>(<em>x,x</em>_f) is infinity or undefined.</li></ul><p>In the first case, when <em>x</em> and <em>x</em>_f are the end points of an edge {<em>x</em>, <em>x</em>_f} in <em>H</em>, one may take <em>T</em>*(<em>x</em>) = <em>T_1</em>*(<em>x</em>) to be the intersection of the set <em>T</em>(<em>S</em>(<em>x</em>)) with the <em>S</em>(<em>T</em>(<em>x</em>_f)):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/409/1*_hhrCcQAQXmxJqG7vgiYmQ.png" /></figure><p>What is the set <em>T</em>(<em>S</em>(<em>x</em>))? It is the set of terminal points of all walks in <em>H</em> which start from <em>x</em> in <em>X</em> (Layer 0) and terminate on the set of translations <em>T</em> (Layer 2). In short, let us give a name to the set of terminal points of all walks starting from some word<em> z </em>in Layer 0 or in Layer 4 and terminating on Layer 2: We call them “forward translational projections of <em>z</em>,” when they start from Layer 0, and “backward translational projections of <em>z</em>,” when they start from Layer 0.<em> </em>In this terminology, <em>T</em>(<em>S</em>(<em>x</em>)) consists of the forward translational projections of <em>x</em> and <em>S</em>(<em>T</em>(<em>x</em>_f)) of the backward translational projections of <em>x_</em>f. Thus, the set <em>T_1</em>*(<em>x</em>) is defined as the intersection of the forward translational projections from <em>x</em> with the backward translational projections from <em>x</em>_f. Moreover, it can be easily proven that <em>d</em>(<em>x,x</em>_f) = 1 if and only if <em>T_1</em>*(<em>x</em>) is a nonempty set.</p><p>Thus, in our example, the detected proper translations of the 33 words <em>x</em> with <em>d</em>(<em>x,x</em>_f) = 1 are given in the following Table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*ezwkJH557msyYF5EvgUDLA.png" /></figure><p>In the second case, when <em>x</em> and <em>x</em>_f are the end points of a 3-walk in <em>H</em>, {<em>x</em>, <em>y</em>_1, <em>x</em>_1, <em>x</em>_f}, then <em>T</em>*(<em>x</em>) = <em>T_3</em>*(<em>x</em>) can be defined as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/1*QXOt1k0vheJ0vav9cVqu6w.png" /></figure><p>Now, we can have two forward translational projections from <em>x</em> and <em>x</em>_1 and two backward translational projections from <em>y</em>_1 and <em>x</em>_f. Then, <em>T_3</em>*(<em>x</em>) is defined as the intersection of the union of two forward translational projections with the union of two backward translational projections. One can show again that this intersection is nonempty.</p><p>In our example, the detected proper translations of the 18 words <em>x</em> with <em>d</em>(<em>x,x</em>_f) = 3 are given in the following Table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*hknQofEmTYpXVxPfvY_Jbg.png" /></figure><p>Similarly, one can proceed with the case that <em>x</em> and <em>x</em>_f are the end points of a 5-walk in <em>H</em>, {<em>x</em>, <em>y</em>_1, <em>x</em>_1, <em>y</em>_2, <em>x</em>_2, <em>x</em>_f}, by defining <em>T</em>*(<em>x</em>) = <em>T_5</em>*(<em>x</em>) as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7CSyea_cxXR3ECpMCD3H_A.png" /></figure><p>In other words, <em>T_5</em>*(<em>x</em>) is defined as the intersection of the union of three forward translational projections (from <em>x</em>, <em>x</em>_1, <em>x</em>_2) with the union of three backward translational projections (from <em>y</em>_1, <em>y</em>_2, <em>x</em>_f) and, again, it can be shown that this intersection is nonempty.</p><p>In our example, the detected proper translations of the 4 words <em>x</em> with <em>d</em>(<em>x,x</em>_f) = 5 are given next:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/376/1*KCyIuNQZBUaNdk-wlTFdBQ.png" /></figure><p>Concluding, let me say that this is was a first exploratory enquiry of the problems of how to evaluate an existing translation and how to enhance the scope of a more comprehensive translation through an algorithmic graph-theoretic methodology. Although the dataset on which this methodology was tested was rather synthetically created and small in size, subsequently, we intend to probe this methodology on bigger datasets extracted from literary corpora already translated in a second language.</p><p>[<em>All the present computations were implemented in Python and I will be happy to share the Jupyter Notebook that I have used with anyone who would be interested in it.</em>]</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=60650747c175" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>