<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ethnographic Machines on Medium]]></title>
        <description><![CDATA[Stories by Ethnographic Machines on Medium]]></description>
        <link>https://medium.com/@EthnographicMachines?source=rss-8838c1d8bfac------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*y2uW0NaB4HYgFVmkXr9DbQ.jpeg</url>
            <title>Stories by Ethnographic Machines on Medium</title>
            <link>https://medium.com/@EthnographicMachines?source=rss-8838c1d8bfac------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 20:13:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@EthnographicMachines/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Pandemic search]]></title>
            <link>https://medium.com/@EthnographicMachines/pandemic-search-cf2051ea7fe0?source=rss-8838c1d8bfac------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf2051ea7fe0</guid>
            <category><![CDATA[digital-methods]]></category>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[google-trends]]></category>
            <category><![CDATA[corona]]></category>
            <category><![CDATA[who]]></category>
            <dc:creator><![CDATA[Ethnographic Machines]]></dc:creator>
            <pubDate>Fri, 20 Mar 2020 03:19:29 GMT</pubDate>
            <atom:updated>2020-03-24T21:15:49.276Z</atom:updated>
            <content:encoded><![CDATA[<h4>What can search interest on Google tell us about differences in the everyday response to coronavirus across the world?</h4><p>For this project, I am using <a href="https://trends.google.com/trends/?geo=US">Google Trends</a> to compare the reaction to novel coronavirus in different countries. It is a work in progress. I am continuously updating the story below as more data become available and analysis progresses.</p><h3>Methodology</h3><p>I first got the most recent data on the geographic distribution of corona cases over time from the <a href="https://www.ecdc.europa.eu/en/publications-data/download-todays-data-geographic-distribution-covid-19-cases-worldwide">website of the European Centre for Disease Prevention and Control</a>. The datasheet provides the numbers of new confirmed cases and new deaths as they are reported by each country each day. On March 11, 2020, 105 countries were part of the reporting.</p><p>I then queried Google Trends with a geographical filter for each of the 105 countries in ECDC data. This is useful for exploring search trends on a country by country basis. Also, rather than querying for specific search terms, which would each have an unknown array of synonyms and need to be translated to all the languages relevant for each geographical filter, I queried for 24 so-called ‘topics’, which are <a href="https://support.google.com/trends/answer/4359550?hl=en">groups of search terms that share the same concept in any language</a>.</p><p>Besides the topic of coronavirus itself :</p><ul><li><em>‘/m/01cpyy’: ’Coronavirus’</em></li></ul><p>These include five topics related to medicine:</p><ul><li><em>‘/m/0hnfc’: ’Epidemiology’</em></li><li><em>‘/m/01byzl’: ’Severe acute respiratory syndrome’</em></li><li><em>‘/m/0cjf0’: ’Fever’</em></li><li><em>‘/m/03vvz’: ’Immune system’</em></li><li><em>‘/m/025sjz4’: ’Immunity’</em></li></ul><p>Four topics related specifically to health care:</p><ul><li>‘/m/07__7’: ’Vaccine’</li><li>‘/m/0g1gcq’: ’Intensive care unit’</li><li>‘/m/01mvn6’: ’Medical ventilator’</li><li>‘/m/04b_zc’: ’Respirator’</li></ul><p>Four economic topics:</p><ul><li><em>‘/m/06bmj’: ’Recession’</em></li><li><em>‘/m/0drqp’: ’Stock market’</em></li><li><em>‘/m/02gqxg’: ’Health insurance’</em></li><li><em>‘/m/08jvtd’: ’Travel insurance’</em></li></ul><p>Two topics related to imposed restrictions:</p><ul><li><em>‘/m/07l88z’: ’Lockdown’</em></li><li><em>‘/m/069q9’: ’Quarantine’</em></li></ul><p>Four topics related to remote work:</p><ul><li><em>‘/m/0zdp3l0’: ’Slack’</em></li><li><em>‘/m/0vpwj8v’: ’Google Hangouts’</em></li><li><em>‘/m/011c8m4f’: ’Zoom Video Communications’</em></li><li><em>/m/026wfg’: ’Skype’</em></li></ul><p>And four topics related to preparedness at home:</p><ul><li><em>‘/m/035bpb’: ’Delivery’</em></li><li><em>‘/m/09gtd’: ’Toilet paper’</em></li><li><em>‘/m/01kr41’: ’Mask’</em></li><li><em>‘/m/0b23px’: ’Hand sanitizer’</em></li></ul><p>The result is a series of plots that allow us to explore how search interest in corona-related topics has developed in tandem with new corona cases in different countries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_O0A2CeWtM4LWepYueVAtw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*23Bhv-94fIjMnpKff4MUrw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nxKQjwcwZR6p0b8MUKz0_Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4Kg1CEBCn6fc6kchw1stPQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gLKygC5o1LY510MJkfw5BA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g0n4fLY3XUDYcfmKa2jxLQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*04KoJ-Ig1evg6IQhCL7kbA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b5M4ULVzgIlxN6DpjVu-cw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*byy3T-t8LKxW-U1edAg0sA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rxXaCmCvVHOfZZd1PFEnFw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wGbPRkzQ2oODvOdCB8eN3w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QNSRraic0g2hCr1oDXQr7Q.png" /><figcaption>Search trends for a selection of corona-related topics in Denmark and new cases of corona/new deaths per day (click to enlarge).</figcaption></figure><p>Google Trends provides results as an index between 0 and 100 where 100 is peak search volume for the queried topic within the queried time period (e.g. peak interest for ‘Coronavirus’ within the 3 months prior to March 11, 2020) and 50 is half that search volume, 10 is one-tenth of it, and so forth. This allows us to compare searches for the same topic in different periods. For example, we can say that the search interest in ‘Hand sanitizer’ has increased by a factor of 50 in Denmark during the last week of February and the first week of March. But we cannot say that the search interest in ‘Travel insurance’ was higher in January than the search interest in ‘Hand sanitizer’, only that the search interest in ‘Travel insurance’ has increased by a smaller factor than the interest in ‘Hand sanitizer’.</p><p>In order to explore what might drive these search patterns, I define three time periods that are critical to understanding the history of the pandemic:</p><ol><li>The Wuhan period covers the time range from January 8, 2020, when the novel coronavirus is identified and the first death in mainland China is reported, over January 29, when the Hubei regional quarantine begins, to February 5, when the number of infected in Wuhan (capital of the Hubei region) exceeds 10,000. In the latter half of this period, we start to see reported cases outside mainland China but the main clusters of mass infection are still in China.</li><li>The Out-of-China period covers a time range from February 18, a couple of days before Northern Italy and parts of South Korea see a surge in domestic cases, over February 22, when 11 municipalities in Veneto and Lombardy go into quarantine, and February 24, when domestic cases surge in Iran, to March 1, when authorities in South Korea advice all citizens to stay home.</li><li>The Pandemic period covers the buildup to March 11, when WHO declares the situation a pandemic and the days following that declaration. It begins on March 2, when the ECDC increases the risk level from moderate to high for the entire European Union.</li></ol><h3>Search for ‘Coronavirus’ seems to be driven by an interest in events in neighboring countries, not by domestic corona cases</h3><p>While the 11 countries that experience peak search interest for ‘Coronavirus’ between January 8 and February 5 (China, Taiwan, South Korea, Thailand, Malaysia, Sri-Lanka, Vietnam, Cambodia, Indonesia, Mongolia, and the Philipines) are all in Asia, they are not all equally affected by coronavirus during that time period. In fact, Indonesia and Mongolia report no cases until March, Sout Korea report no cases until mid-February, and Sri-Lanka, Cambodia or the Philipines only report one or two cases in January and February. Yet, search interest in a country like South Korea is actually marginally higher during the Wuhan period in late January, than during its own outbreak mid-February.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O57ATxPEazdhw2H4O3cJOg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OdhRk1DBwENgQDzSvVk0Cg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JSfC4FGoXB1R_Prrnb2zoA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_R3Gqrl1CvfJM0meF3mQpg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8GCizblYb6MWrKVCsyI8TA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3M9-eJYs8uJe__w7lHSObQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5pfkLtRhEqFoCX4NEc0IDw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sEH0Fr1OPDJlSSXJZwjAkg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZTdveMJciNzJtpj7r8f0Bw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eQ-HqIT76RtL4rvwAPLxmg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kQf3iwJz_-dAbWwx2P9clg.png" /><figcaption>Countries where search volume for ‘Coronavirus’ peaks in the Wuhan period between January 8 and February 5, 2020 (click to enlarge).</figcaption></figure><p>A similar pattern is found for 10 of the 11 countries that experience peak search interest for ‘Coronavirus’ between February 18 and March 1. In Pakistan, Mexico, Jordan, Azerbaijan, Iraq, Iran, Afghanistan, San Marino, Kuwait, and Bahrain the surge in search interest precedes the first reported corona cases. This is the period where cases are starting to mount in countries outside of mainland China, for example in South Korea, Italy, and Iran where the first two cases are both reported and declared dead on February 19 and cases rapidly increase over the following days. Most of the other countries that experience a peak search interest in those days are found in the region and, with the exception of Iraq and Afghanistan, their surge in search interest is preceded by the surge in Iranian cases.</p><p>Japan is the only country where national search interest does not precede reported Corona cases in this period. Besides a high number of cases on board a quarantined cruise ship (not counted in the reported Japanese cases), Japan becomes one of the first countries outside mainland China to deal with the epidemic on a larger scale. The first case is reported already in mid-January, and travel restrictions are imposed in mid-February, but what really drives search interest to its peak is, in this case, when all schools and high-schools are closed on February 27.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1qIWnlMUMcKR0aYeT0VzJg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZOT8TqshM8iKT8i_DiBS2A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4LPwCfM-gRr3one5gE2v0Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uvgIha41pgl-YjCVNKEnhQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dG7W7x0jGPNWg3-VzsSYrA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yaWEkEeZWpbOivBFPraNdA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*imetu9EHPI2KYTQksOD90Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TNwX3eUkWcpbhnaKlg2FiA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F9oIyaYG6o4bWzUWRq-Aiw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8afU8yua2ZIoyH0_rglQJw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MgP280SSF4XJFubML-eQcQ.png" /><figcaption>Countries where search volume for ‘Coronavirus’ peaks in the Out-of-China period between February 18 and March 1, 2020 (click to enlarge).</figcaption></figure><p>As we move closer to March 11 when WHO declares the pandemic, this is when the majority of countries (82 in total) see their search interest in ‘Coronavirus’ peak. However, if we focus on the subset of these countries (27 in total) that see their sharpest weekly increase in search interest already in late February during the Italian, Iranian and South Korean outbreaks, we find the same pattern again: search interest precedes reported cases.</p><p>One of these countries is Italy itself, which, of course, sees a very significant spike in search interest during the early days of the outbreak in its Northern regions. But we also find a large group of European neighbors, such as Switzerland, Sweden, Norway, Netherlands, Liechtenstein, Germany, France, Croatia, Greece, Belarus, Latvia, or Lithuania, which all see search interest increase sharply during the Italian outbreak and prior to them reporting cases of their own. Similarly, we find countries in the neighborhood of Iran, such as Oman, Saudi Arabia, Georgia or Armenia which all see search interest increase during the Iranian outbreak and prior to them reporting cases of their own.</p><p>In Latin America, the first confirmed case is reported in Brazil on February 25, but search interest in Ecuador, Dominican Republic, Chile, and Brazil surges earlier and again seems to be driven by outbreaks elsewhere. Something similar could be true for Africa, although here the first case arrives in Egypt on February 14, which could in principle have prompted the increase in search interest in late February. The next case is not confirmed in Egypt until March, however, and in Algeria, Nigeria, and Morocco there are no domestic cases to explain the increase in search interest.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HKrop4ctAHgrsF3jl_efoA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ar_s8v7Ce28LzSO9I-SPyQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*njtqgxDWkitszGK85ixqbw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PV_Ga4NaRiXv2br-H470VQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nJueklAccfOZPVEZju-Pug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5ilzssWUXxiZO_vfHCh-FA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WjZnSXUdHGiZxEqd0l9GLw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EQx2vB1c3GTiP2KDSg7F5A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rsl8ThmgSvbdSQvfexEbUg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HvDoPcIKrSv7h6i6Dn04mw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YYYjWM3HBRC6tMUhauOFUA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_MhYSDQ7G7jSdhifN-hLxw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FtjSK2bNsEItg7tCHWQHrg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EXCaTCmuokOMIBMY0T7Fug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kWmhMhXY5M7EGb2mJX2rfg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DfoC3vMMmVAPqN6fj0fU-g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1CcYJUDFhbv2HSpLBY7JXA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s2D-BL2Fc3pjGJXFonWEVA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DgQiqUhXY6FoIXNL4YV4Iw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*22uFeVTbdIGvhppA60zycg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U3yKCs0tQAX5nGlsIYV5ag.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-HKPRERzUhTEQfbbkISi4Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pfY_8IzhRvoY4ywNSfW9_A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X9Sh96QGbj0uZjZbSdDw_w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4lTyqq5csYQ4sQVUy66jlA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ndaDSxx7KcxgjjX1Q9jBRA.png" /><figcaption>Countries where search volume for ‘Coronavirus’ peaks in the Pandemic period between March 2 and March 11, but where the sharpest weekly increase in search volume occurs in the Out-of-China period between February 18 and March 1 (click to enlarge).</figcaption></figure><h3>Search interest in ‘Hand sanitizer’ generally peaks later but is again not driven by domestic cases</h3><p>Only three countries see search interest in ‘Hand sanitizer’ peak during the Wuhan period from January 8 to February 5. South Korea and Japan experience this peak prior to the first domestically reported corona cases, whereas Vietnam experiences the peak following a series of reported cases. In countries like Taiwan and China, which one might expect to find in this group, the overall search interest in ‘Hand sanitizer’ for the entire period is simply too weak to detect a clear signal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I2v5sNBC8BP1y00scXcIgg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fE_Zgb0j-n9sS6BfQsA5Yg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FVWZcQP7bq2I-Fxry1D_Tw.png" /><figcaption>Countries where search volume for ‘Hand sanitizer’ peaks in the Wuhan period between January 8 and February 5, 2020 (click to enlarge).</figcaption></figure><p>Among the countries that see search interest in ‘Hand sanitizer’ increase as the epidemic spreads outside mainland China, only one, Thailand, has a prior history of domestic corona cases. In all other instances the search interest in ‘Hand sanitizer’ precedes the first reported corona cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MrtvyluFrPJ-1ocjXvfslQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yx5b1U7kbb0xDUW6QSzr0A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YB_dghpcSrqiSxkO4uOyuA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lvyB2JYaw7vn6jFaMeD6bw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rwo1pLCdNDo0RWKHsblIkQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wvqsc_VgF-ZwEq8UBuSRfg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_PGWNdJegS-R9A7TX6AwEg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dFmRlDsiedE0P8Kw1lPWuw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B0DuLtHiA55FGAEbOrT6cQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PTn0USsjBijI0-BR6IefFQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JJtfbvMJkdEz6HtILMlcnA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5W84mFnuba2GaE1kFBTTwg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*71DnLNDo8axGWbbjvSUbpQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4_bVOBpAx5m_2Si4oiOevQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q_sFvBea7JOlC3jD1AzBpA.png" /><figcaption>Countries where search volume for ‘Hand sanitizer’ peaks or increases sharply in the Out-of-China period between February 18 and March 1 (click to enlarge).</figcaption></figure><h3>Search interest in video conferencing software seems to be driven by domestic corona cases</h3><p>Based on what we have seen with search volumes for ‘Coronavirus’ or ‘Hand sanitizer’ we could speculate that Google users are always ahead of the curve, seeking relevant information based on what happens in neighboring countries and prior to outbreaks of coronavirus in their own countries. But this is not the case. If we follow search interest in topics related to remote work, such as video conferencing software like Skype or Google Hangouts, there is no clear change in the signal until the first domestic cases of coronavirus are reported.</p><p>At the current point of data harvest on March 11, 2020, most countries had yet to see any significant change in search volumes for ‘Skype’ or ‘Google hangouts’, but in the 21 countries that had seen a spike in ‘Skype’ searches and the 9 countries that had seen a spike in ‘Google Hangouts’ searches, there had already been reported corona cases when the spike happened.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-XjadFEY1q3EF6PfJjJ0aQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gz2VJBwFzXgR4EXxf1zZkQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KuA86L-GaoXDpLXsKat4Kw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fCv6Dk8MBFQVgY94GYinuw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kR6IP-QF-iknSrPA0enkmw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wJhaE2tFExTR4iFuIkUaCQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rCq93LqubtCFHTAEiuh_1Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lpl4E99VI7_5yKNJn9eVzQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DZRXML1mYLpbqyL6XSE7yw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n2nMq4sgaEShYhmhZAgI7g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_BaEGAbpFfWjwycPejPwxg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8goJdlrNPN2avzoW_40p7A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wGbPRkzQ2oODvOdCB8eN3w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i8FkcmaFSuuU86u3EW5wWQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fOTvkys2JVDEtmVa9HDdKw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Gpmqu-MseJBz5ywqYKbRw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nB-qdStf4aYWdpbxj9LPWw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HqVtUM0BmstiZ-r4qKwS5g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RWKzVOre89aMSgpovH7NTA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-M4R9IJ6q2yVpkZDChwMOg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p5YUz4wWWC9Jphessy1vAg.png" /><figcaption>Countries with a clear spike in search volume for ‘Skype’ (click to enlarge).</figcaption></figure><p>Indeed, most of the countries (Singapore, Italy, Romania, Ireland, Spain, Poland, and the Netherlands) that see a corona-related spike for ‘Google Hangouts’ also see a corona-related spike for ‘Skype’. The pattern seems to support that whereas a public in a country becomes interested in the virus itself, or in ways to prevent it, through stories from other countries, the same public only becomes interested in ways to mitigate the challenges posed by social distancing measures and lockdowns once those problems are acute and domestically present.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4dlz4mMu9YAf2mgMDaxyiw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R4dZjwCbkVyLw5HwPATh0A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9yU9p_NTckfGtOmxd8qDNw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IKfvkBNpalybpzHTmWlNGQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EbYRtx9H1kO35s24xTxd-A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hutQapOdHZm21oFUCMi9rQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XhF0QmW3U0okRvXXsrusZg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MHaAwKu0Pspat5qs-5NE-A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4qZJZnuCcW5G33SGCRH6Nw.png" /><figcaption>Countries with a clear spike in search volume for ‘Google Hangouts’ (click to enlarge).</figcaption></figure><h3>The interest in virus and prevention measures is sparked over longer distances than the interest in mitigation measures.</h3><p>We can substantiate that idea by comparing corona-related search peaks for ‘Medical ventilator’ and ‘Lockdown’ (see below).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YBuC_ROPsamZ1RGCawIDnQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WEx_1hFj1OYUDW-WxZfrug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*34ez3hFKFpn1GYYOQ37FnQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GKjuBUXFRa6ed7KH96fdxg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*apvZ6hziKnwo7e8_v-ieaw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DKWGfnI7AVcQxfBGne8cJQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*28kCsW_lXsqVRsZqlVUaxQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PLoqXtgO9cWNAp42vhKxmw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BDzy7aUxMXKRYqRUhAZUNw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qTTzpD6VzJ1mhpjjZ4HaoA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dkfwl5dtZVHk_eWHTUKSMQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*baDIQu6tvdziGvaiTxAWpg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4ze6hcxKOCOAPKcO6eZL1w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D1xJDXzihNmYmd0qC6jtEQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pz1kHTHQvigHe2QAZ3Wg0A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rvKYBFOprIKLRNkEWPynkQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_Vd6-1JAG0zW6_Wq2KmHgQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2OqFf6fPZ5ToSYF8loMMCg.png" /><figcaption>Countries with a corona-related spike in search volume for ‘Medical Ventilator’ (click to enlarge).</figcaption></figure><p>Whereas many countries are capable of becoming interested in medical ventilators well before they experience any domestic corona cases, this is almost never the case for their capacity to become interested in lockdowns. Although some countries do experience a modest rise in search interest during the Wuhan or Italian lockdowns, it is nothing compared to the rise they experience once the first domestic corona case has been reported.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-FZT8zo_ByXV2LZ6-tTuoA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ylzt9EXTq_M10Xw2ymJKaw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*weqe9rXV1FgYnIUyn52b0A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kaKiOuIUOhHHHX8lbuS43A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yFlILI2iYXttbs0lSlcwvw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rDxt3SsTKT9ggQP_nLL6Nw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iQSA8Q8MtE81UCauYIZL7w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GEDNkXeojWOXuQXYAZeJ4Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GLRDKl2NiYeD_bKabvU33w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CxsuJq5IpZAwi3tS5MemxA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Icajy4VQI6kTeC4miIYnA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GmD4yCNX685bl-GM58b0ww.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RIBGt3EW0igzL2RGYr3USg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R3ZAxJ3Iibdht1JFHrIEoQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7bpwoCMyzD-5uvxamnmEjQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CeAKvooYKZ4ctYta8eKtyA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IlhXDgf0cPK7V_cBjcDK0A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2vKq-lDc2NJhiojHCIqJLA.png" /><figcaption>Countries with a corona-related spike in search volume for ‘Lockdown’ (click to enlarge).</figcaption></figure><p>So, it seems that interest is sparked long-distance not just for the disease itself, but for the medical challenges it poses (such as a lack of medical ventilators). But this is not true for the more practical and everyday circumstances of living with the epidemic (such as dealing with a lockdown of public institutions or transportation). In the latter case, interest seems to be sparked rather short-distance by the fact that corona is breaking out domestically.</p><p>Across all of the 24 search topics I have analyzed here, this is seen nowhere more clearly than in the search volumes for ‘Toilet paper’. 32 experience a corona-related spike in search interest for this topic, but only two of them (the Philipines and South Africa) have no confirmed corona cases at the time the spike happens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LI_H8QnBcGnuTfYi11Ff3A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y3gytk_7PGJCDrhgVj5o1A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yx2yunOz1D69KcjR2K8qDQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*siRI6IEgdy0kp9RdNSytwg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RQP4e1axdpXesThjQSsfUg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PNpVgYglIVQsPLnlifX2Qw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vDAFbSEWQtA2MUWPgGWMUg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OT-n87v296rqr4kD7tBK5g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g0n4fLY3XUDYcfmKa2jxLQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pYPOV3jaDbOoYt42g7y_Pg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DOnaUyPmXPEZ3PHcxHxu8w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZmC1nwjOx6eh9zcvZCnHjg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wZPXkB6doZ87GsL9I1kO4Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7MVcZNDU11YXDTQuLP26gg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SLG93SVjsAbd02kh1eG5sQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZOOeKmYYeHoSN2B-PHZ-3w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S42QSDQykq-C-KiS45YEAg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SUq-y01IP9LrYoCNof5qyw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DvQ6_uKB7rVhrONI2Y1Rzw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UTrKVRpW73D9O-eK5ttyqg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ieNZJwce_1OH_r3h_F9MLA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TvWBcL-AKfUjkcsh_tXk4A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7VPCy02TeZTqUHJx6IpL0Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6H9xpn_-MuZ7PQs4jbLkXQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JqyGFne7eAYuTUE-nzfiyA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-PJJjmjsT9pfJe-F8T046g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1S_sA0j9tLlpp824Hi0LQQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-bSqn317cUQsIbdqVkd5Eg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jHxgUF1JLoixKNMAiIhj8g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*93DebRH4cAa4xqpqOQNu3A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NQoQoG4ZvoCKNSwJVaGL1A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Upk9JkliVjcIsSVKqTn7bg.png" /><figcaption>Countries with a corona-related spike in search volume for ‘Tolet paper’ (click to enlarge).</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf2051ea7fe0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to natural language processing with Cortext]]></title>
            <link>https://medium.com/@EthnographicMachines/introduction-to-semantic-analysis-with-cortext-19f355b7289a?source=rss-8838c1d8bfac------2</link>
            <guid isPermaLink="false">https://medium.com/p/19f355b7289a</guid>
            <dc:creator><![CDATA[Ethnographic Machines]]></dc:creator>
            <pubDate>Wed, 20 Feb 2019 10:37:25 GMT</pubDate>
            <atom:updated>2020-02-16T13:32:24.230Z</atom:updated>
            <content:encoded><![CDATA[<h4>Tutorial 06 in a series on Controversy Mapping</h4><p>In this tutorial, we will go through the steps of uploading a dataset to Cortext, extracting significant noun phrases, and mapping these noun phrases as networks based on their co-occurrence in documents. Cortext is a piece of software that allows you to do natural language processing: computationally reading and comprehending text. It is useful if we do not know which words to search for, but want to know, bottom-up, what kind of language a corpus of documents is using.</p><p>We will use<a href="https://drive.google.com/open?id=1vk4cFsLKMLcx3zX9QFOn9nizmbfVjkxd"> a dataset of full-text content from our 173 circumcision related articles </a>on Wikipedia <a href="https://drive.google.com/open?id=1Po3nqPR5ugZth6Wz9G5ovW1j9Tggr3aN">produced with this Python script</a>. Before we start, you will need to run that script and set up a Cortext account. Go to the <a href="https://managerv2.cortext.net/">Cortext Manage</a>r to do so.</p><h3>Preparing and uploading your dataset</h3><p>The easiest way to get text into Cortext is in a spreadsheet format. Cortext will automatically consider each row as a separate document. In the example below I use the CSV export from the script above. Each row is a Wikipedia article from the ‘Circumcision’ category. Column ‘B’ contains the full-text for the article. The other columns contain various kinds of metadata which will be useful for analysis at a later stage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AQYKr8TlVLUeU0aembKFWA.png" /></figure><p>After dragging and dropping the file into Cortext I can click “Accept &amp; Upload”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a-gFMVGARr_4XefhRvVaWQ.png" /></figure><p>Cortext will ask you how to parse the uploaded dataset. Choose “csv” from the “Corpus Format” dropdown and tick “standard csv separated by ; and minimal quoting” (in other cases you might have to use “default csv” or “tab separated” depending on how your csv file is separated). Leave the other settings in their default positions and click “start script”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OLM1eUVOhmuG3cxWwebQlg.png" /></figure><p>The .csv file will now be indexed as a database (.DB) in Cortext. If all goes well you should be able to see a green flag that says ‘finished’ next to the indexed database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0FWLp7YDGNuZA452gbWKnA.png" /></figure><h3>Terms extraction</h3><p>We can now proceed to extract noun phrases from the abstracts in the dataset. To do so, launch the ‘Terms extraction’ script. You should be able to see the column headers from the original spreadsheet as textual fields. Select ‘Page_text’ (or whatever is the name of the column containing your text) to specify that this is where the text from which you want to extract terms is located. Furthermore:</p><ul><li>Leave ‘Minimum frequency’ at 3. This sets a minimum threshold for terms to be included (namely terms that are found at least 3 times in the dataset). In very large datasets you may try to increase the minimum frequency further.</li><li>Leave ‘List length’ at 100. This determines how many terms the algorithm will extract. After having applied the minimum frequency criteria, it will select the 100 terms with highest specificity core. High specificity means that a term has a very uneven distribution across the documents in the corpus.</li><li>Leave ‘Maximal length’ at 3. This simply means that terms cannot be longer than 3 words (trigrams). You may at a later point decide to go back and increase the length to 4 or 5 if you have many terms broken abruptly into 3-word multi-terms.</li><li>Finally, select ‘yes’ for ‘Lexical extraction advanced settings’. This will open a sub-menu where you can switch the frequency computation from ‘sentence level’ to ‘document level’. Do so. This will ensure that your minimum frequency is counted as occurring at a minimum in 3 documents, not sentences.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vDzUDLJoJKYHggv7Gg02gQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FQgBHwN3ZAMRVx94PcqLow.png" /></figure><p>When you are done with the settings, run the script. Again you should be able to see a green flag when it is done. You can click the eye icon next to the results file to see the list of extracted terms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a5pxagiQeidkw3zlRMy0iQ.png" /></figure><ul><li>The ‘Forms’ column shows you the different versions of a noun phrase that Cortext has decided to consider as one term.</li><li>The ‘Stem’ column shows you the root of the noun phrase that will be used to recognize different versions of it.</li><li>The ‘Main form’ column shows you how the noun phrase will be labeled on your maps.</li><li>The ‘Occurrences’ column shows you how many times a term is found in the dataset. Remember that you asked the script to count occurrences on the document level. The occurrence count, therefore, reflects how many documents a term is found in. You can sort the column by clicking on the header.</li><li>The ‘Cooccurrences’ column shows you how many times a term is found together with other terms. Cooccurrence is defined as being found in the same document. You can sort the column by clicking on the header.</li><li>The ‘Specificity chi2’ column shows you the specificity score for each term. The specificity score is a statistical measure of how a term is distributed across the documents in the dataset. If a term is very <em>evenly </em>distributed across all documents (i.e if it occurs with the same frequency in almost every document) it gets a low specificity score. If a term is very <em>unevenly </em>distributed across all documents (i.e. if it occurs with high frequency in a particular subset of documents but not at all in other documents) it gets a high specificity score. In the latter case, we say that a term as a biased distribution. Eventually, Cortext is geared towards selecting these terms as important, the assumption being that we already know the theme of our dataset and thus its generic language and that the reason for employing semantic analysis is to discover discursive differences below this generic level. If we were to run the ‘Terms extraction’ script again but expand the list of extracted terms to 200 or 300, this expansion would follow the ranking of terms by specificity. Rather than the top 100 most specific terms in the dataset, we would get the top 200 or top 300 most specific terms. You can sort the column by clicking on the header.</li></ul><h3>Map co-occurrence networks</h3><p>When we have a list of extracted terms we can map how they occur together in our documents. To do this, we have to produce a co-occurrence network. Select the “Map Heterogeneous Networks” and select ‘Terms’ in both ‘First Field’ and ‘Second Field’ under ‘Nodes Selection’. This will produce a network of terms connected to terms through co-occurrence in the same documents. Set the ‘Number of nodes’ to 100, since you have only extracted 100 terms. Run the script.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-6jBJ-vjLH7ombxSALWlPg.png" /></figure><p>When the script is finished you can click the eye icon in the “Map explorer” to navigate and interact with the resulting network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AL1l1QKwwBjUDh7yYQA8DA.png" /></figure><p>You will notice that the network is very well clustered. This is partly due to the fact that Cortext has already selected terms with high specificity (they will, as a function of their biased distribution in the set, co-occur much more with some terms than they do with others), but it is also due to the fact that the network mapping imposes an edge filter that ‘punishes’ weak edges between otherwise well-connected nodes. Every edge between two terms has a weight that is determined by the number of documents in which the two terms co-occur. This weight is normalized on the degree to which both terms tend to co-occur with all other terms in the dataset (their global co-occurrence). If two terms are very likely to co-occur, they are thus obliged to have a relatively heavier edge between them for that edge to be included in the network. To explore what this means you can try to run the network mapping script again, but this time, under “Edges”, select “no” for “Automatically define the edges” and switch “Proximity Measure” from “distributional” to “raw”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5QxYudZXoAtJPMLGyMcpZg.png" /></figure><p>As you can see, the same network without edge filtering is considerably less clustered.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DOCkcLwzsupBeTgFzgvstg.png" /></figure><p>Eventually, you should try to extract more terms and map them as co-occurrence networks (with the distributional edge filtering on) to see when the gradual addition of less specific terms disrupts the clustered nature of the network. Below I try first with 300 terms and then 500 terms. Around 500 terms the clusters are no longer retained.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TiX4BMgAhl9tm5_tsDRizA.png" /><figcaption>Co-occurrence: 300 terms</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cHirhoTRcIpYsdKWJeQuWg.png" /><figcaption>Co-occurrence: 500 terms</figcaption></figure><p>We can interpret these clusters as discourses. They effectively mean that a group of articles tend to use the same words together that other articles are not using together. There is still a male / female distinction between these clusters, but the fact that we extract noun phrases bottom-up from the text rather means that we can see a more nuanced landscape of sub-topics. In the visualization below we have exported the results from Cortext to Gephi and annotated the resulting network to show this structure:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Et3qeOyOfNWBbK-qlNVa6Q.png" /><figcaption>Example of an annotated co-word network based on semantic analysis of full text from Wikipedia pages.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19f355b7289a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Telling stories with data]]></title>
            <link>https://medium.com/@EthnographicMachines/mapping-controversies-hand-in-1-d3ec9f1d0dc0?source=rss-8838c1d8bfac------2</link>
            <guid isPermaLink="false">https://medium.com/p/d3ec9f1d0dc0</guid>
            <category><![CDATA[wikipedia]]></category>
            <dc:creator><![CDATA[Ethnographic Machines]]></dc:creator>
            <pubDate>Thu, 14 Feb 2019 10:30:52 GMT</pubDate>
            <atom:updated>2020-02-16T09:58:14.960Z</atom:updated>
            <content:encoded><![CDATA[<h4>Tutorial 05 in a series on controversy mapping</h4><p>In this tutorial series, we have used a number of scripts that call the Wikipedia API, crawl Wikipedia, and scrape Wikipedia. We began by selecting a Wikipedia category that covers our topic. We used <a href="https://drive.google.com/open?id=1osTL_sQ608t8kVI4PsTVUjfOhlw0upWK">a script to call the API and ask for all pages that are members of this category</a>. We allowed the script to crawl the subcategories inside the seed category, making API calls to ask for all pages that are members of the subcategories and all pages that are members of the subcategories of subcategories. This produced an initial dataset of 173 pages from the Circumcision category and its subcategories on Wikipedia.</p><h3>Visualize your protocol</h3><p>In order to tell that story, let us try to visualize the protocol. Here is the initial step:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/630/1*MPRr7vMgmWM2f6bedmBBHA.png" /></figure><p>We then used the 173 pages as input for a series of scripts that harvest different kinds of relationships between the pages construct networks to represent them. The list below provides an overview of those scripts, as well as some additional possibilities, and gradually expands the visual protocol:</p><ul><li>A <a href="https://drive.google.com/open?id=1ZvkmrNPPnChFpF_Opwe8UpfIKPaisvYW">script that calls the API to retrieve all links to Wikipedia pages from each of the pages found in a Wikipedia category </a>(in our case the 173 pages from the Circumcision category). The script produces two networks (output as .gexf files): In the <em>first </em>network (‘AllLinksNet_membersonly’ ) the nodes are the pages from the category and the edges the links between them. In the <em>second</em> network (‘AllLinksNet_allpages’) the nodes are the pages from the category (these nodes are tagged with ‘level0’, ‘level1’ and ‘level2’ depending on where they are found in the subcategories of the category) AND all the pages that are not in the category but are still linked to by the pages in the category (these nodes are tagged ‘non member’). The edges are the links between the member pages and the links from the member pages to the non-member pages.</li><li>A <a href="https://drive.google.com/open?id=1bNfMC6ZFRAUDZlKJF5_0nfI02YgZYI35">script that calls the API to retrieve the HTML from each of the pages found in a Wikipedia category and scrapes only the links found as part of the text</a> (in our case the 173 pages from the Circumcision category). The script produces two networks (output as .gexf files): In the <em>first </em>network (‘InText_LinkNet_membersonly’ ) the nodes are the pages from the category and the edges the IN-TEXT links between them. In the <em>second</em> network (‘Intext_LinkNet_all’) the nodes are the pages from the category (these nodes are tagged with ‘level0’, ‘level1’ and ‘level2’ depending on where they are found in the subcategories of the category) AND all the pages that are not in the category but are still linked to by the pages in the category (these nodes are tagged ‘non member’). The edges are the IN-TEXT links between the member pages and the IN-TEXT links from the member pages to the non-member pages.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_EJ5aS887dEyTjKiCZPWTQ.png" /></figure><ul><li>A <a href="https://drive.google.com/open?id=1m7unIJJgyRZGd81oLdCmprbSjiQG4mJh">script that calls the API to retrieve all the literature references to pages outside Wikipedia from each of the pages found in a Wikipedia category </a>(in our case the 173 pages from the Circumcision category). The script produces one network (output as .gexf file): The nodes are the pages from the category and the edges represent the number of external references that are shared between two nodes. If more references are shared, the edge weight increases.</li><li>A <a href="https://drive.google.com/open?id=15Za4pcw8F6eFxrxf0WIDbz0zuOO19glu">script that calls the API to retrieve the editors that revised each of the pages found in a Wikipedia category</a> (in our case the 173 pages from the Circumcision category). The script produces two networks (output as .gexf file): The <em>first </em>network is monopartite (one type of nodes). The nodes are the pages from the category and the edges represent the number of editors that are shared between two nodes. If more editors are shared, the edge weight increases. The <em>second</em> network is bipartite (two types of nodes). The nodes are 1) the editors that revise pages and 2) the pages found in the category. The edges from an editor to a page represents the fact that the editor has been revising the page.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j0ZWvhV4pgbcH2F-2jRxAw.png" /></figure><ul><li>A <a href="https://drive.google.com/open?id=1XGE9EeOz-A3428RlDxNyrh8Og1DocmdL">script that calls the API to retrieve the full text for each of the pages in the category</a> (in our case the 173 pages from the Circumcision category). The script produces a spreadsheet (output as .tsv file) with one column containing the names of the pages in the category and column containing the text of the page. This file will be necessary when we do semantic analysis.</li><li>A <a href="https://drive.google.com/open?id=1jhGKe-DSwMLUgwf0_9khYxHwkwYRqDRG">script that calls the API to retrieve the full text for each of the pages in the category and queries that text for the presence of a set of search terms specified by the user</a> (in our case the 173 pages from the Circumcision category + the search terms ‘muslim’ and ‘jewish’). The output of the script is a spreadsheet (output as .csv file) with one column containing the names of the pages in the category and the other columns containing the occurrence count for each of the search terms. This file can be merged with a network in Gephi.</li><li>A <a href="https://drive.google.com/open?id=1NxkhXkJ8UMAwnBdWM8gmbtaa5zCAT8WW">script that calls the API to retrieve the revision history for each of the pages in the category or a smaller set of user-specified pages</a>. The output is a dataset for producing a timeline.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yHu9kbkftgNY_THiBI-VOg.png" /></figure><p>Eventually, being able to visualize your data protocol (or at least describe in a clear and concise manner to your readers) is a key part of any digital methods project in general, and all controversy mapping projects in particular. Including a visual diagram of your protocol is, therefore, advisable. <a href="https://docs.google.com/presentation/d/1j3g6Iq3c7NJ7vjJi2lWZ22n41JkhocyGstoWj_D320w/edit?usp=sharing">The ones above are simply done with a template in Google Slides</a>. You can edit your own. The layout is up to you. Below is a simpler alternative with some icons included to emphasize the different kinds of data outputs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G256c3C8MwQgLluBNS2tIQ.png" /></figure><h3>Annotate your visualizations</h3><p>In <a href="https://medium.com/@EthnographicMachines/visual-network-analysis-with-gephi-d6241127a336?source=friends_link&amp;sk=64f68697a7df91a2b2f505e57beca7f0">Tutorial 03</a> we covered how to do a basic visual network analysis of Wikipedia articles about circumcision connected by direct hyperlinks. We concluded that there were two distinct thematic groups of articles about male and female circumcision respectively. This finding easily be annotated directly on the network:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*runBD3mIY1BIb2UZ00pSUA.png" /><figcaption>Example of a network of Wikipedia pages with annotated clusters.</figcaption></figure><p>And once we have that visual model, a network where the location of a node in a specific cluster can be taken to mean something, we can project other kinds of information onto it. For example, in the case of circumcision, we might be interested in different religious beliefs. Below are three examples of the same network, with the same annotated clusters, but with nodes sized by the degree to which they mention ‘christianity’, ‘judaism’, or ‘islam’. It allows us to tell the story that christian and judaic circumcision practices are mainly a male issue, where as islamic practices are both male and female (hardly surprising, but nonetheless a story!).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SdrboEeTjom1IaFBplJUDA.png" /><figcaption>Example of the same network of Wikipedia pages (with similarly annotated clusters) displaying the distribution of 3 selected issues (note that your version should include 6 selected issues).</figcaption></figure><h3>Scraping pages for keywords and adding the results as node attributes in Gephi</h3><p>In order to tell that story, we first need to search the circumcision articles for the presence of different keywords. <a href="https://drive.google.com/open?id=1jhGKe-DSwMLUgwf0_9khYxHwkwYRqDRG">This Python script</a> allows you to search for mentions of a keyword on a set of Wikipedia pages (you will need the <a href="https://drive.google.com/open?id=1QWgZ5Bx3hqPd_M415PKjqyPARBw6wJ68">.json result file from the scrape of category members</a> as input).</p><p>The script produces <a href="https://drive.google.com/open?id=1K8qMrAq-L7UlI6pqIr6-hGmZDfJpYRslsGXLWjjZq74">a .csv output (in this case I have queried members of the Circumcision category for mentions of terms like ‘Muslim’ or ‘Jewish’)</a>. If I want to use that information in Gephi, I need to import the .csv file into a project with a network of the same nodes. In this case, I have a network opened already with member pages from the Circumcision category connected by in-text links. If I switch from ‘Overview’ to ‘Data Laboratory’ (very top above the workspace tabs) I can see the list of nodes in my network displayed as a table. Currently, there are no attributes except for the ‘Modularity Class’ I computed for coloring earlier in this tutorial.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oLQtpu0BpT2EoOhpXKmRSA.png" /><figcaption>The Data Laboratory showing a table of nodes.</figcaption></figure><p>To import the .csv with keyword mentions, click ‘Import Spreadsheet’ in the bar above the nodes table. Select the .csv file from the right location on your computer. We now have to go through 3 steps to complete the import. <em>First</em>, some general CSV options will appear. You just have to check that the rows and columns are parsed correctly. Below I can see by keywords figuring as independent rows with a column for each page. That looks right. Click ‘Next’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U5nN8ZpAIM5KfCoz0OE_5w.png" /><figcaption>CSV import, first window.</figcaption></figure><p><em>Second, </em>I need to check if the values are parsed correctly. Since my values are numbers (counting mentions of a keyword for each page) the values should be parsed as integers (not strings). This also looks right. Click “Finish”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LJtcL8TJnHfaxaIgyu_Szg.png" /><figcaption>CSV import, second window.</figcaption></figure><p><em>Third, </em>I need to make sure that the import is appending the .csv to my existing network rather than opening the .csv as a new workspace. This is not the default setting. Therefore, I have to change the tick-box to ‘Append to existing workspace’ before clicking ‘OK.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k8kUTiXu8sDlJS32Yx09NA.png" /><figcaption>CSV import, third window.</figcaption></figure><p>When I inspect the nodes table in the ‘Data Laboratory’ I should now be able to see my existing nodes, still with the ‘Modularity Class’ column intact, but also with the new columns for keyword counts that I just imported. In this particular case, if I scroll to the bottom of the node list, a few nodes have been created that were not in my network already. I can spot them because they do not have a ‘Label’. The reason for this redundancy is the fact that my network only includes those pages from the Circumcision category that have in-text links to other pages from the Circumcision category. My scrape for keywords, however, return results on all pages from the Circumcision category, including some that are not linked in my network. We can select these redundant nodes on the table, right-click the selection and delete it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g2NrmALaRiN7v1u_KGY_ng.png" /><figcaption>Deleting redundant nodes in the Data Laboratory after .csv import.</figcaption></figure><p>When we switch back to the ‘Overview’ from the ‘Data Laboratory’ we will now have some new options available in the ‘Appearance’ tab. Below I first size the nodes by the degree to which they mention the term ‘Jewish’ and then color them by the degree to which they mention the term ‘Jewish’. In both cases, I click the ‘Spline…’ option in the bottom left corner of the ‘Appearance’ pane before applying sizes or colors. This allows me to use a different (non-linear) distribution of the mention counts when I visualize them. Given that many pages mention ‘Jewish’ once or twice, whereas a few mentions ‘Jewish’ more than 20 times, and given that I am interested in making all mentions visible, I choose a distribution that gives higher weight to the smaller mention counts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*16iCMUrmk8Y1BJR4gXSDSQ.png" /><figcaption>Sizing nodes by mentions of the term ‘Jewish’</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pgh_ddwaP9oXE_EWeSkjqw.png" /><figcaption>Nodes sized by the degree to which they mention ‘Jewish’.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BD1rWxko2PbvLPbwOZCWmw.png" /><figcaption>Selecting a non-linear spline for node sizes.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pTCyrAiz2KqsOyrn7jw63g.png" /><figcaption>Nodes resized with a non-linear spline to give more equal visual presence to nodes that only mention ‘Jewish’ once or twice.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RVx_-6ggy-OxnOPUxtLxLw.png" /><figcaption>Coloring nodes by mentions of the term ‘Jewish’ (also with a non-linear spline)</figcaption></figure><h4>Exporting and annotating your visualization</h4><p>While the ‘Overview’ and ‘Data Laboratory’ views in Gephi are useful for exploring and manipulating your graphs, ‘Preview’ is where you will be able to export images of your graphs. For example for display on your blog post.</p><p>Below we use the following presents (left pane) to make our network legible (these are just suggestions, choose a style that suits you!):</p><ul><li>Nodes: set ‘Border Color’ to white.</li><li>Nodes: set ‘Opacity’ to 70</li><li>Node Labels: tick ‘Show Labels’</li><li>Node Labels: untick ‘Proportional size’</li><li>Edges: set ‘Opacity’ to 70</li><li>Edges: untick ‘Curved’</li></ul><p>Click ‘Refresh’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zdgVdBZVD1Q8lABy6r5Geg.png" /><figcaption>Changing the presets in ‘Preview’</figcaption></figure><p>You can now click ‘Export’ at the bottom of the ‘Presets’ pane on the left. Choose a .png export and in ‘options’ go for a resolution of 3500 x 3500. You can use can now annotate this .png file to make, for example by highlighting or describing clusters below is an example of how this can be simply done in Google Slides (see the Slides <a href="https://docs.google.com/presentation/d/1j3g6Iq3c7NJ7vjJi2lWZ22n41JkhocyGstoWj_D320w/edit?usp=sharing">project here</a>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H45AWZI44UhSzlp_I1EZXA.png" /><figcaption>Annotating a network .png in <a href="https://docs.google.com/presentation/d/1j3g6Iq3c7NJ7vjJi2lWZ22n41JkhocyGstoWj_D320w/edit?usp=sharing">Google Slides</a>.</figcaption></figure><p>Now that we have a network with annotations that help the reader make sense of the clusters, we can consider overlaying that space with additional information. In the example below I use a simple keyword search in order to show how much a given Wikipedia page talks about a given issue (in this case I compare ‘Christianity’, ‘Jewish’, and ‘Muslim’). I have imported the .csv with the result file for the keyword search into Gephi and I have sized and colored my nodes by the occurrence count for a specific keyword (<a href="https://medium.com/@AKMunk/mapping-controversies-with-digital-methods-scrapers-crawlers-apis-17e0c96c340a">see how in Tutorial 02</a>). The first thing I want to ensure is that I only show labels for the nodes that actually talk about the issue I am visualizing. I begin by selecting all the nodes on the nodes table, right-clicking them to select ‘edit all nodes’, and then un-tick the ‘Label Visible’ option in the ‘edit’ pane on the left. I then sort the nodes on the ‘christianity’ column so that I have the nodes that mention ‘christianity’ at the top of my node list. I can now select these nodes and tick ‘Label Visible’ back on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LrECSsl-BC7Xi1CuK_pKbA.png" /><figcaption>Making labels visible only for nodes that mention ‘christianity’ in the Gephi Data Laboratory.</figcaption></figure><p>I now switch back to the ‘Preview’ and select the following presets:</p><ul><li>Nodes: set ‘Border Color’ to white.</li><li>Nodes: set ‘Opacity’ to 50</li><li>Node Labels: tick ‘Show Labels’</li><li>Node Labels: untick ‘Proportional size’</li><li>Edges: set ‘Opacity’ to 70</li><li>Edges: untick ‘Curved’</li></ul><p>Click ‘Refresh’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xjuz8C8zsLl5bK3kzC2v9w.png" /><figcaption>Changing the presets in ‘Preview’</figcaption></figure><p>I can now, again, export a .png (remember to set the resolution to 3500 by 3500) and bring it into <a href="https://docs.google.com/presentation/d/1j3g6Iq3c7NJ7vjJi2lWZ22n41JkhocyGstoWj_D320w/edit?usp=sharing">Google Sheets</a> for annotation. Since this objective of this visualization is to <strong>compare the distribution of different keywords across the same network </strong>I will place all the images next to each other on the same visualization. I will also retain, as much as possible, the annotation of the clusters in the network to make it as easy as possible for the user to interpret what different distribution patterns might mean (in this case that Judaism and Christianity are considered issues in relation to male but not female circumcision, whereas Islam is considered an issue in relation to both).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8jQ7GY-qvSZoD5xII62HWw.png" /><figcaption>Making a visual comparison of different keywords projected on to the same network in <a href="https://docs.google.com/presentation/d/1j3g6Iq3c7NJ7vjJi2lWZ22n41JkhocyGstoWj_D320w/edit?usp=sharing">Google Sheets</a>.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d3ec9f1d0dc0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Visual network analysis with Gephi]]></title>
            <link>https://medium.com/@EthnographicMachines/visual-network-analysis-with-gephi-d6241127a336?source=rss-8838c1d8bfac------2</link>
            <guid isPermaLink="false">https://medium.com/p/d6241127a336</guid>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Ethnographic Machines]]></dc:creator>
            <pubDate>Tue, 12 Feb 2019 10:18:07 GMT</pubDate>
            <atom:updated>2020-02-16T10:16:07.913Z</atom:updated>
            <content:encoded><![CDATA[<h4>Tutorial 03 in a series on controversy mapping</h4><p>In this tutorial, we will cover the basics of doing a visual network analysis in Gephi. We will use the <a href="https://drive.google.com/open?id=1qt3RJhfNFPU7hkz4zdXPnt__H1qgGqX_">list of Wikipedia articles from the Circumcision category</a> harvested in <a href="https://medium.com/@AKMunk/mapping-controversies-with-digital-methods-scrapers-crawlers-apis-17e0c96c340a">Tutorial 02</a> as a point of departure and consider different ways to think about and work with these articles as networks.</p><p><a href="https://gephi.org/">Gephi</a> is a piece of open-source software that lets you visualize, manipulate and explore networks. A number of great introductions to the software are already available, such as <a href="http://www.martingrandjean.ch/gephi-introduction/">this one</a> or <a href="https://gephi.org/users/">these</a>.</p><p>Networks (a.k.a. graphs) can be stored in a variety of file formats (for example .gexf, .gdf, or .gml) but all of them consist of a list of <em>nodes</em> and a list of <em>edges</em> connecting the nodes. Visually speaking the nodes are the dots and the edges the lines between the dots, but nodes can represent any kind of entity and edges can represent any kind of relationship between two entities. Depending on the file format, nodes and edges can also have a variety of attributes associated with them (e.g. color, category, weight, etc).</p><h3>Building different networks from Wikipedia articles</h3><p>So, in order to build a network, we need to think about ways in which articles on Wikipedia can be said to relate to each other. The most obvious way is the direct hyperlinks that editors of an article provide to other articles on Wikipedia when relevant. In the screenshot below the editors of the ‘Circumcision’ article have decided to make in-text references to the articles about ‘foreskin’, ‘human penis’,’glans’,’circumcision device’, ‘anesthesia’, ‘physiologic stress’, ‘elective surgery’, and so forth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4u3hyKmxk8cReC2hjyumPQ.png" /><figcaption>Two types of hyperlinks in the ‘Circumcision’ article on Wikipedia, namely 1) in-text links to other Wikipedia articles (highlighted words in the text) such as ‘foreskin’, ‘human penis’ or ‘glans’, and 2) external references to relevant sources and literature (highlighted numbers in superscript and hard brackets).</figcaption></figure><p>Based on these hyperlinks to other Wikipedia articles, we can build a network where the nodes are articles like ‘circumcision’, ‘foreskin’, or ‘anesthesia’, and the edges are direct references between them. Below is an example. Two distinct clusters are clearly visible suggesting that articles in the ‘Circumcision’ category are thematically related in two distinct ways. We can draw that conclusion because we have constructed the network in such a way that an edge between two nodes reflects a thematic relationship between two articles (the link to the ‘foreskin’ article from the text in the ‘circumcision’ article has been put there by the editors because it is thematically relevant). These two groups of thematically related articles revolve around male and female circumcision (a.k.a. female genital mutilation) respectively. Rather than being part of the same issue, the debates about male and female circumcision seem to be associated with different people, different problems, and different events.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*igb9C9-r1scvSh9GPpsSHA.jpeg" /><figcaption>A network of articles from the ‘Circumcision’ category on Wikipedia <strong>connected to each other by hyperlink</strong>s. Nodes colored by Louvain modularity; graph layout by ForceAtlas2.</figcaption></figure><blockquote>Question: Can you explain why the network of Wikipedia articles connected by hyperlinks (above) shows us how these articles are thematically related?</blockquote><p>You can build the network yourself using <a href="https://drive.google.com/open?id=1ZvkmrNPPnChFpF_Opwe8UpfIKPaisvYW">this Python script</a> (open as a notebook in Jupyter) which calls the Wikipedia API and asks for all links from a list of articles (you will need <a href="https://drive.google.com/open?id=1QWgZ5Bx3hqPd_M415PKjqyPARBw6wJ68">this .json file with the results of the category scrape</a> from tutorial 02 as input). The script outputs two networks:</p><ol><li><a href="https://drive.google.com/open?id=1eiVQdVl9Se2nZ0YFn6WH6HueJu7FsfFs">A .gexf file with articles from the ‘Circumcision’ category connected to each other by wiki-links.</a></li><li><a href="https://drive.google.com/open?id=1xoDhh0kG7wIDD8k9NuVqwtN_MQC6bkXM">A .gexf file with articles from the Circumcision category connected to each other AND TO ALL OTHER ARTICLES THEY CITE by wiki-links.</a></li></ol><p>We will get back to how to open and visualize these .gexf files in Gephi. But before we get to that we should ask ourselves if the most interesting relationship to map between a large number of Wikipedia articles is always their hyperlinks to each other? What if, for example, we were interested in the sources they reference in support of their claims rather than the articles they point to for further reading? In accordance with Wikipedia’s editor guidelines, each article is equipped with a set of references to external sources (provided as footnotes with a reference list at the bottom of each article).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-i6ojf0yQ5OdPkggG4_6WQ.png" /><figcaption>Example from the reference section of the ‘Circumcision’ article on Wikipedia.</figcaption></figure><p><a href="https://drive.google.com/open?id=1m7unIJJgyRZGd81oLdCmprbSjiQG4mJh">This Python script</a> calls the Wikipedia API and asks for all external references from a list of articles. It then builds <a href="https://drive.google.com/open?id=1R5RvXaOJ9iCoV8L3LAcQCRYASdGEYDBd">a network (.gexf file for Gephi) where the articles on the list are the nodes (same as the first network) but the edges reflect the volume of external references shared by two articles (although the edges look like direct links from one node to another, they are in reality projected through a set of external references that both nodes are citing).</a> The result is visualized below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_JEM8zEUg9yz1V3D9dFQ9Q.jpeg" /><figcaption>A network of articles from the ‘Circumcision’ category on Wikipedia <strong>connected by the degree to which they reference the same external sources. </strong>Nodes colored by Louvain modularity; graph layout by ForceAtlas2.</figcaption></figure><p>The first thing to notice is that the edges are now weighted (represented here as varying thickness). This was not possible in the first network of articles connected by hyperlinks since a link from one article to another can only meaningfully be counted once. In this case, however, two articles could have 5% or 95% of their external references in common, which in fairness should count differently in terms of how strongly the two articles can be said to be related. Such differences in relationship strength can is represented as differences in edge weight.</p><blockquote>Question: Can you explain how two nodes are related to each other when they are connected by an edge in the network above? And can you explain why some edges get a different edge weight thann others?</blockquote><p>The next thing to notice is that the same pattern of two clusters centered on male and female circumcision is found in this network as well. We could be tempted to say that the analysis of the network of circumcision articles connected by shared external references is, therefore, simply confirming what we already know from the analysis of the network of circumcision articles connected by direct hyperlinks. In reality, however, it is telling us something new. What we can say now, on top of the fact that articles about male circumcision point to different related topics than articles about female circumcision, is that articles about male circumcision draw on a different body of knowledge than articles about female circumcision. They simply reference a completely different set of reports and academic publications in support of what they state to be true.</p><blockquote>Question: Can you explain how the network of articles connected by shared references is built differently than then network of articles connected by direct hyperlinks? And can you explain why the analysis of these two networks permit us to learn different things about the way circumcision debates are presented on Wikipedia?</blockquote><p>Deciding how to build a network (i.e. what should count as nodes and edges) is never trivial. On the contrary, it involves important analytical choices. <a href="https://drive.google.com/open?id=15Za4pcw8F6eFxrxf0WIDbz0zuOO19glu">This Python script builds a network of editors connected to articles through revising them</a>. It is yet a third way to build a network, this time bi-partial with both editors and articles as nodes, that will allow us to see if the same editors write all the articles about circumcision, or if there are different editor communities interested in different issues (male and female circumcision, perhaps). At the end of the day, it is up to us, the controversy mappers, to make a reasoned choice about what kind of relations we are going to use make what kind of claims about regions, communities or themes in the terrain we are mapping. <a href="#e8d3">Richard Rogers (2018) thus encourages us to not naïvely accept the metrics offered to us by tools and media platforms but to think critically about how we repurpose them to answer specific questions.</a></p><h3>Some basic operations: Layout, coloring, node size</h3><p>When you open a graph file the following import report should appear (I am opening the <a href="https://drive.google.com/open?id=18g-icNcy7cWBqv31FhdSXOehK1MvioHq">network of Circumcision pages connected by in-text links</a>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IOvzhsVHLbMH8wm5QAIgpQ.png" /><figcaption>Opening a .gexf file (or any other graph file format) generates an ‘import report’. If you are happy with the information displayed here, just click ‘OK’.</figcaption></figure><p>Generally, you should be able to recognize the number of nodes and edges displayed here. Also, if Gephi encounters problems opening the graph file these will be displayed in the ‘Issues’ window. You will normally be able to open the file anyway. Click ‘OK’ if you are happy with the information you see. The graph will open and be displayed in a random layout (i.e. nodes placed randomly in space) such as the one below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TOyreA5CROpYsMHngTvLdA.png" /><figcaption>Initial random layout of the graph in the ‘Overview’ pane.</figcaption></figure><p>You can turn node labels on and off by clicking the big ‘T’ in the menu at the bottom of the ‘Graph’ window, and you can scale the label sizes by using the slider on the right side of the menu at the bottom of the ‘Graph’ window. If you want to see all the information for a specific node you can use the ‘Edit node attributes’ tool (cursor with a small question mark) located in the menu on the left side of the ‘Graph’ window. You can zoom in and out of the graph using the mouse scroll. You can also right-click and drag the graph to reposition your view.</p><h4>Layout</h4><p>In order to help us explore the structure of the graph and see clusters, bridges, and structural holes, we can use a force-directed layout algorithm. Force-directed layouts (or force vector or spring-based layouts) will push nodes apart from each. Edges between nodes will act as springs pulling these nodes together. Stronger edges (heavier edge weights) will act as stronger springs. We can choose the ForceAtlas2 Layout from the ‘Layout’ dropdown on the left side of the ‘Graph’ window:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*NW3i7VSWRtQPdhJMZTRn_w.png" /><figcaption>Choosing a layout algorithm.</figcaption></figure><p>Before running the layout let us review some of the parameters available to us. Eventually, the idea is to iteratively keep tweaking these parameters as we see the result of the layout. This, then, is just the initial setup. ‘Scaling’ will control the size of the area over which your layout can spread. If you need more space between nodes, increasing the scaling is a good option. Gravity controls the degree to which nodes are pulled towards the center of the network. If you have parts of the network floating far away from the center, increasing gravity is a good option. Finally, if you have a cluttering of nodes overlapping each other you can turn on ‘Prevent overlap’. However, this option should not be used until the Layout is otherwise complete as it may prevent nodes from finding their place. When your parameters are set, click ‘Run’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*tg8YONtNLSqXAUXgeqRbag.png" /><figcaption>Setting the parameters of the ForceAtlas2 layout</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vy0KHriyJrK7HwjLeRtgYQ.png" /><figcaption>The result of a ForceAtlas2 layout with Scaling 50, and Gravity set to ‘Stronger’ but decreased to 0.02.</figcaption></figure><h4>Size</h4><p>Given that this is a directed network where one page points to another page through a link it could make sense to visualize the most cited nodes. We can do that in the ‘Appearance’ pane to the left of the ‘Graph’ window. Select ‘Nodes’, ‘Ranking’ and the icon with growing concentric circles. From the dropdown, select ‘In-Degree’. This will size the nodes by the volume of incoming edges from other nodes in the network. Set minimum and maximum size for the nodes and click ‘Apply’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*7ba1teDrVPvgdivODAW9yQ.png" /><figcaption>Setting node size in the ‘Appearance’ pane.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E7tLo5dQ8XEO9d9y9YAj2Q.png" /><figcaption>Nodes sized by in-degree</figcaption></figure><h4>Color</h4><p>Finally, let us add some color to the nodes. As is the case with sizing, we can color nodes by many different parameters. In this case, we will calculate the modularity of the graph and color the nodes by the resulting modularity classes. From the ‘Statistics’ pane on the right of the ‘Graph’ window, run ‘Modularity’. For now, we can use default settings. The algorithm will try to cut the graph into communities where nodes are strongly related to each other inside the community and weakly connected to nodes in other communities. Click ‘OK’ to run the algorithm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mVC_W8HS5gqHBPEQqRVYsw.png" /><figcaption>Running the Modularity algorithm.</figcaption></figure><p>When the results are in we can open the ‘Appearance’ pane on the left side of the ‘Graph’ window and select ‘Nodes’, ‘Partition’ and the color palette icon. From the dropdown menu, select ‘Modularity Class’ and click ‘Apply’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*ltl6kCVUY8Sr-5sL39sJ0w.png" /><figcaption>Coloring nodes by Modularity Class.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EHpAWDVWnSgsAEoULY4lOQ.png" /><figcaption>Nodes colored by modularity.</figcaption></figure><p>You have now learned to build different networks from articles on Wikipedia and to think about the analytical implications of these choices. You have also learned how to open these networks in Gephi and conduct basic visual network analysis by using a force-directed layout and the modularity statistic to find clusters, and by using simple centrality metrics to find important nodes by different criteria. In the next tutorial, you will learn how to work with timelines in Tableau.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d6241127a336" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mapping controversies with digital methods: scrapers, crawlers, APIs]]></title>
            <link>https://medium.com/@EthnographicMachines/mapping-controversies-with-digital-methods-scrapers-crawlers-apis-17e0c96c340a?source=rss-8838c1d8bfac------2</link>
            <guid isPermaLink="false">https://medium.com/p/17e0c96c340a</guid>
            <category><![CDATA[controversy-mapping]]></category>
            <dc:creator><![CDATA[Ethnographic Machines]]></dc:creator>
            <pubDate>Tue, 12 Feb 2019 02:30:28 GMT</pubDate>
            <atom:updated>2020-02-05T16:21:04.266Z</atom:updated>
            <content:encoded><![CDATA[<h4>Tutorial 02 in a series on controversy mapping</h4><p>In this tutorial, you will learn how to harvest information from a large volume of Wikipedia pages for use in your controversy mapping. We will cover three basic techniques, namely:</p><ul><li>Scraping</li><li>Crawling</li><li>And interacting with Application Programming Interfaces (APIs)</li></ul><p>A <em>scraper</em> is a tool that automatically grabs specific information from a web page. Sometimes also known as <em>web scrapers</em>, because they work off the HTML structure of web pages, or <em>screen scrapers</em>, because they exploit the fact that the HTML contains instructions about how information should be displayed on your screen, they can be extremely useful to the controversy mapper because of their versatility and ability to systematically gather data in a way that is adaptable to the purpose of the mapping.</p><p>A <em>crawler </em>is a tool that automatically navigates a set of web pages according to a set of predefined rules. Used in combination with a scraper, crawling allows us to explore the web within certain parameters while looking for data to harvest.</p><p>Finally, an <em>API</em>, or Application Programming Interface, is a tool that most media platforms offer so that other programs (such as an app) can interact with them. APIs will have a syntax and a set of ‘calls’ or ‘requests’ that you will have to learn in order to know how to ask them for things. They are not as versatile as the scrapers, but they are often significantly faster and provide data that is less messy.</p><blockquote>Question: Can you explain the difference in principle between a scraper, a crawler and an API?</blockquote><p>In the introductory example below we will use an <a href="http://tools.medialab.sciences-po.fr/seealsology/">online tool called ‘Seealsology’</a> to get a sense of scraping, crawling and API interaction in practice. Sealsology is developed by <a href="https://densitydesign.org/research/contropedia-visualizing-controversial-topics-on-wikipedia/">Density Design</a> and the <a href="https://medialab.sciencespo.fr/">SciencesPo médialab</a> as a quick and dirty way to explore the semantic environment of a set of Wikipedia pages. It is based on the so-called ‘See also’ section, that Wikipedia editors can choose to include at the bottom of a page as a way to suggest other relevant Wikipedia pages to readers. Below is a screenshot of the ‘See also’ section for the <a href="https://en.wikipedia.org/w/index.php?title=Circumcision_controversies&amp;oldid=882221134">page on ‘Circumcision controversies’</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fflEG0y-CR8FfxCKoIjheg.png" /><figcaption>‘See also’ section for the<a href="https://en.wikipedia.org/w/index.php?title=Circumcision_controversies&amp;oldid=882221134"> page on ‘Circumcision controversies’</a> (February 2019)</figcaption></figure><p>As you open the <a href="http://tools.medialab.sciences-po.fr/seealsology/">Seealsology interface</a> you will be able to paste the URL of the Wikipedia page (or set of pages) that you wish to scrape into the box on the left. We will call this page the seed because it is the location from where the exploration begins (if there are several start pages we can talk about a list of seeds). We will use the URL for the page on ‘Circumcision controversies’ (namely: <a href="https://en.wikipedia.org/wiki/Circumcision_controversies">https://en.wikipedia.org/wiki/Circumcision_controversies</a>) as our seed. Before we click ‘START CRAWLING’, however, we have two more parameters to set: ‘Distance’ and ‘Parent links’. For now, let us untick ‘Parent links’ and set ‘Distance’ to 1. Click ‘START CRAWLING’ and observe what happens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gVS79tkuNCoIYM9dl-nszw.png" /><figcaption><a href="http://tools.medialab.sciences-po.fr/seealsology/">Seealsology</a> interface with ‘Distance’ set to 1, ‘Parent links’ unticked, and the URL for the ‘Circumcision controversies’ page as the seed.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2GfFbwNzf2d8YCHTHLU8ww.png" /><figcaption>Result after scraping the ‘Circumcision controversies’ page for ‘See also’ links at crawl distance 1.</figcaption></figure><p>You will observe that ‘Seealsology’ has now scraped the list of ‘See also’ links from the seed page and represented them as a network where the seed node is colored red. This is possible because of the way the HTML of a Wikipedia page is structured. You can explore this yourself in most browsers by using the inspect tool. In Chrome, for instance, right-click the ‘See also’ section and select ‘Inspect’ to open a panel with the corresponding HTML code. As you can see, all links on the list are tagged ‘&lt;a href=’ and appear inside the tags ‘&lt;li&gt;’ and ‘&lt;/li&gt;’. The list itself appears inside a ‘span class’ with the id ‘See also’. Effectively these tags allow a scraper to be programmed to always find the right links inside the right span class on any Wikipedia page, making sure, for instance, to not get all links from the text or all links in the footnotes instead.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rwd0Gy008z26mo8ruPtzTQ.png" /><figcaption>Inspecting the HTML code (left) of a ‘See also’ section (right) in Google Chrome (to open the panel, right-click the ‘See also’ section and select ‘Inspect’).</figcaption></figure><blockquote>Question: Can you explain what a scraper does? And can you explain how Seealsology uses scraping? What is the ‘seed’ and how does the scraper find more pages from this ‘seed’?</blockquote><p>The ‘Distance’ parameter that we set to 1 refers to the crawl distance of the scraper. We can increase it to 2 or 3 and run the crawl again. This will cause Seealsology to scrape the ‘See also’ links from the pages found through the ‘See also’ links from the seed (crawl distance 2), and in turn the ‘See also’ links from this new set of discovered pages as well (crawl distance 3). On the network visualization, the crawl distance at which a page is found is represented by the color of the node (called ‘level’ in the top left legend). Whereas Seealsology works as a <em>scraper</em> on each page, we say that it is <em>crawling</em> when it is automatically navigating from page to page according to a set of predefined rules (in this case the distance parameter).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EdNhvhaBwx3tlbp_3AmNRA.png" /><figcaption>Result after scraping the ‘Circumcision controversies’ page for ‘See also’ links at crawl distance 2.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DSGbPeed234RZkQyXQvIJg.png" /><figcaption>Result after scraping the ‘Circumcision controversies’ page for ‘See also’ links at crawl distance 3.</figcaption></figure><blockquote>Question: Can you explain what a crawler does? And can you explain how Seealsology uses crawling? What difference does it make if we change the crawl ‘distance’?</blockquote><p>Used in combination, scraping and crawling afford an extremely powerful method for gathering large volumes of digital traces in a structured manner. In principle, we could redesign the scraper to gather any type of data from any kind of webpage, as long as the data we are looking for is organized under a recognizable set of HTML tags. Similarly, we could redesign the crawler to follow any kind of path to other web pages as long as this path is visible to the scraper in the HTML. However, note that the fact that this is <em>possible</em> does not necessarily mean that it is ethical or indeed legal in all scenarios (notably if we were to scrape data behind paywalls or other access restrictions).</p><p>What we begin to see from the exploration of ‘See also’ links connecting pages around ‘Circumcision controversies’ is a pattern. At one end of the network is a cluster of pages about the anatomy of the penis, in the other a cluster of pages about reproductive rights and women’s issues. In the center is a cluster of pages on children’s rights and violence against men. We know that these clusters are formed by the fact that their articles can be found up to 3 ‘See also’ links away from the page on ‘Circumcision controversies’ and the fact that some of these articles refer more to each other than they do the rest.</p><p>What are not yet able to see are the articles that may refer directly <em>to</em> the ‘Circumcision controversies’ page without ever receiving a reference <em>from</em> the pages in the ‘See also’ network of the ‘Circumcision controversies’ page. These pages are difficult to get through scraping since we do not find links to them in the HTML of the pages we are looking at. That does not make them any less relevant for our exploration of the topic!</p><blockquote>Question: Can you explain why scraping links from a specific seed page does not necessarily permit us to find all the links to this seed from other pages?</blockquote><p>Seealsology solves this problem by interacting with Wikipedia’s API. If we tick ‘Parent links’ and run the crawl again we will see a new set of articles show up on the network as yellow nodes (‘level -1’). These pages are obtained by making an API call to Wikipedia and request a list of all pages linking to the ‘Circumcision controversies’ page. The Wikipedia API does not have a call that gets ‘See also’ links specifically, so Seealsology visits all the pages on the list returned by the API and uses the scraper to determine whether or not the link to ‘Circumcision controversies’ is, in fact, a ‘See also’ link. In this case, including these ‘Parent links’ with the assistance of the API allowed us to find an actor like <a href="https://en.wikipedia.org/wiki/Genital_Autonomy_America">Genital Autonomy America</a> whose mission it is to protect the birthright of “children and babies to keep their sex organs intact”. Their page links to ‘Circumcision controversies’, although the same is not true the other way around.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vjhXuohU8NVSTT81nRX7rg.png" /><figcaption>Result after scraping the ‘Circumcision controversies’ page for ‘See also’ links at crawl distance 2 and with ‘Parent links’ ticked.</figcaption></figure><blockquote>Question: Can you explain what an API is? And can you explain how Seealsology interacts with Wikipedia’s API?</blockquote><h3>Crawling and scraping a category on Wikipedia</h3><p>While we could use Seealsology as a tool to build a dataset of Circumcision related pages on Wikipedia (indeed, this is the intended use) we can also go straight to the editors of Wikipedia who explicitly take on this task of categorizing how pages relate to larger topics or debates. They do so when they build so-called ‘Categories’ (<a href="https://en.wikipedia.org/wiki/Category:Circumcision">see the one on circumcision here</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vSrAsxt9nAFFAsGa5d6_jw.png" /></figure><p>A category is a topical collection of Wikipedia articles. A category can both have sub-categories (with collections of articles on various sub-topics) and be part of more general categories (with collections of articles on more general topics). If we were to crawl and scrape this category we would, therefore, have to think about crawl depth (similar to the distance above). Inside the <a href="https://en.wikipedia.org/wiki/Category:Circumcision">‘Circumcision’ category</a>, we find a number subcategories. <a href="https://en.wikipedia.org/wiki/Category:Circumcision_debate">‘Circumcision debate’</a>, for example, or <a href="https://en.wikipedia.org/wiki/Category:Female_genital_mutilation">‘Female genital mutilation’.</a> And if we open these categories, more subcategories will appear. We could thus say that pages found directly in the <a href="https://en.wikipedia.org/wiki/Category:Circumcision">‘Circumcision’ category</a>, such as <a href="https://en.wikipedia.org/wiki/Religious_male_circumcision">‘Religious male circumcision’</a> or <a href="https://en.wikipedia.org/wiki/Holy_Prepuce">‘Holy Prepuce’</a>, are at depth 0, and we could decide to delimit our crawl by stating that it will go no further than depth 2 from the seed.</p><blockquote>Question: Can you explain crawl depth refers to in a structure of sub-categories inside categories?</blockquote><p>There is no standardized solution to finding a good seed category to start from or to set the right crawl depth. This requires careful and manual consideration from case to case. In the particular instance of circumcision, however, a depth of 2 from the seed category provides a very <a href="https://drive.google.com/open?id=1qt3RJhfNFPU7hkz4zdXPnt__H1qgGqX_">manageable output of 173 pages</a> that are all related to the topic. We used a custom Python script, which also includes some API calls, to scrape the category. You can <a href="https://drive.google.com/open?id=1j-lpFHiqmkz5b_hHaIz4IFHkQArvMtbn">download it as a Jupyter Notebook here</a> and try for yourself.</p><blockquote>Question: Can you find a good seed and depth limitation to crawl and scrape a category on Wikipedia?</blockquote><h3>Scraping relationships between pages in a category</h3><p>Once we have our list of circumcision-related Wikipedia pages we can use Seealsology to map how they relate to each other. This could be with the objective of spotting a pattern in how pages are organizing thematically, for instance with the <a href="http://tools.medialab.sciences-po.fr/seealsology/">Seealsology tool</a> we just used above. In order to turn our list of page names from the Circumcision category into a list of URLs that can be pasted into Seealsology we need to add the following prefix to all the page names: ‘<a href="https://en.wikipedia.org/wiki/">https://en.wikipedia.org/wiki/</a>’. This can be done easily in a spreadsheet editor such as Google Sheets (see below) where a column is added and a formula combining the root URL with a page title pasted into each cell. Explore the resulting spreadsheet <a href="https://drive.google.com/open?id=1K8qMrAq-L7UlI6pqIr6-hGmZDfJpYRslsGXLWjjZq74">here</a>. The list of URLs can be directly inputted to Seealsology and the crawl started at an appropriate depth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_TGCdTrE-ajE6EEC6TTv2A.png" /><figcaption>Converting page titles (column A) to Wikipedia URLs (column C) in <a href="https://drive.google.com/open?id=1K8qMrAq-L7UlI6pqIr6-hGmZDfJpYRslsGXLWjjZq74">Google Sheets</a>.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jJL7QV42GbudHjgYcCXFfw.png" /><figcaption>Result in <a href="http://tools.medialab.sciences-po.fr/seealsology/">Seealsology</a> using all pages in the Circumcision category as seeds and the crawl depth set to 3 (no parent links).</figcaption></figure><p>You will notice that many of the seed pages (red nodes) remain unconnected to the network after the crawl has finished running. This is due to the fact that the editors of these pages have not equipped them with a ‘See also’ section. Clearly, if we are interested in mapping how circumcision pages on Wikipedia relate to each other, restricting relationships to ‘See also’ links have considerable weaknesses. Even in those cases where a ‘See also’ section is, in fact, present, it can quickly be established that many relevant links to other circumcision pages are provided outside of this section (for instance as part of the text).</p><p>You have now learned how scrapers, crawlers, and APIs offer different opportunities for curating digital datasets. In the next tutorial, you will learn how to use a combination of Python scripts and visual network analysis to go beyond SeeAlsology and ‘see also’ links to map other relationships around circumcision on Wikipedia.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17e0c96c340a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to Controversy Mapping]]></title>
            <link>https://medium.com/@EthnographicMachines/introduction-to-controversy-mapping-6961f03f9a8a?source=rss-8838c1d8bfac------2</link>
            <guid isPermaLink="false">https://medium.com/p/6961f03f9a8a</guid>
            <dc:creator><![CDATA[Ethnographic Machines]]></dc:creator>
            <pubDate>Fri, 01 Feb 2019 07:08:25 GMT</pubDate>
            <atom:updated>2020-02-04T10:52:49.976Z</atom:updated>
            <content:encoded><![CDATA[<h3>Observing Controversies Online</h3><h4>Tutorial 01 in a series on controversy mapping</h4><p>Controversy mapping is a set of methods for charting and navigating complex socio-technical debates. We call them controversies in order to distinguish the terrain from simpler discussions where actors ‘just’ have opinions about a question. Controversies are more complicated for a number of reasons. For instance, actors will often not agree on the nature of their disagreement. They will fight to define what the relevant questions are to ask and how they should be answered. Actors will realize how different ways of posing questions lead to different types of arguments, with different types of evidence being mobilized and different types of expertise being called upon to settle disputes. They will favor certain types of experts and emphasize certain types of evidence over others. Indeed, clear and unambiguous expert advice may not be available, perhaps because the relevant field of science is itself controversial (see for example <a href="#0a3e">Harry Collins’ (1975) classic study of gravitational waves</a>), perhaps because different experts have different stakes in the debate (see for example <a href="#32d5">Charis Thompson’s (2002) analysis of how different fields of bioscience approach the problem of population control among elephants in the Amboseli National Park differently</a>), or because actors outside the formal knowledge institutions realize that they have stakes in how knowledge is produced and therefore decide to intervene in that process (see for example <a href="#9acf">Steven Epstein’s (1995) analysis of how patients and next of kin involved themselves as co-producers of knowledge in the diagnosis of HIV</a> or <a href="#e8fa">Sarah Whatmore and Catharina Landström’s (2011) account of their participation in the Pickering Flood Action Group)</a>. Of course, the way in which such complexities are configured depends on the context and is liable to change over time. The cartographic toolbox must, therefore, be capable of adapting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OEpeyX7NBsYEm1XG9KnS-Q.png" /><figcaption>Mapping the prevalence of different issues in the international climate change negotiations over time. <a href="http://climaps.eu/#!/map/absolute-and-relative-visibility-of-issues-in-unfccc-negotiations-1995-2013">Subproject from the EMAPS (Electronic Maps to Assist Public Science) collaboration.</a></figcaption></figure><p>Controversy mappers are in a situation where the terrain itself is constantly evolving. It would be the equivalent of a geographic mapmaker waking up every morning to find that the tectonic plates had shifted, changing the landscape and making the previous days’ bearings and triangulations obsolete. Like James Cook surveying the New Zealand coastline for the first time, the controversy mapper does not have the luxury of an established set of reference points. The core issues of the MMR vaccine debate are not transferable to the HPV vaccine debate; the issue of same-sex marriage has different stakeholders asking different questions in different countries; much of what we thought we knew about the pros and cons of nuclear energy has been disrupted by the urgency of manmade climate change. What makes controversies interesting is precisely this ability to bring actors together in new ways, to form alliances that would otherwise have been unimaginable and create fault lines where previously there were none. They are, as <a href="#4751">Sarah Whatmore (2009)</a> puts it, ‘generative events’. <a href="#3e3a">Tommaso Venturini (2010) thus compares the terrain surveyed by the controversy mapper to a magma flow</a>: it is the ground itself that is being reconfigured. Contrary to a geographical base map which, once done, can become a stable grid that serves as a tool for making sense of all sorts of additional and unrelated information (for instance by overlaying it with data about election results, biodiversity, transport infrastructure, or language zones) a controversy map, as a matter of course, is only good for the particular controversy it purports to make navigable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QAwSX6LSCx9O-hZz.jpg" /><figcaption>Surveying virgin terrain: A sample of James Cook’s first maps of the New Zealand coastline (1773). River Thames and Mercury Bay, Bay of Islands, Tolaga Bay. The image is in the public domain from <a href="https://commons.wikimedia.org/wiki/File:River_Thames_and_Mercury_Bay_in_New_Zealand;_Bay_of_Islands_in_New_Zealand;_Tolaga_Bay_in_New_Zealand_(map_by_Captain_James_Cook,_1773).jpg">Wikimedia Commons</a>.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/0*aug1yuiFhb-9ka9b.png" /><figcaption>Overlaying information on a well-known base map: The results of the 2012 U.S. presidential election by county. The image is in the public domain at <a href="https://commons.wikimedia.org/wiki/File:2012nationwidecountymapshadedbypercentagewon.svg&#39;">Wikimedia Commons.</a></figcaption></figure><h3>The circumcision debate on Wikipedia</h3><p>As an example, let us consider the debate about circumcision as it appears on English Wikipedia. The goal of our mapping project is to <strong>provide an overview of actors and issues </strong>in the debate. This may seem simple enough at the outset, but we have to do so by <strong>following the actors in the controversy </strong>rather than our own preconceived ideas about what is relevant and who is important (<a href="#3e3a">see Venturini 2010 and 2012</a>). Given the fact that controversies are complicated precisely because different actors have different ideas about which questions to ask and how to answer them, to provide an overview of actors and issues is often not simple at all.</p><h4>What are the issues?</h4><p>Let us begin by figuring out what the discussion is about. Wikipedia offers several good starting points, such as the main <a href="https://en.wikipedia.org/w/index.php?title=Circumcision&amp;oldid=877652375">page about ‘Circumcision’</a>, the <a href="https://en.wikipedia.org/w/index.php?title=Circumcision_controversies&amp;oldid=880084258">page about ‘Circumcision controversies’</a>, or <a href="https://en.wikipedia.org/w/index.php?title=Ethics_of_circumcision&amp;oldid=880343195">the page about the ‘Ethics of circumcision’</a>. A quick browse of these pages should make it clear that we are not dealing with a binary debate between supporters and opponents of circumcision. Rather, what we observe is a range of issues (or debates within the debate). For example:</p><ul><li>Does circumcision reduce the risk of HIV/AIDS? (<a href="https://en.wikipedia.org/w/index.php?title=Circumcision_and_HIV&amp;oldid=878453701">Dedicated page</a>)</li><li>Does circumcision reduce the risk of HPV and thus cervical cancer?</li><li>Does circumcision reduce the risk of penile cancer?</li><li>Does circumcision reduce the risk of urinary tract infections?</li><li>Does circumcision reduce the risk of transferring other STDs like syphilis or chlamydia to women?</li><li>Is circumcision a cost-effective way to reduce the above-mentioned risks?</li><li>What are the pros and cons of different pain management techniques during the procedure?</li><li>Does circumcision affect the sensitivity of the penis and thus the ability to feel sexual pleasure as an adult?</li><li>Does circumcision affect the parental bond?</li><li>Can parents make a decision for a child when the procedure is irreversible and not necessary from a medical point of view?</li><li>What weight should be given to religious arguments in that respect?</li><li>What weight should be given to arguments about a child’s need to be socially accepted in that respect?</li></ul><p>All of these issues assume that we are talking about the so-called ‘non-therapeutic’, i.e. not medically necessary, circumcision of infant boys. There are additional arguments to be made for (or against) circumcision as a medical treatment for conditions like phimosis or balanoposthitis. Likewise, voluntary circumcision of adult males, including for religious reasons, clearly reduces the ethical dilemmas.</p><blockquote>Question: Can you find more circumcision-related issues to add to the list?</blockquote><p>When a subtopic grows to a certain size, Wikipedia editors will typically branch it off from the main topic and create a dedicated article for it. For a mature topic like Circumcision, this practice effectively means that we have a large number of potentially relevant Wikipedia articles to consider. Rather than simply searching for more, we can follow how editors group articles in topical categories and sub-categories, such as <a href="https://en.wikipedia.org/wiki/Category:Circumcision">this one on circumcision</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RPJlsM-yCJrldBdP.png" /></figure><p>Some articles (e.g. ‘Religious male circumcision’ or ‘Holy Prepuce’) are placed directly in the main category, while others are grouped into sub-categories inside the main category (in this case ‘Circumcision debate’, ‘Female genital mutilation’ and ‘Mohel’). Indeed, the main category is itself part of even more general categories. These more general categories, into which editors have grouped the Circumcision category (e.g. ‘Human penis’ or ‘Men’s rights’), are visible at the bottom of the category page:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o5y7Db2tZLwrBrLwW1gfiw.png" /></figure><blockquote>Question: What sub-categories are contained in the Circumcision category? What are the sub-categories of these categories? And what other categories is the Circumcision category itself a sub category of? Can you find even more circucission related issues to add to the list when you browse the articles of these categories and sub-categories?</blockquote><p>As a mapmaker, you must now come to terms with the fact that the list you are building is <em>not a view from nowhere</em>. On the contrary, you are very much surveying the controversy from<em> somewhere</em>, first and foremost from a specific media platform, Wikipedia, but also from a specific anglophone context and a specific moment in time.</p><p>Wikipedians follow <a href="https://en.wikipedia.org/wiki/Wikipedia:Ten_Simple_Rules_for_Editing_Wikipedia#Rule_2._Learn_the_five_pillars">certain principles for editing articles</a>, such as attempting to write neutrally, always cite sources and never argue from authority. These principles are enforced by the community on itself, a process that can be observed on the <a href="https://en.wikipedia.org/wiki/Talk:Circumcision">‘talk’ pages behind the actual articles</a> (click the ‘Talk’ tab on the top left of any article to see the ‘talk’ page). It stands to reason that surveying the circumcision debate on social media like Twitter or Instagram, or from more partisan sources like the websites of actors in the controversy, is unlikely to produce a list of issues identical to the one produced by surveying Wikipedia (see <a href="#b105">Jean Burgess’ (2016) cross-platform analysis of the #gamergate controversy</a>).</p><blockquote>As an example: compare <a href="https://www.reddit.com/r/explainlikeimfive/comments/1xfwlk/eli5_the_circumcision_debate/">this debate on Reddit</a> to your list of issues from Wikipedia.</blockquote><p>However, the fact that controversy maps are always situated perspectives is not exclusively attributable to different media. We can conduct two simple experiments to demonstrate this while staying within the boundaries of the Wikipedia platform.</p><p><em>First, </em>at the top of each Wikipedia page is a tab that says ‘View history’. It allows you to inspect all previous versions of any page (e.g. <a href="https://en.wikipedia.org/w/index.php?title=Circumcision&amp;action=history">the revision history of the ‘Circumcision’ page</a>). If you inspect the very first version of the <a href="https://en.wikipedia.org/w/index.php?title=Circumcision&amp;oldid=784879">‘Circumcision’ page from 2003</a> you will for instance notice that the issue of HIV/AIDS prevention is not present at all, while, compared to more recent versions of the page, significant space is dedicated to the issue of penile cancer prevention and the dubious scientific evidence underpinning it. This is potentially important. Since the medium remains constant, the change in the composition of issues could suggest that the circumcision debate actually looked different in 2003 than it does today.</p><blockquote>Question: can you identify other changes to the composition of issues by revisiting earlier versions of circumcision-related Wikipedia pages?</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/1*WgZqBnU85CN8hlP_pr9DEg.png" /><figcaption>Different language versions of the <a href="https://en.wikipedia.org/w/index.php?title=Circumcision&amp;oldid=880145881">‘Circumcision’ page (January 2019)</a>, available in the sidebar on the left side of the page.</figcaption></figure><p><em>Second, </em>in the sidebar to the left of each page is a list of links to that page in different languages. These are not translated versions of the English original, but independently authored articles. This effectively means that different groups of Wikipedia editors, writing from positions in different cultural contexts, but still adhering to the same media-specific guidelines, can make circumcision-related issues visible in different ways. Again, since the medium remains constant any change in the composition of issues across different language versions of the same Wikipedia article could suggest that the circumcision debate actually looks different in different countries.</p><blockquote>Question: How will it affect your list of issues if you switch to different language versions of the ‘Circumcision’ page?</blockquote><h4>Who are the actors?</h4><p>Once we have surveyed the issues we can move on to build a list of actors. We do it in this order, and not the other way around, because controversy mapping, drawing on inspiration from ANT (again, see <a href="#3e3a">Venturini 2010</a>), defines an actor as anyone or anything that makes a difference (acts) in a situation. Given the diversity of circumcision-related issues, we should assume that actors are not, in fact, actors in all situations across the controversy. We can begin with some initial observations from the <a href="https://en.wikipedia.org/w/index.php?title=Circumcision&amp;oldid=880145881">‘Circumcision’ page</a>:</p><ul><li>The World Health Organisation (WHO) and UNAIDS seem to be making a difference to the issue of HIV/AIDS prevention by recommending circumcision in areas with high endemic HIV rates.</li><li>The British Medical Association (BMA) seems to be making a difference to the issue of whether parents can decide to circumcise their children when it is not medically necessary, by stating that its members are not obliged to perform non-therapeutic circumcision. So does the German Academy for Pediatric and Adolescent Medicine, by recommending against non-therapeutic circumcision of infants, and the Royal Dutch Medical Association, by equating male circumcision to female genital mutilation.</li><li>Some reformist Jewish rabbis in the U.S. seems to have made a difference to the issue of what weight should be given to religious arguments by inventing an alternative welcoming ritual, Brit Shalom, that does not involve circumcision.</li></ul><p>When we say that these are actors because they make a difference to the debate, then what exactly do we mean? They are not observable to the controversy mapper simply because they make recommendations, have opinions, or come up with new inventions. The reason we can see them and the reason we can record them on our list is the fact that they are cited on Wikipedia. If we had worked from the assumption that things like medical associations or rabbis have some inherent relevance in the circumcision debate that entitles them, <em>a priori, </em>to be treated as actors, then we might have called them up or written them an e-mail to ask about their position. That is not the approach taken here. We assume nothing about relevance or actor status, except for the extremely simplistic actor-network theoretical idea that something must act in order to be acknowledged as an actor. So what is it that the BMA or the WHO have done that counts as acting here? Sure, they both <em>want</em> to make a difference to the circumcision debate, but what really counts is the fact that they, unlike so many other opinion holders, have managed to become mentioned in the relevant Wikipedia article. It indicates that others, namely the Wikipedians, have acknowledged their role as actors. This is, in the particular setting of a controversy mapping project on Wikipedia, the difference we can positively survey.</p><p>It adds to the media specificity of the mapmaking. Since we are committed to following the controversy wherever it takes us it also means that we cannot, on this particular map, begin to include actors simply because we feel they are missing. We are effectively trying to ‘outsource’ such decisions to the field, and in this case the field is defined by a mix of people editing Wikipedia, the technical infrastructure they have available for doing so, the principles governing the editorial process, and, of course, the wider landscape of stuff happening in the controversy (claims being made, events happening, etc.) that the editors are trying to represent. In order to get a sense of how this works, we can, again, visit the <a href="https://en.wikipedia.org/w/index.php?title=Talk:Circumcision&amp;oldid=875942207">‘talk’ page </a>behind the actual article. Below is an example where editors debate how the issue of penile cancer prevention should be represented. Notice the argument about giving more weight to the position of the American Cancer Society. This might eventually result in the ACS gaining presence as an institution that makes a difference and this as an actor in our mapping.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RzQCDw04i7kQMBKKshzd2g.png" /><figcaption>Wikipedians debate evidence that circumcision prevents penile cancer. Excerpt from the <a href="https://en.wikipedia.org/w/index.php?title=Talk:Circumcision&amp;oldid=875942207">Talk page on ‘Circumcision’ (January 2019)</a>.</figcaption></figure><p>Besides actors which are explicitly mentioned as influential in the text, we can also inspect the citations underpinning different claims in the text by following the footnotes. Below is an excerpt of the section on ‘Uses’ from the ‘Circumcision’ page. Notice, for instance, how two sources are cited in support of the claim that “Circumcision may be used to treat pathological phimosis, refractory balanoposthitis, and chronic or recurrent urinary tract infections”. Considering the guidelines of Wikipedia, which encourage citation, these two sources, both from pediatric textbooks, are clearly acting in the sense of ensuring that the claim is maintained on Wikipedia.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ysnGS_uy4TsLQLT_QN5_zQ.png" /><figcaption>‘Uses’ section, <a href="https://en.wikipedia.org/w/index.php?title=Circumcision&amp;oldid=875227144">Wikipedia page for Circumcision (January 2019)</a></figcaption></figure><blockquote>Question: Which actors can you add to the list by inspecting circumcision-related pages on Wikipedia? How do these actors make a difference? In relation to which issues?</blockquote><p>As we expand the list you may have noticed that actors, defined solely as who- or whatever makes a difference in the situation, makes for a quite heterogeneous bunch. From what we have already covered we already have a mix of people and institutions. But why stop there? Circumcision is clearly a very physical phenomenon and material circumstances can make a big difference. At the very top of the <a href="https://en.wikipedia.org/w/index.php?title=Circumcision&amp;oldid=877652375">page about ‘Circumcision’</a> a so-called ‘disambiguation’ announces that this page is focussed on ‘male circumcision’ whereas information about ‘female circumcision’ is found on the <a href="https://en.wikipedia.org/w/index.php?title=Female_genital_mutilation&amp;oldid=878815045">page about ‘Female genital mutilation’</a>. It thus not only appears that knowledge about male and female circumcision is organized in different places, but that there is a notable difference in the connotations carried by the two procedures.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ARmsWC0_ZQFyTcL5uDQ5g.png" /><figcaption>Headline from the <a href="https://en.wikipedia.org/w/index.php?title=Circumcision&amp;oldid=877652375">‘Circumcision’ page on Wikipedia (January 2019)</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*KDZKeGuaanVgFUVv.jpg" /><figcaption>Male circumcision. The image is licensed under the Creative Commons Attribution-Share Alike 1.0 at <a href="https://commons.wikimedia.org/wiki/File:Circumcision_illustration.jpg">Wikimedia Commons</a>.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/0*0MgbhAdqhbZ2JYVq.png" /><figcaption>Different types of female circumcision (a.k.a. female genital mutilation). The image is in the public domain at <a href="https://commons.wikimedia.org/wiki/File:FGC_Types.svg">Wikimedia Commons.</a></figcaption></figure><p>From the mapmaker’s point of view, this represents a major topographical feature in the landscape of the controversy. It appears that the physical difference in male and female circumcision (or, indeed, female genital mutilation) leads to at least two different casts of actors, touch upon at least two different sets of issues, and develop along at least two different timelines (this is what J<a href="#31f9">ohn Law and Vicky Singleton (2014) calls ‘multiplicity’</a>). In other words: clitoris and foreskin deserve a spot on our list of actors! In fact, even if we stay within the bounds of male circumcision, the different surgical methods available for the removal of the foreskin, and their different affordances vis-a-vis the age of the patient or the circumstances of the surgery, makes a difference to how circumcision may be cast as problematic. Here are a few examples of actors that make a difference in this respect:</p><ul><li>Glomco clamp</li><li>Mogen clamp</li><li>Pastibell</li><li>Prepex</li><li>Shang ring</li></ul><p>Their advantages and potential complications are described in detail on the <a href="https://en.wikipedia.org/w/index.php?title=Circumcision_surgical_procedure&amp;oldid=871256000">‘Circumcision surgical procedure’ page</a>.</p><blockquote>Question: Can you describe how these different non-human actors make a difference to different issues in the circumcision debate?</blockquote><p>You have now learned how to draft a list of actors and issues. In the next tutorials, we will discuss how to gauge the importance of different actors and how to do it at scale with digital methods.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6961f03f9a8a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Amager Fælledsagen på Facebook]]></title>
            <link>https://medium.com/@EthnographicMachines/amager-f%C3%A6lledsagen-p%C3%A5-facebook-mere-end-flygtige-likes-og-60c8ffc562d3?source=rss-8838c1d8bfac------2</link>
            <guid isPermaLink="false">https://medium.com/p/60c8ffc562d3</guid>
            <category><![CDATA[dkmedier]]></category>
            <category><![CDATA[amager-fælled]]></category>
            <category><![CDATA[sosiale-medier]]></category>
            <category><![CDATA[facebook]]></category>
            <category><![CDATA[dkpol]]></category>
            <dc:creator><![CDATA[Ethnographic Machines]]></dc:creator>
            <pubDate>Sat, 11 Nov 2017 13:05:22 GMT</pubDate>
            <atom:updated>2017-11-12T22:46:29.416Z</atom:updated>
            <content:encoded><![CDATA[<p>I April 2016 blev Amager Fælleds Venner oprettet som en åben gruppe på Facebook. Halvandet år efter, i November 2017, havde næsten 9.000 brugere været aktive i gruppens diskussioner (også selvom gruppen kun havde knap 8.000 medlemmer). Godt og vel en femtedel af dem havde bidraget med opslag eller kommentarer, og en substantiel del af disse (ca. 1.200 brugere) havde bidraget med opslag eller kommentarer mere end en gang. Der er altså ikke tale om en lille gruppe af agitatorer, der har formået at skabe opmærksomhed på Facebook. Tværtimod lader protesten til at have samlet en reelt interesseret og diskutererende gruppe borgere omkring sig.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F241795366&amp;dntp=1&amp;url=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F241795366&amp;image=http%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F665479114_1280.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1280" height="720" frameborder="0" scrolling="no"><a href="https://medium.com/media/75203a66b07c231607c017c0bf588832/href">https://medium.com/media/75203a66b07c231607c017c0bf588832/href</a></iframe><h3>Brugere engagerer sig i takt med sagens udvikling</h3><p>Hvis det forholdt sig sådan, at protesten mod byggeplanerne på Amager Fælled udelukkende var et socialt mediefænomen, der i højere grad skyldtes Facebook end borgernes forhold til strandengen, så ville tilstrømningen af nye aktive brugere ikke nødvendigvis have noget at gøre med de faktiske begivenheder i sagen. Det modsatte er tilfældet. I oktober 2016 tiltrak en underskriftindsamling mod byggeplanerne likes og kommentarer fra over 700 brugere, der ikke havde været aktive i gruppen før. Folkeringen omkring Fælleden i maj 2017 formåede at levere noget lignende. Det helt store spring i tilstrømningen af nye brugere kom i august 2017. Her ryddede By og Havn dele af bevoksningen på strandengen som forberedelse til en VVM undersøgelse. Over 1.700 nye brugere blev aktive i gruppen i den måned, en tendens der fortsatte måneden efter med mere end 2.000 nye brugere. Det var også den måned, september 2017, hvor Frank Jensen tog konsekvensen af vaklende politisk tilslutning til byggeplanerne og besluttede at undersøge mulighederne for at bygge andre steder end på strandengen.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=%2F%2Fembed.chartblocks.com%2F1.0%2F%3Fc%3D5a0701c39973d29373dbd0a1%26t%3D255122883951c29&amp;url=https%3A%2F%2Fpublic.chartblocks.com%2Fc%2F5a0701c39973d29373dbd0a1%3Ft%3D255122883951c29&amp;image=http%3A%2F%2Fd3ugvbs94d921r.cloudfront.net%2F5a0701c39973d29373dbd0a1.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=chartblocks" width="800" height="450" frameborder="0" scrolling="no"><a href="https://medium.com/media/0fd8f4a2a89eac2a3f8ef68c88f4cdab/href">https://medium.com/media/0fd8f4a2a89eac2a3f8ef68c88f4cdab/href</a></iframe><h3>2700 opslag fra 590 forfattere</h3><p>Vi kan nu spørge, hvordan og hvor meget de nye brugere har bidraget? Har deres engagement været forbipasserende i form af et par likes, eller har de bidraget mere substantielt til diskussionerne i gruppen? Her bliver det klart, at nye brugere i ret vid udstrækning har påtaget sig en indholdsskabende rolle.</p><p>Hvis vi kigger på antallet af brugere, der er <em>forfattere til et opslag</em>, så er det måned for måned nogenlunde stabilt i forhold til antallet af opslag. Der er således i gennemsnit godt og vel dobbelt så mange opslag, som der er forfattere i en given måned. Når der bliver forfattet 750 opslag i august 2017, er der også næsten 300 brugere, der står bag dem. Det er altså ikke sådan, at den samme kernegruppe af brugere bare bliver mere aktive og producerer mere indhold som sagen udvikler sig. Der bliver derimod flere indholdsproducerende brugere over tid. Det bliver særligt tydeligt, når vi kigger på andelen af nytilkomne opslagsforfattere i en måned. Her er det hver måned mellem hver anden og hver tredje opslagsforfatter, der ikke tidligere har været aktive i den rolle.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=%2F%2Fembed.chartblocks.com%2F1.0%2F%3Fc%3D5a06f72b9973d21271dbd0a2%26t%3D31aa90729e7c9a2&amp;url=https%3A%2F%2Fpublic.chartblocks.com%2Fc%2F5a06f72b9973d21271dbd0a2%3Ft%3D31aa90729e7c9a2&amp;image=http%3A%2F%2Fd3ugvbs94d921r.cloudfront.net%2F5a06f72b9973d21271dbd0a2.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=chartblocks" width="800" height="450" frameborder="0" scrolling="no"><a href="https://medium.com/media/fad7ca2791e1cf067e4921869bf84c8f/href">https://medium.com/media/fad7ca2791e1cf067e4921869bf84c8f/href</a></iframe><h3>22.000 kommentarer og 212.000 reaktioner</h3><p>Over hele perioden er der 570 brugere, der har været opslagsforfattere, hvoraf 250 af dem har været det mere end en gang. Tilsvarende er der 1725 brugere der har været <em>kommentarforfattere</em>, hvoraf 1125 af dem har været det mere end en gang. Hvis vi forsøger at estimere størrelsen på den gruppe brugere, der har haft en løbende samtale med hinanden, så må den nødvendigvis både inkludere de brugere, der <em>regelmæssigt</em> har produceret opslag, og de brugere, der <em>regelmæssigt</em> har kommenteret på andres opslag. Det er til gengæld et åbent spørgsmål, hvad reglmæssigt vil sige i denne sammenhæng. Der er 650 brugere, der har forfattet mindst 5 opslag eller kommentarer, 400 brugere, der har forfattet mindst 10 opslag eller kommentarer, 100 brugere, der har forfattet mindst 50 opslag eller kommentarer, og 50 brugere, der har forfattet mindst 100 opslag eller kommentarer. Det er under alle omstændigheder sådan, at mere end hver femte bruger, der har været aktiv i gruppen, har bidraget med indhold til diskussionen.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=%2F%2Fembed.chartblocks.com%2F1.0%2F%3Fc%3D5a06fb2a9973d29871dbd0a1%26t%3Df94c0edb6214c29&amp;url=https%3A%2F%2Fpublic.chartblocks.com%2Fc%2F5a06fb2a9973d29871dbd0a1%3Ft%3Df94c0edb6214c29&amp;image=https%3A%2F%2Fwww.gravatar.com%2Favatar%2Fb263c8c5373a7dcce3b936a5a974fd12%3Fd%3Dmm&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=chartblocks" width="800" height="450" frameborder="0" scrolling="no"><a href="https://medium.com/media/85127e9a858634df1a3bb463e2af63e0/href">https://medium.com/media/85127e9a858634df1a3bb463e2af63e0/href</a></iframe><p>Og så er der de 7.000 brugere, der aldrig har forfattet opslag eller kommentarer, men har deltaget i diskussionen ved at like andres bidrag eller ved at reagere med en emoji (for eksempel ‘griner’, ‘vred’, ‘ked af det’, ‘wow’ eller ‘elsker’). Det er nærliggende at argumentere for, at denne gruppe, der udgør 80% af de aktive brugere, har deltaget mere eller mindre ‘gratis’ i protesten fra sidelinjen. Det kræver ikke videre stillingtagen eller indsats at smide et enkelt like eller en emoji i forbifarten, og det er også sådan, det forholder sig, for over halvdelen af de brugere, der ikke forfatter opslag eller kommentarer. 4.000 af de aktive brugere har således kun reageret en enkelt gang på indhold fra gruppen. I den anden ende af skalaen finder vi til gengæld 125 brugere, der alle har reageret mindst 50 gange på indhold fra gruppen uden nogensinde at have forfattet et opslag eller en kommentar. Deres engagement kan næppe siges at være forbipasserende.</p><p>Det er også muligt at argumentere for, at der er forskel på blot at like indhold, og at tage stilling til indhold ved at reagere på det med en emoji. Der er smidt 32.000 emojis over hele perioden, og de fortæller en slags historie. Der er for eksempel overvægt af vrede og triste emojis i august 2017 når By og Havn rydder vegetation på strandengen, mens der er overvægt af elsker emojis i måneden efter når Frank Jensen beslutter at søge alternative byggegrunde.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=%2F%2Fembed.chartblocks.com%2F1.0%2F%3Fc%3D5a0701309973d22d73dbd0a1%26t%3D1cd204bd56cc6fd&amp;url=https%3A%2F%2Fpublic.chartblocks.com%2Fc%2F5a0701309973d22d73dbd0a1%3Ft%3D1cd204bd56cc6fd&amp;image=http%3A%2F%2Fd3ugvbs94d921r.cloudfront.net%2F5a0701309973d22d73dbd0a1.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=chartblocks" width="800" height="450" frameborder="0" scrolling="no"><a href="https://medium.com/media/a0c726f2edee7f70bdbd0f410e0d6c53/href">https://medium.com/media/a0c726f2edee7f70bdbd0f410e0d6c53/href</a></iframe><h3>Konklusion</h3><p>Et af de kritiske spørgsmål, der i øjeblikket rejses i forhold til de sociale medier og deres rolle i den demokratiske samtale, handler om inflationen i meninger. Er det simpelthen blevet for let at ‘like’ sig til alle mulige tilfældige holdninger, som man ikke for alvor behøver stå på mål for? Når man ser debatten udspille sig om Amager Fælled på Facebook, lægger man med det samme mærke til den rolle, en gruppe som Amager Fælleds Venner spiller. Her kunne man, med udgangspunkt i ovennævnte kritik, spørge, om der ikke blot er tale om en hård kerne af agitatorer, der dygtigt har formået at skabe en masse interesse for deres sag gennem lette likes og reaktioner fra folk, der ikke er egentligt engageret i sagens substans? Det mener jeg, at vi kan afvise. Der er simpelthen for mange brugere, der ikke har været engagerede fra starten, og alligevel bliver tilbagevendende forfattere til indlæg og kommentarer i debatten.</p><h3>Note om datagrundlaget</h3><p>Datasættet er høstet og visualiseret af Tekno-Antropologisk Laboratorium på Aalborg Universitet i København gennem Facebooks åbne API. Det indeholder kun selvoffentliggjort materiale fra gruppen Amager Fælleds Venner. Data indeholder ikke personoplysninger. Du kan høste et lignende dataset med appen Netvizz, men vi har brugt et skræddersyet værktøj til formålet. Netværket i videoen er layoutet i Gephi med ForceAtlas2. Kontakt Anders Kristian Munk (akm@learning.aau.dk) hvis du har yderligere spørgsmål til data eller metoder.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=60c8ffc562d3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>