<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Eni Mustafaraj on Medium]]></title>
        <description><![CDATA[Stories by Eni Mustafaraj on Medium]]></description>
        <link>https://medium.com/@enimust?source=rss-82d778d27ffe------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*ps3LejxCqhM_Uv8-.png</url>
            <title>Stories by Eni Mustafaraj on Medium</title>
            <link>https://medium.com/@enimust?source=rss-82d778d27ffe------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 10:43:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@enimust/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[It’s a “Google-drenched society”, but we still suffer from an information drought]]></title>
            <link>https://medium.com/@enimust/its-a-google-drenched-society-but-we-still-suffer-from-an-information-drought-4be35132b3d5?source=rss-82d778d27ffe------2</link>
            <guid isPermaLink="false">https://medium.com/p/4be35132b3d5</guid>
            <category><![CDATA[google]]></category>
            <category><![CDATA[algorithm-auditing]]></category>
            <category><![CDATA[books-and-authors]]></category>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[news-literacy]]></category>
            <dc:creator><![CDATA[Eni Mustafaraj]]></dc:creator>
            <pubDate>Wed, 19 Dec 2018 17:09:53 GMT</pubDate>
            <atom:updated>2018-12-19T19:32:35.266Z</atom:updated>
            <content:encoded><![CDATA[<h4>Google should help the public better understand news sources</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*ESAh04xSHDopczaWQEHpqQ.jpeg" /><figcaption>The First Thanksgiving 1621. By J.L.G. Ferris, painted in 1932. This image is part <a href="https://sheg.stanford.edu/history-assessments/first-thanksgiving">of an assignment in historical thinking</a> by the Stanford History Education Group (SHEG).</figcaption></figure><p>There is much to learn from Sam Wineburg’s latest book, <a href="https://www.amazon.com/dp/B07H9FWT73/">Why Learn History (When It’s Already on Your Phone)</a>. In one chapter, he discusses how Howard Zinn’s magnum opus, <em>A People’s History of the United States</em>, may undermine the teaching of history by neglecting to expose the challenging process of reconciling contradictory sources.</p><p>Wineburg is interested in finding ways to familiarize students with the methods — which he names “intellectual moves” — that historians use to reason about sources and evidence. In one anecdote, Wineburg shares how one student, Ramon, feels like a cheater after glancing at sourcing information at the end of the document before reading content in its entirety. In his words:</p><p><em>Rather than celebrating sourcing as an astute act of historical reading, here was a future history teacher who considered it illegitimate to examine a document’s attribution before analyzing its contents.</em></p><p>Sourcing, as <a href="https://www.youtube.com/watch?v=ledQNzuiHgU">this short video from Wineburg’s Stanford History Education Group</a> explains, is about asking when a document was written, who wrote it, and why, before engaging with the document. It is a crucial step in assessing credibility of information. However, Wineburg’s research suggests that <a href="https://ed.stanford.edu/news/stanford-researchers-find-students-have-trouble-judging-credibility-information-online">young people struggle with this task</a>, dispelling the myth of “digital natives”. In general, most users (young and old) adopt a time-saving cognitive shortcut: rely on the information Google displays at the top of its search result page, as this aptly named paper, “<a href="https://academic.oup.com/jcmc/article/12/3/801/4582975">In Google We Trust</a>”, explains. However, while Google makes it easy for users to find content, it is not that helpful for conveying information about the sources that produce content on the web.</p><p>Take for example Top stories in Google search results, which I have been <a href="https://medium.com/@enimust/are-googles-top-stories-politically-biased-it-s-complicated-e9c68e269ed9">auditing recently</a>. Whenever you search for people or events in the news, Google shows as the first result a panel with Top stories (Figure 1). These stories come from various news outlets, some of them well known and others not. Let us consider the three sources for the Top stories in Figure 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VHJ0GB-Z-Cf5OzXxUH1cCw.png" /><figcaption>Figure 1: Screenshot of Top stories for “michelle obama”. This result was captured on Oct 8, 2018, at 4:08 PM EST. Three stories are shown, but one can scroll horizontally to see up to ten stories.</figcaption></figure><p>Depending on where you live (in the U.S. or Britain), you might recognize <em>abc7NY</em> and <em>The Independent</em> and if not, googling for them leads you to their Wikipedia pages. However, there is no Wikipedia page for <em>BYU-I Scroll.¹ </em>As <a href="https://dl.acm.org/citation.cfm?id=3201095">our previous research has shown</a>, only about 38% of news publishers in the U.S have Wikipedia entries that show <a href="https://medium.com/@enimust/the-information-panels-on-google-and-facebook-uncovering-their-blind-spots-2e8210b2e697">as Knowledge Panels in Google</a>.</p><p>How to verify that <em>BYU-I Scroll</em> is a legitimate news source, instead of an impostor website, as the many fake news sites from 2016? Here I want to turn this question around. Why should every Google user have to do this verification? Isn’t it Google’s job to make sure that only reliable sources are shown in Top stories, especially after the debacle of including <a href="https://mashable.com/2017/10/02/google-algorithm-fail-las-vegas-gunman-fake-news/#9BkewPSMHOqJ">news from 4Chan in Top stories</a>? Google <a href="https://www.theverge.com/2017/10/3/16413082/google-4chan-las-vegas-shooting-top-stories-algorithm-mistake">hasn’t shared details</a> about how sources for Top stories are selected, but for Google News (which also show news stories from thousands of publishers from around the world), there is <a href="https://support.google.com/news/publisher-center/answer/7526416?hl=en&amp;ref_topic=9010462">a long verification process</a> that takes 1–3 weeks and involves human judgment. We can assume that as part of this verification, Google is collecting some kind of information about these news sources.</p><p>Google should make some of this information easily accessible to users, either as part of the Top stories card or by making the names of the sources links that lead to verification information shared on a dedicated Google page. Users shouldn’t have to be burdened with verifying news sources that Google has already verified.</p><h4><strong>Why should Google share information about news publishers in Top stories?</strong></h4><ol><li><strong>The large number of unfamiliar sources displayed in Top stories</strong>. Google shouldn’t direct users to unfamiliar sources without providing first some basic information about them (e.g., full name, location, category). <a href="https://medium.com/@enimust/are-googles-top-stories-politically-biased-it-s-complicated-e9c68e269ed9">As described in this post</a>, I’ve collected several thousand Top stories panels. Here is <a href="https://docs.google.com/spreadsheets/d/1gLxT9CvFzK12xXMZeB34MbevQqL7v0VNztnr7FStHJ4/edit?usp=sharing">a list of news publishers</a> (more than one thousand) that I found. The majority of these names are unfamiliar to me and I suspect to most other readers as well. Our analysis indicated that the sources that are displayed in the 1st and 2nd positions of Top Stories are usually major national news outlets, but the 3rd position is often a local or niche source. It seems like Google is trying to encourage us to burst the “filter bubble” by presenting us with local sources (like <em>BYU-I Scroll</em>, a student newspaper in Idaho).</li><li><strong>Inform the public about news sources and news production.</strong> When searching Google for news, users are directed to the websites of various types of news organizations: national broadcast TV stations, their local affiliates, radio stations, print newspapers, personal blogs, magazines, watchdog organizations, etc. Providing the source’s category builds awareness of the rich and varied landscape of news production. Not all sources have the same journalistic standards or the same news gathering means. By making their category explicit , Google could help the public think more deeply about how news is produced and the nature of these stories.²</li><li><strong>Avoid intentional or unintentional confusion through name variations.</strong> The <a href="https://docs.google.com/spreadsheets/d/1gLxT9CvFzK12xXMZeB34MbevQqL7v0VNztnr7FStHJ4/edit#gid=0">list of sources I shared</a> contains many duplicates (highlighted in shades of gray), that is, the same website is shown in Top stories with different names. Sometimes the differences are very minor, maybe an extra space or letter (<em>The Hill </em>and <em>TheHill)</em>, or a longer name is used that better captures the identity of the source (<em>LEX18 Lexington KY News</em> versus <em>LEX18.com</em>), but sometimes the purpose of different names is not clear or beneficial. Some examples are shown in Figure 2. <em>The Tennessean</em> has a news story about midterms, but instead of its name, the card shows “Policing the USA” , which is not a news organization that exists. Similarly, there is a story about Jeff Sessions from <em>ABC 33/40</em>, a TV station in Alabama, that apparently calls itself “Alabama’s News Leader”, which is obviously not their name. Finally, there is <em>CBN</em> (Christian Broadcasting Network) which appears in our dataset with five different names (<a href="https://docs.google.com/spreadsheets/d/1gLxT9CvFzK12xXMZeB34MbevQqL7v0VNztnr7FStHJ4/edit#gid=0">lines 181–185 in this document</a>). It is puzzling that a news organization needs to use five different variations of their name.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*mNChov7rKNJC8tKa136b-g.png" /><figcaption>Figure 2: Three Top stories cards (from different queries), which show sources that use different names to sign their stories.</figcaption></figure><p>There are more reasons than the ones I’ve listed here, but these strike me sufficiently important to warrant more contextual information from Google.</p><h4><strong>In Conclusion</strong></h4><p>Google is continuously indexing the content of several thousands news sources world-wide, this is why it is able to show “fresh” news on its Top stories panel, often only a few minutes old. If Google “trusts” these sources, it should share some reasons for this trust with its audience. Since the fake news crisis of 2016, Google has <a href="https://techcrunch.com/2018/03/20/google-news-initiative/">been investing hundreds of millions of dollars</a> in strengthening journalism. However, in this blog post, I’m advocating for a feasible intervention in Google’s interface: Google, turn Top stories into an educational tool! Don’t only use it to show news headlines, but also to provide additional information about sources that created the news. This would be a way to demonstrate by example the importance of considering the source alongside the content, because news doesn’t come from nowhere. Wineburg’s book³ invites us all to learn to think like historians, by always considering the source of information. Google can and should help with this task!</p><h4><strong>Footnotes</strong></h4><p>[1] BYU might be a recognizable acronym to many Americans. However, to me, an immigrant, it was unfamiliar.</p><p>[2] Google adds the word “Satire” or “Fact-check” to articles in these categories.</p><p>[3] The phrase “Google-drenched society” is borrowed by Wineburg’s book, <a href="https://books.google.com/books?id=PIZuDwAAQBAJ&amp;pg=PA3&amp;lpg=PA3&amp;dq=%22Google-drenched+society%22&amp;source=bl&amp;ots=pzxasS3gDe&amp;sig=PYosA86JTcFLjA2FixHsRsrCDwA&amp;hl=en&amp;sa=X&amp;ved=2ahUKEwi43u_erqzfAhXvmuAKHd_CAzcQ6AEwAHoECAEQAQ#v=onepage&amp;q=%22Google-drenched%20society%22&amp;f=false">see page 3</a>.</p><h4>Acknowledgement</h4><p>I’m grateful to my research collaborator <a href="https://twitter.com/@exploredeeper">Emma Lurie</a> for insightful discussions to improve this blog post.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4be35132b3d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Are Google’s Top Stories Politically Biased? It’s Complicated.]]></title>
            <link>https://medium.com/@enimust/are-googles-top-stories-politically-biased-it-s-complicated-e9c68e269ed9?source=rss-82d778d27ffe------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9c68e269ed9</guid>
            <category><![CDATA[top-story]]></category>
            <category><![CDATA[media-bias]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[algorithm-auditing]]></category>
            <category><![CDATA[kavanaugh]]></category>
            <dc:creator><![CDATA[Eni Mustafaraj]]></dc:creator>
            <pubDate>Sun, 11 Nov 2018 17:37:41 GMT</pubDate>
            <atom:updated>2018-11-11T18:27:36.239Z</atom:updated>
            <content:encoded><![CDATA[<h4>An Algorithm Audit of Top Stories During the Confirmation of Brett Kavanaugh</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j3h36eCak2oosKywhGx6FA.png" /><figcaption><strong>Figure 1</strong>: The “Top stories” results for the query “kavanaugh fbi investigation” on Nov 3, 2018, 8am. On the left-side image, the first two are right-leaning publications and the third source is <em>Russia Today</em> (RT.com). Two hours later on the same day (right-side image), the Top stories panel had a new format with two new sources and a different ranking order.</figcaption></figure><p>In August 2018, President Trump <a href="https://twitter.com/realdonaldtrump/status/1034456273306243076">tweeted</a> the following about Google:</p><p><em>Google search results for “Trump News” shows only the viewing/reporting of Fake News Media. In other words, they have it RIGGED, for me &amp; others, so that almost all stories &amp; news is BAD. Fake CNN is prominent. Republican/Conservative &amp; Fair Media is shut out. Illegal? 96% of results on “Trump News” are from National Left-Wing Media, very dangerous. Google &amp; others are suppressing voices of Conservatives and hiding information and news that is good. They are controlling what we can &amp; cannot see. This is a very serious situation-will be addressed!</em></p><p>This tweet was informed by an article published on <em>PJ Media</em>, titled <a href="https://pjmedia.com/trending/google-search-results-show-pervasive-anti-trump-anti-conservative-bias/">“96 Percent of Google Search Results for ‘Trump’ News are from Liberal Media Outlets”</a>. After its publication, various sources, including <a href="https://www.washingtonpost.com/opinions/i-wrote-the-article-about-media-bias-in-google-searches-regulation-isnt-the-answer/2018/08/29/15bdaae2-abaa-11e8-8f4b-aee063e14538_story.html">the author</a>, made it clear that it was <a href="https://www.politifact.com/truth-o-meter/statements/2018/aug/29/donald-trump/no-96-google-news-stories-trump-arent-left-wing-ou/">not a scientific study</a> with generalizable results, but a one-time snapshot observation. Although this particular inquiry was methodologically flawed, its premise is worth investigating: what sources does Google show when one searches for events or people in the news? Is there anti-conservative bias on Google?</p><p>With my students at Wellesley College, I am working on these questions from multiple perspectives. In this blog post, I will discuss one case study that focuses on the Top stories shown in Google search during Brett Kavanaugh’s confirmation hearings. <strong>Here is a summary of what we learned</strong>:</p><ol><li><em>Fox News</em> was the source with the largest number of unique stories in our (two month long) data collection. It was also the only conservative news outlet in the top 10 most prevalent sources.</li><li><em>Fox News</em> stories were more likely to be in the first position in the Top stories than any of the other media outlets.</li><li>The competition for the 1st position in Top stories is “fierce”, with only 21% of sources being able to secure the position. Meanwhile, the 3rd position is up for grabs frequently, 59% of sources were found at least once in that position.</li></ol><p>What do these results suggest? A simple interpretation is that Kavanaugh’s confirmation was the biggest news story of the season and Google users across the political spectrum were extremely engaged and thirsty for updates. While non-conservative readers have many options to access up-to-date news, <em>Fox News</em> is the dominant conservative outlet, as <a href="http://www.journalism.org/2014/10/21/political-polarization-media-habits/">a Pew study found out in 2014</a>. By choosing <em>Fox News</em> more often, Google’s algorithm was most likely reflecting the popularity and authority of this news source in this particular news event. However, one conspiratorial interpretation of the strong showing of <em>Fox News</em> would be to suggest that Google has been actively promoting conservative content to balance for the higher number of center and liberal sources. Thus, one could claim that Google is displaying a conservative bias, the opposite of what many people, including President Trump believe. Can Google display conservative bias in some stories and anti-conservative bias in others? Nothing is impossible. We believe that researchers need to pursue several high-interest news stories over an extended period of time, in order to better understand whether the bias comes from the algorithm or reflects publisher and reader choices.</p><p>If you are curious about how we came to these findings, please read on.</p><h4><strong>Auditing Algorithms</strong></h4><p>How can we know whether Google has a political bias? Given that we cannot look inside Google’s “black box” algorithms, researchers have proposed a method called <a href="https://pdfs.semanticscholar.org/b722/7cbd34766655dea10d0437ab10df3a127396.pdf">algorithm auditing</a>. We performed a “scraping audit” by sending queries to the Google search platform and recording the results. However, there are several important variables to consider when auditing Google search results:</p><p><strong>The query phrase.</strong> Slight variations in the query phrase will alter the results, despite Google’s herculean efforts to “understand” the intent of the query. Concretely, as humans, we can agree that the queries “cruz beto” and “beto vs cruz” have the same intent (look up information about both candidates in the Texas 2018 US Senate race), and while the algorithm displays similar results, they are not identical (Figure 2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BjJp3V6BmTLDAswpZUXc1w.png" /><figcaption><strong>Figure 2:</strong> Results for two similar queries: “cruz beto” and “beto vs cruz”. Two of the three stories are the same, but the order for the 1st and 2nd one differs.</figcaption></figure><p><strong>The location</strong>. For queries such as “weather” or “movies”, Google uses our current location (inferred from the IP address of the device) to show the local weather or movies playing near us. This works for other queries as well. For example, my search for “what’s on the ballot”, showed in the third place a link to my local government, despite the lack of explicit geographical location on the query (Figure 3).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lK4Nd6T83GXjZvbUAigIHA.png" /><figcaption><strong>Figure 3:</strong> A screenshot that indicates how Google search uses a the device location to provide location-specific results.</figcaption></figure><p><strong>Individual search history.</strong> Google’s algorithms learn over time our informational preferences and adjusts themselves to better adapt to these preferences. This is known as personalization, but another name for it is the “filter bubble”, a term popularized by Eli Pariser in his <a href="https://www.ted.com/talks/eli_pariser_beware_online_filter_bubbles?language=en">2011 Ted talk</a>, in which he shows side-by-side examples of two searches for Egypt (performed by two of his friends), with one page containing news about the protests in Egypt and the other with no mention of them.</p><p><strong>Device type</strong>. Mobile phones and laptop/desktop computers are different (internet speed, page loading time, screen size, modes of interaction, etc.), thus when creating a list of results for different devices, Google takes these factors into account.</p><p><strong>Google products.</strong> Search results about news events depend on the Google product being used. <em>PJ Media</em>’s author searched the News tab of Google search. Another Google product people use regularly is <a href="https://news.google.com/">Google News</a>. Meanwhile, Google’s traditional search also shows news stories in a carousel format at the top of the page, labeled as Top stories. This feature has become particularly popular on mobile phones, <a href="https://www.poynter.org/news/google-shifts-mobile-its-referrals-news-sites-keep-growing">increasing referrals</a> from Google search to publishers.</p><p><a href="https://dl.acm.org/citation.cfm?doid=3290265.3274417">Researchers who audit Google search results</a>, typically recruit geographically diverse participants to study how location, personalization, or query differences influence the search results. However, they have found that personalization might not play as big a role as previously suspected. In our study, we are interested in the baseline behavior of the algorithm for ranking news sources (without the adjustments it makes for location and personalization), therefore we keep many of these variables constant. We use the same laptop computer, with the same IP address, and the same blank-state browser in incognito mode (one device, one location, and no search history personalization). We programmatically send the same queries repeatedly to Google search and then capture the composition of the Top stories panel. This method allows us to perform a temporal analysis of the Top stories ranking algorithm.</p><h4><strong>A case study: Kavanaugh’s Top stories</strong></h4><p>For the remainder of this post, I will focus on one of the biggest political events of 2018: Brett Kavanaugh’s confirmation to the U.S. Supreme Court. Over a period of 8 weeks (Sep 5 — Nov 3), we tracked the following queries multiple times a day and almost always observed the presence of the Top stories panel: <em>brett kavanaugh</em> (342 observations), <em>kavanaugh hearing</em> (342), <em>kavanaugh confirmation</em> (342), <em>kavanaugh vote</em> (227), <em>kavanaugh</em> (278), <em>kavanaugh fbi investigation</em> (157). Some queries have fewer observations because they were added later to our set of queries. Each observation consists of 3–10 stories that appeared in the Top stories panel (3 for lower-ranked Top stories, such as the right-side example in Figure 1) and 10 for the carousel-style Top stories (left-side example in Figure 1). We discovered a total of 3,708 news stories (with unique URLs) created by 332 news outlets¹. Here are some of the results of our analysis.</p><h4><strong>Top 10 Most Represented Sources (by number of unique stories)</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/503/1*4_jix0fHeAE5bCiiEzhMtw.png" /></figure><p>This list is composed of the largest cable and TV networks, national newspapers, and online publishers that cover political issues. Together, these 10 sources contributed 48% of all stories shown during this period. Thus, publishers that produce more news stories appear more frequently on Top stories.</p><p>According to <a href="https://www.allsides.com/home">AllSides</a> (a nonprofit that uses experts and the community to label bias in media), <em>Fox News</em> is the only right-leaning source among the sources in our list. Should more right-leaning sources be on this list? Outside the top 10, we found several right-leaning sources: <em>The Washington Examiner</em> (13th overall) with 57 stories, <em>Wall Street Journal</em> (16th) with 52 stories, and several other sources clumped together in 20th position: <em>Breitbart</em> (41 stories), <em>New York Post</em> (41 stories), and <em>National Review</em> (40 stories). What factors determine the order? What percentage of each publisher’s stories appear on Top stories? We’ll address these new questions in our future work.</p><h4><strong>Which publishers appear most often in the Top 3 positions in Top Stories?</strong></h4><p>Although the Top stories panel is a carousel of 10 stories, users initially see three of them. Thus, the first three visible positions constitute a “first impression” that likely influences readers perception of the evolving story. So, which news outlets are displayed in the top three positions?</p><ul><li>1st position: 70 sources (out of 332)²</li><li>2nd position: 94 sources (out of 332)</li><li>3rd position: 195 sources (out of 332)</li></ul><p>The major takeaway here is that in a crowded space of publishers, some have a better chance than others to “secure” the 1st position. Here is a look at the publishers who did so most often (in terms of the number of unique stories):</p><ol><li><em>Fox News</em> — 34% (113 out of 330 stories)</li><li><em>CNN.com —</em> 29% (80 out of 282 stories)</li><li><em>The New York Times — </em>28% (35 out of 126 stories)</li><li><em>NBC News — </em>19% (25 out of 133 stories)</li><li><em>Washington Post — </em>15% (30 out of 206 stories)</li></ol><p>It seems that when it comes to placement on the 1st position, the number of stories published doesn’t weigh the most. <em>The New York Times</em> and <em>NBC News</em> both had fewer stories than <em>USA Today</em>, <em>The Hill</em>, and <em>Vox</em>, but were chosen more often for the 1st position. Most likely having a bigger audience (which is correlated to name recognition) matters more. Is the algorithm rewarding sites that are already popular? It seems that way. Kavanaugh’s confirmation was the biggest story in the U.S for more than a month, thus its coverage was non-stop, as it was the interest of the public. Therefore, the algorithm might be rewarding popular sources, according to the “exploitation-exploration” tradeoff (see next section). However, this is only an hypothesis and we’ll need to study many events like this one to be able to generalize.</p><h4><strong>Position 3 is the “Exploration position”</strong></h4><p>As noted above, 195 out of 332 sources appeared at least once in the third position. Compared to the to the 1st position (only 70 sources appeared), the sources in the third position are much more varied. An example of that was <em>RT.com</em> (the Russian propaganda publisher) in Figure 1, but here are a few examples from other outlets which don’t usually publish politically-related content that appeared in the 3rd position:</p><ul><li><em>CBR</em> (Comic Book Resources): Samuel L. Jackson Reacts to Pulp Fiction/Brett Kavanaugh Mash-Up</li><li><em>Billboard</em> (music charts): Sara Bareilles Reveals She Released Feminist Anthem ‘Armor’ Early Because of the Kavanaugh Hearings</li><li><em>TMZ.com</em> (gossip tabloid): Marco Rubio Drowned Out by Protesters While Talking Brett Kavanaugh</li></ul><p>In machine learning, this strategy is known as the <a href="https://medium.com/@dennybritz/exploration-vs-exploitation-f46af4cf62fe">exploitation-exploration tradeoff</a>. Agents (humans or algorithms) typically exploit what they already know (for example, we always read news from the same 2–3 sources), but once in a while agents try something new (exploration). The data in this case study seems to suggest that Google’s algorithm is using the 3rd position in Top stories as the “exploration position” to 1) encourage users to engage with unfamiliar news sources or perspectives and to 2) learn from their behavior which sources are worth “promoting”.</p><h4><strong>So, is Google biased?</strong></h4><p>We found that <em>Fox News</em> (a conservative publisher) had the largest number of stories and the largest number of 1st positions in Google’s search panel of Top stories. The other sources, while not conservative, were a mix of very popular center or left-leaning sources. This particular case study indicates that publishers and audiences play an important role in shaping the composition of the Top stories, maybe more so than any political bias. If anything, Google might be actively trying to burst the “filter bubble” by ensuring that a wide range of sources are displayed in the 3rd position.</p><p>We’ll continue monitoring Google’s Top stories and share more findings. If you have questions or requests please leave a comment.</p><h4><strong>Footnotes</strong></h4><p>[1] Automatically, our program found 332 unique names in the source field of an article. However, an inspection of this list revealed that several sources are listed with different names. For example, we found both <em>The Huffington Post</em> and <em>HuffPost</em>, which are the same organization, or <em>Hot Air</em> and <em>HotAir.</em> As a result, the number 332 is an overestimation. What is the source of such errors? It is not clear to us whether the publishers or Google’s algorithms choose the names that appear in these stories. We will try to find out in future work.</p><p>[2] As mentioned in [1], these numbers are overestimations, because we didn’t collapse duplicates into a single source. We only did this for the top 10 sources. Concretely, <em>Fox News</em> contains also the stories for <em>Fox News Insider</em> and <em>Fox Business</em>. <em>CNN</em> and <em>CNN.com</em> were merged together, as well as <em>USA TODAY</em> and <em>USA Today</em>, <em>Huffington Post</em> and <em>HuffPost</em>, <em>NBC News</em> and <em>NBCNews.com</em>.</p><h4><strong>Acknowledgement</strong></h4><p>I’m very grateful to my research collaborator <a href="https://medium.com/@exploredeeper">Emma Lurie</a> for great discussions and suggestions to make this post clearer.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9c68e269ed9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The information panels on Google and Facebook: Uncovering their blind spots]]></title>
            <link>https://medium.com/@enimust/the-information-panels-on-google-and-facebook-uncovering-their-blind-spots-2e8210b2e697?source=rss-82d778d27ffe------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e8210b2e697</guid>
            <category><![CDATA[wikipedia]]></category>
            <category><![CDATA[news-literacy]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[misinformation]]></category>
            <category><![CDATA[facebook]]></category>
            <dc:creator><![CDATA[Eni Mustafaraj]]></dc:creator>
            <pubDate>Sun, 23 Sep 2018 22:18:44 GMT</pubDate>
            <atom:updated>2024-05-30T21:18:40.443Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*AZMHZdnV-Q6BBqg4VTnGlg.png" /><figcaption>Figure 1: Two Knowledge Panels that are displayed when one searches Google for the names of these publications. Notice that the text snippets (extracted from Wikipedia) don’t mention that these publishers have a history of inaccurate claims (read article for details on this).</figcaption></figure><p>Recently, Mike Caulfield wrote a <a href="https://twitter.com/holden/status/1043312691262021633">Twitter thread</a> and <a href="https://hapgood.us/2018/09/22/a-suggested-improvement-to-facebooks-information-panel-stories-about-not-from/">blog post</a> praising Facebook’s new information panel for news publishers. Mike and I are collaborating (with many others) on the <a href="https://hapgood.us/newspapers-on-wikipedia-charity-challenge/">NOW (Newspapers on Wikipedia) project</a>, to improve the coverage of local newspapers on Wikipedia, so that a relevant Knowledge Panel for these publishers can be displayed when one googles the name of a publication (see examples in Figure 1). Mike was inspired to start this project when he and I discussed <a href="https://dl.acm.org/citation.cfm?id=3201095">a paper</a>, co-authored with my student Emma Lurie, in which we calculated that only 38% of publishers had a relevant Knowledge Panel (from a dataset of more than 7,200 US-based media outlets listed in <a href="http://www.usnpl.com/">USNPL</a>).</p><p>In the meantime, Emma and I have continued this line of research — exploring what Knowledge Panels can tell us about the credibility of online sources, especially of the ones recognized as partisan or inaccurate. I’ll briefly summarize here some recent findings (we have a paper in submission that goes more into the details of the research):</p><h4><strong>1) Not all info panel text snippets are equally informative and they rarely address factual accuracy.</strong></h4><p>What should a text snippet convey about an unfamiliar publication? Our analysis of a few hundred such text snippets indicated large variability in how and whether these snippets mention political/ideological bias, founders, ownership, dates and places pertaining to the publication. Importantly, only rarely did a snippet mention issues of factual accuracy. Zero Hedge and Bipartisan Report shown in Figure 1 above are good examples. They both have a history of false claims (see <a href="https://www.snopes.com/tag/zerohedge/">Snopes for ZeroHedge</a> and <a href="https://www.snopes.com/tag/bipartisan-report/">Snopes for Bipartisan Report</a>), and were among the publications that Google provided with the panel of “Reviewed Claims” <a href="https://www.poynter.org/news/google-suspends-fact-checking-feature-over-quality-concerns">in a discontinued, fact-checking experiment</a>.</p><p>Concretely, if one were to read the opening paragraph of the <a href="https://en.wikipedia.org/wiki/Zero_Hedge">Wikipedia entry for Zero Hedge</a> (from which the text snippet in the Knowledge Panel in Figure 1 is extracted), one would learn some relevant tidbits, such as “source of … rumors and gossip”, “associated with alt-right”, and “pro-Russian bias”. None of this can be assumed by the snippet in the info panel, and this can have consequences. As <a href="http://brenthecht.com/publications/icwsm17_googlewikipedia.pdf">research</a> by Northwestern’s Brent Hecht and collaborators has shown, the presence of the Knowledge Panel in Google’s search results makes readers less likely to access the corresponding Wikipedia entry. Thus, in order for these panels to be meaningful in the context they are being used by Google and Facebook (to help users understand the reliability of a source), such text snippets might need to refer to a publisher’s lack of factual accuracy, as the examples in Figure 2 do.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/919/1*-jbtXj4oX16_478j8I0c4w.png" /><figcaption>Figure 2: Two examples of Knowledge Panels that explicitly address the lack of factual accuracy by these two publishers, by pointing out that they promote conspiracy theories.</figcaption></figure><p>In both cases, these websites are explicitly described for what they do, promote conspiracy theories.</p><h4>2) Snippets are being edited to improve/damage reputation or send certain signals to different audiences.</h4><p>We compared the content of Knowledge Panels for the search results of more than 1,100 publications, collected in January 2018 and September 2018. Change is inevitable, but some of these Wikipedia snippet changes make one wonder of their motives. In Figure 3, I show side by side two versions of the Knowledge Panel for Bipartisan Report. The January snippet contains a very blunt portrayal of Bipartisan Report (“clickbait”, “heavily skewed headlines”), while the September snippet has removed these references and added information that seems to send a signal of reliability based on ownership information (Jeff Brotman was the co-founder and president of Costco Corporation, a well-known wholesale retailer).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/1*zdP5SxtuzW4oYF_oZS2c9w.png" /><figcaption>Figure 3: Changes in the Knowledge Panel of Bipartisan Report, which convey two very different portrayals of this publisher.</figcaption></figure><p>Interestingly, the Wikipedia page (see Figure 4 below) acknowledges the issues with this article (and <a href="https://en.wikipedia.org/wiki/Talk:Bipartisan_Report">its Talk page</a> has a substantial discussion of the various changes), but if readers don’t click through to read the page, they will not know about the controversial depiction of this publisher. Should Google make users aware of this contested description? Or entirely avoid surfacing information from Wikipedia pages flagged for certain quality issues?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2BLDxMFqsO1FrmGqV8vhfg.png" /><figcaption>Figure 4: The Wikipedia page for Bipartisan Report has a large section at the top warning about its quality. However, that doesn’t seem to prevent Google’s algorithm from surfacing the text snippet on the Knowledge Panel, without an indication of the disputed content.</figcaption></figure><p>While the changes in the Bipartisan Report panel illustrate the possible use of the Wikipedia snippets to either damage or salvage the reputation of a publisher, there are other changes that are puzzling in their nature. Here is one, concerning the magazine American Renaissance, a white supremacist publication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*OAAhoEk5Vxwwf3JAAbICfQ.png" /><figcaption>Figure 5: Knowledge Panels for American Renaissance on Jan and Sep 2018. The change of the text snippets makes one wonder which audiences are being targeted.</figcaption></figure><p>Both text snippets shown in Figure 5 acknowledge that American Renaissance is a white supremacist publication, but the provenance of the categorization differs. In January, the snippet lists third-party, well-known organizations as sources for the “white supremacist” label, however, in the September snippet, we read that the publisher self-describes as a “white-advocacy organization”. This shift of perspective (who does the labeling?) needs to be a matter of debate. Should these information panels tell us what the organizations think about themselves (how is this different from “About Us” pages which literacy experts suggest to avoid) or how other (especially watchdog) organizations regard them?</p><p>I don’t have answers and I don’t know how we can solve these issues without increasing the burden on Wikipedia editors. However, I think it’s important to raise awareness about these issues, so that we continue to actively address them. Furthermore, Google and Facebook need to better acknowledge the limitations of their initiatives and increase their support for Wikipedia and other knowledge production organizations.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e8210b2e697" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The fake news story that fooled even Maggie Haberman]]></title>
            <link>https://medium.com/@enimust/the-fake-news-story-that-fooled-even-maggie-haberman-c22453c47169?source=rss-82d778d27ffe------2</link>
            <guid isPermaLink="false">https://medium.com/p/c22453c47169</guid>
            <category><![CDATA[credibility]]></category>
            <category><![CDATA[fake-news]]></category>
            <category><![CDATA[misinformation]]></category>
            <category><![CDATA[fact-checking]]></category>
            <category><![CDATA[platform]]></category>
            <dc:creator><![CDATA[Eni Mustafaraj]]></dc:creator>
            <pubDate>Wed, 19 Jul 2017 03:14:34 GMT</pubDate>
            <atom:updated>2017-08-09T02:59:36.174Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GxESMZDdUxJ5v3B4q2vWQg.png" /><figcaption>A screenshot of a (cached) deleted tweet by Maggie Haberman, who first tweeted a story from a fake news website and then deleted it, when her followers explained that it was already marked as fake by Snopes.com.</figcaption></figure><p>We all believe we’re too savvy not to fall for fake news stories, but it keeps happening to the best of us, every day. You can be <a href="https://en.wikipedia.org/wiki/Maggie_Haberman">Maggie Haberman</a>, the NY Times political reporter of the day, and fall for it. In fact, the busier you are, the easier it is, because platforms such as Twitter, Facebook, and Google don’t help us quickly evaluate the content we are being served. To Ms. Habberman, <a href="http://www.elle.com/culture/career-politics/a45485/maggie-haberman-new-york-times-trump-profile/">a notorious multi-tasker</a>, it happened after Snopes.com <a href="http://www.snopes.com/mass-grave-kkk-estate/">had already debunked the story</a> she tweeted about. This is because Twitter, differently from “post-2016 election” Facebook, has yet to start collaborating with fact checkers to surface such useful information to its overwhelmed users.</p><p>But, relying on third-party fact-checking might be already too late. In this case, the story was published on June 30, but the Snopes rebuttal came on July 2nd, two days after the story had gone viral on Facebook and Twitter. After two days, most people have already moved on to other stories. What might work instead, as <a href="http://onlinelibrary.wiley.com/doi/10.1111/jcom.12166/abstract">demonstrated by an experiment</a> by Georgetown professor Leticia Bode, is immediately following the story with a link to a trusted source that contradicts the false information. This, however, works when false information has been around for a long time, for example: no, vaccines don’t cause autism, or no, climate change is not a hoax.</p><p>What can one do, when a fact-check for a story doesn’t exist yet? I think the solution is simple: the platforms must surface useful signals that make it easy for us to verify the credibility of a web source before we start spreading information. I wrote about <a href="https://medium.com/@enimust/are-women-evil-hacking-googles-search-results-eebfbbffe179">the idea of “nutrition fact labels” in a previous post</a>, and the rest of this article explores more signals to display to users at the same time they see a story.</p><h3>The anatomy of a fake news website</h3><p><a href="http://journal.webscience.org/317/">In research dating back in 2010</a>, my colleague <a href="https://twitter.com/takis_metaxas">Takis Metaxas</a> and I documented the first Twitter bot-enabled political misinformation attack being orchestrated from a website, <em>coakleysaidit.com</em>, that was registered on the same day (Jan 15, 2010) as the Twitter bots which were spreading this misinformation. We showed that an easy way to check an unknown website is to verify its domain registration credentials and look at the timing of registration and entities involved. If someone would have done this (by using the <a href="https://whois.icann.org/en/lookup?name=jacksontelegraph.com">Whois service</a>) before the fake story on <em>the mass grave of tortured black men</em> went viral, they would have discovered that the website, J<em>acksonTelegraph.com</em> was registered on June 23, 2017, only one week before the KKK story publication, in the name of a Bob Smith, living in Singapore (see annotated screenshot below). Even the human checkers of Snopes.com missed this fact. Instead they wrote: <em>the website JacksonTelegraph.com appears to be a fake news website, because its “contact” page provides no physical address, no telephone numbers, no listing of editorial or business staff (or other personnel).</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lMyE5C-G0HICmdBaoWpDWQ.png" /><figcaption>Screenshot from whois.icann.org, a web service that allows to check the credentials of every website. It provides information about the person or company that registered the name of the website, on what date, and for how long. A registration of one year (like the one for JacksonTelegraph) is the minimum registration time, and might be a signal of a short-term business opportunity, often found in scam websites.</figcaption></figure><p>Manually checking the origin of a website’s domain costs time and most web users might not even know that it’s possible or what it means. But for the algorithms powering Twitter and Facebook, this check costs a fraction of a millisecond. It should be their responsibility to display to users information that warns about the credibility of a web source. Even an old-fashioned, one-person-operated website, such as <a href="https://www.scamadviser.com/check-website/jacksontelegraph.com">Scam Adviser</a>, can tell you to be cautious about a website like JacksonTelegraph, only based on its structural features and without using any complicated AI to fact-check the content of the stories. Why is this technically difficult for Twitter and Facebook? Technology can do these checks for us. It is the platforms that have to decide whether they want to support their users, or burden them with the tasks of manually verifying the credibility of web sources.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c22453c47169" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Presidents in the Klan? SEO techniques to hack history.]]></title>
            <link>https://medium.com/@enimust/presidents-in-the-klan-seo-techniques-to-hack-history-953e48dc413c?source=rss-82d778d27ffe------2</link>
            <guid isPermaLink="false">https://medium.com/p/953e48dc413c</guid>
            <category><![CDATA[google]]></category>
            <category><![CDATA[nutrition-fact-label]]></category>
            <category><![CDATA[search]]></category>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[algorithms]]></category>
            <dc:creator><![CDATA[Eni Mustafaraj]]></dc:creator>
            <pubDate>Wed, 19 Jul 2017 03:06:24 GMT</pubDate>
            <atom:updated>2017-08-15T15:17:38.032Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*rlkdiciNq5ifiIszHaK-Sw.png" /><figcaption>This screenshot shows how Google answered the question “presidents in the clan”. The answer was lifted from a website that calls itself “Nigeria’s Internet newspaper”.</figcaption></figure><p><em>Sometimes, Google can be bad for learning history. Very bad. </em>This is how Peter Shulman, an associate history professor at Case Western Reserve University in Ohio, started <a href="https://twitter.com/pashulman/status/834577016464760832">a Twitter thread on Feb 22, 2017</a>. Professor Shulman was lecturing on the reemergence of the Ku Klux Klan in the 1920s when a student asked the question: Was President Warren Harding a member of the KKK?</p><p>This is what happens next, as told by the journalist Adrienne Jeffries in an article for The Outline, <a href="https://theoutline.com/post/1192/google-s-featured-snippets-are-worse-than-fake-news"><em>Google’s featured snippets are worse than fake news</em></a>:</p><blockquote><em>Shulman was taken aback. He confessed that he was not aware of that allegation, but that Harding had been in favor of anti-lynching legislation, so it seemed unlikely. But then a second student pulled out his phone and announced that yes, Harding had been a Klan member, and so had four other presidents. It was right there on Google, clearly emphasized inside a box at the top of the page.</em></blockquote><p>Professor Shulman quite rightly asked publicly on Twitter: “how did such awful search results wind up featured on Google?” Does this question sound familiar? In my post <a href="https://medium.com/@enimust/are-women-evil-hacking-googles-search-results-eebfbbffe179"><em>“Are Women Evil? Hacking Google’s search results,”</em></a> the journalist <a href="https://www.theguardian.com/profile/carolecadwalladr">Carole Cadwalladr</a> asked the same. And the aftermath of these questions played in the exact same way. Once this new episode became public, one more time Google changed something (that we don’t know) in its algorithms. Searching today for “presidents in the Klan” doesn’t show a featured snippet anymore, but a Wikipedia article.</p><p>For myself, the interesting question is always: how do such dubious websites manage to fool Google’s intelligent algorithms?</p><h3>SEO Techniques to fool Google</h3><p>This particular website, <em>www.thetrentonline.com,</em> describes itself as “Nigeria’s leading Internet newspaper”. This is not a fake news website, it was registered in 2013 in Nigeria and seems to contain many stories about Nigeria and Africa, as well as stories from the Internet. These stories have catchy titles and lots of photos, a technique used by many clickbait websites that make a living through online advertising. It’s impossible to know for sure why this article was picked by Google (given the opacity of its algorithms), but here are some of the Search Engine Optimization (SEO) techniques this particular website used.</p><ol><li><strong>Repost content from others</strong>. I’m being generous and calling this technique reposting instead of stealing, because the article contains a reference to the original story. Reposting stories from other sources, by often embellishing them with catchy titles and photos, is a common SEO technique (because it’s cheaper than generating new content, and Google’s algorithms value meaningful content). The article about the Presidents in the Klan was originally posted in the blog, I Love Black People, and <em>thetrentonline.com</em> provides a link to it: http://iloveblackpeople.net/<strong>2014/07/</strong>five-us-presidents-were-members-of-the-ku-klux-klan/. However, this link is broken, but the story can be found in another link in the same website: http://iloveblackpeople.net/<strong>2016/03/</strong>five-us-presidents-were-members-of-the-ku-klux-klan/, reposted two years later.</li><li><strong>Appearance of fresh content</strong>. Searching the Nigerian website for the phrase “ku klux klan” displays nine times the article “REVEALED: 5 US Presidents Members Of Racist Cult Ku Klux Klan (PHOTOS)”, every time with a new date, starting in July 19, 2014, until the most recent one January 24, 2017. Reproducing the same article with different dates to show freshness of content is a typical SEO technique, but Google is supposed to not fall for this trick. Given that the last timestamp was only one month before Shulman’s class episode, it seems that this apparently “fresh” content did influence Google.</li><li><strong>Use of images with captions</strong>. <a href="http://iloveblackpeople.net/2016/03/five-us-presidents-were-members-of-the-ku-klux-klan/">The original post in the blog “I love black people”</a> didn’t contain photos of the US presidents. The Nigerian website improved the original article by adding photos of four presidents. To add the photos in the article they used the special HTML5 tags: &lt;figure&gt; and &lt;figcaption&gt; (use of such tags indicates technical sophistication, also valued by Google’s algorithms). The text in these four figure captions was what Google extracted to display in its featured snippet. The inconsistency between number in the title (five) and presidents in the article (four) would have alerted a careful human editor about the questionable quality of the article, but an algorithm is not trained to make such judgements about facts.</li><li><strong>Tags for the article</strong>. The Nigerian website assigned several tags to the post: ku klux klan, ku klux klan members, racists US presidents, US presidents who were members of ku klux klan. Such tags reinforce the content of the article, but were not part of the original blog post, which tagged the article simply under news and politics.</li></ol><p>This analysis makes clear one thing: using SEO techniques to create the appearance of a “carefully crafted” website, allows one to fool Google’s search algorithms and grab the coveted position zero as a featured snippet.</p><h3>What should Google do?</h3><p>The most desirable outcome would be for Google’s algorithms to capture the implicit topic of each query. For example, to a human, “presidents in the klan” sounds like a history question. Therefore, a human would value an answer that comes from a source that has some authority and expertise in history. One can argue that inferring the implicit topic of a query, given the ambiguity of natural language, is a difficult problem. I agree. Instead, my proposal is to provide the user with <strong>signals</strong> that would simplify the judgement of the source’s authority on this topic.</p><p>Concretely, if the featured snippet would have been augmented with a few signals about the source of this article, the students in Shulman’s class wouldn’t have been so quick in accepting Google’s answer. I have written about <a href="https://medium.com/@enimust/are-women-evil-hacking-googles-search-results-eebfbbffe179">the “nutrition fact label” approach to augment search results in another post</a>. In this particular case, it would have been helpful to show the following facts (because it would have likely put a student learning about the history of USA on guard):</p><ol><li>Location of website: Nigeria</li><li>Category of website: Internet News</li><li>Author: Wires | The Trent</li></ol><p>Notice that these facts are judgement-free. If Google had to say that this content was stolen or that the website contains a lot of clickbait articles, that would have been debatable, because that is how most of the websites function nowadays. However, users either don’t know or forget that Google doesn’t create or endorse the content that it displays, though often finds it in the most bizarre corners of the Internet. This is why it should be responsible for telling us what it knows about the source, so that we don’t need to put our trust blindly in its search results, by assuming that Google can vouch for the source.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=953e48dc413c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Are women evil? Hacking Google’s search results.]]></title>
            <link>https://medium.com/@enimust/are-women-evil-hacking-googles-search-results-eebfbbffe179?source=rss-82d778d27ffe------2</link>
            <guid isPermaLink="false">https://medium.com/p/eebfbbffe179</guid>
            <category><![CDATA[web]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[search]]></category>
            <dc:creator><![CDATA[Eni Mustafaraj]]></dc:creator>
            <pubDate>Wed, 19 Jul 2017 03:00:37 GMT</pubDate>
            <atom:updated>2017-09-11T14:08:53.554Z</atom:updated>
            <content:encoded><![CDATA[<p><em>And how Google can use its AI to make search better and empower its users.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XohgIiiKAejAG-kPYZcM6Q.png" /><figcaption>Google search results for the phrase “are women evil”. Original image appears in <a href="https://www.theguardian.com/technology/2016/dec/04/google-democracy-truth-internet-search-facebook">this Guardian’s article.</a></figcaption></figure><p><em>Are women evil?</em> When the journalist <a href="https://www.theguardian.com/profile/carolecadwalladr">Carole Cadwalladr</a> was suggested this search phrase by Google’s autocomplete, the results were not something she had imagined. <em>“Every woman has some degree of prostitute in her. Every women has a little evil in her.” </em>was displayed highlighted in a box, Google’s direct answer.</p><p>Why did Google show this result in the position zero? <a href="https://www.theguardian.com/technology/2016/dec/04/google-democracy-truth-internet-search-facebook">Cadwalladr talked to a few experts</a> who told her about SEO (search engine optimization) techniques, the power of network (sites linking to one another), and the information war being raged by the tech-savvy rightwing. She blamed it on Google. It worked! <a href="https://www.theguardian.com/technology/2016/dec/05/google-alters-search-autocomplete-remove-are-jews-evil-suggestion">Google cleaned up the results</a> the next day. However, Google didn’t explain how that happened in the first place, except for reminding everyone that it is not responsible for the content. Nor did Google explain how it changed its ranking algorithm, so that this particular article is now relegated in the 4th page of results.</p><p>I was curious to know how Google’s algorithm was fooled and so I investigated. Indeed, whoever wrote that article certainly knows about SEO best practices. Here are the details:</p><ol><li>The URL of the article contains the query in its title: <em>sheddingoftheego.com/2015/06/25/are-women-evil/</em></li><li>The title of the post, bolded in the &lt;h1&gt; HTML tag is also “Are women evil”.</li><li>The phrase “are women evil” is mentioned 8 times.</li><li>The word <em>women</em> appears 189 times, and the word <em>evil</em> 88 times. Together with the word <em>men</em>, they are the top three words (once you remove the so called stop-words: ‘the’, ‘to’, ‘and’) in this 10,000 word article.</li><li>Though written at the 8th grade level (according to the <a href="https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests#Flesch.E2.80.93Kincaid_grade_level">Flesch-Kincaid grade level score</a>), the article also contains many high information words: biological, unconscious, co-morbid, perception, cultural shift, etc.</li></ol><p>Taken altogether these signals make this article very relevant to the query. An algorithm that treats words as strings of characters — and is unaware that their meaning can cause offense — is doing the job it was designed to do: find the page that content-wise is the most relevant match to the query. And while Ms. Cadwalladr and many others wouldn’t think of asking for “are women evil”, there are apparently plenty of people who do it, who then link to or share articles like this one. That is how the Web works and how a search engine learns what people want to see. And some people know more of these secrets than others. If Google is doing us a disservice, it is by keeping us in the dark about how certain groups are taking advantage of its algorithms. As well as by making it difficult for us to know more about the <strong>source</strong> of information from where an answer or snippet is extracted.</p><h3>How can Google do better?</h3><p>What if Google’s snippets came with a “nutrition label” with facts about the web source and the particular page? Something like in this image that I have created:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Dpjhp90uyKh3niEG1VDLw.png" /><figcaption>A Google featured snippet that was augmented by the author with some facts about the webpage from which the context was extracted.</figcaption></figure><p>This information (I call them signals) is something that an AI algorithm can be trained to recognize. As humans, we can tell if a text is a personal opinion, especially if we see 87 times the pronoun “I”, as in this article. When we read that the author’s name is <em>Razor Blade Kandy, </em>we can easily recognize that it’s a pseudonym. Then there is the matter of what this blog is about. Its subtitle says <em>“A new interpretation of masculinity”. </em>A bit vague. Using a tool like Amazon’s Alexa to look up info about this site, one learns that it has a similar audience like<em> mgtow.com, </em>MGTOW = Men going their own way. There is <a href="https://en.wikipedia.org/wiki/Men_Going_Their_Own_Way">a Wikipedia page</a> for that. The acronym is also a category in this blog, thus, the topical affiliation is explicit, not something to be inferred.</p><p>If Google showed such “nutrition labels” for all sources of information in its search results (maybe with different ingredients based on the nature of the source), we would be able to quickly scan what sources are being shown and make our own choices about what to consume and with what kind of skepticism. Just like we do in a supermarket aisle, picking up products and looking at the labels for the amount of added sugar or cholesterol.</p><p>Google might say, “but our AI is not capable of producing labels like that.” If Google says so, we can ask: why not? Why can’t your engineers, some of the most intelligent people on this planet, work on a AI technology that really helps humans make good decisions, by providing constant reminders that the Web is not a well-meaning publishing company, but a universe of voices each with their own agenda. Transparency and contextual information is what we need, not spoon-fed answers. Because if AI is going to take over as the fear-mongers keep reminding us everyday, it will be because tech companies like Google are daily undermining our human agency.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eebfbbffe179" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>