One of the key values of Media Cloud is that it supports media analysis with a global footprint in ways few other resources do. Almost all of my research in Media Cloud includes some geographical components — my recent post on US pandemic news by state is an example that did so at the state level in the US. However, since Media Cloud is so versatile, it can be hard to figure out which approach to searching within places is the best one for you. This blog post explains the available options and makes some suggestions about when to use each.

1. Geographic Collections

Use our curated geographic collections if you want to search online news sites we have validated as published from a place.

Media Cloud has geographic collections for most countries and states/provinces. You can browse these in our Source Manager tool. We created these years ago by importing what we found on the ABYZ News Links website. Since then our own staff, working with partners based in various countries, have expanded and cleaned some of these collections significantly. That means these sources are what I refer to as “validated” — we’ve checked that they are news about that place and we are pulling in stories regularly via any RSS feeds we have associated with each source. It doesn’t mean that the site is necessarily still online; it might be a historical “dead” source that doesn’t publish anymore, but we retain it in our collection to preserve records of the content.

Each country has a “National” collection which includes media sources that cover the entire country or are not specific to a locale (like the India — National collection). Use this collection if you are interested in what the top news in a country is at a high level. “State & Local” collections for each country group together all the non-national sources (like the India — State & Local collection). Use that collection if you want a broader picture of more local or regional news in different places within the country. For many states or provinces we also have specific collections that you’d use if you care about what makes online news in a very specific part of a country (like the Kerala, India — State & Local collection).

The caveat here is that we of course aren’t experts in the online media ecosystem in every country. Some places have more robust coverage than others. Drop us a line if you know a country’s online media well and would like to help us flesh out better collections!

2. Sources Published in a Place

Use our “published in” media tags to search online news sites from a larger, less validated, set of sources we have marked as published in each place.

One of the pieces of metadata we store in our system about media sources is the country of publication. You can use that to search all the media sources we have marked as published in a place. This helps if you want to cast a slightly wider net and are OK with a little more noise in your results from non “validated” sources. This usually only adds a little more content to your search than using the national and state & local collections for a country. For instance, in India this technique returns about twice as many sources, but that only yields about 20% more stories than using the place collections (see an Explorer search showing that). This is because we have many sources marked as “published in India” that we are not collecting stories from dailly. Our system has limited resources, so we have to prioritize which sources we regularly ingest from.

3. Articles “About” a Place

Use our “about a place” story tags to search online news stories that our algorithms think are about a certain place.

The previous two approaches both help when you want to search online news media in a place, but what if you want to search media about a place? Our story processing pipeline takes the text of every story and runs it through our CLIFF-CLAVIN engine to tag it with places our algorithm thinks the story is “about”. You can use those tags to search all stories that are about a place.

What does “about a place” mean? A short description is that CLIFF uses a set of custom heuristics to determine which geographic place is being referred to in text, and then picks the countries and states/provinces with the most places mentioned in a story and tags the story with those. Read our academic paper about CLIFF to learn more.

To take this approach, you need to know the “magic tag” id for the place you are interested in. You can get that for a country by clicking on it in Explorer results, as shown above, or you can get the tag id from the CSV download on our support page.

4. Media Sources “About” a Place

Use our “about a place” media tags to search online news sites that most often write stories about a country.

One piece of metadata about media sources that we generate automatically is the country each source most often writes about. This is based on the “about” tags mentioned earlier. For each source we have a substantial number of stories from, we look at the country that shows up most often as the place it is about (via CLIFF as described about) and tag the media source itself with that country.

This is subtly different than the “articles about a place” approach. Using the media source “about country” tag searches all the stories from media sources that generally write about that country, including any stories that our system does not think are about that country. Using the story-level “about place” tag described above searches only the stories that are about the place, which includes countries and state/province tags. We don’t tag media sources with the state/provide that they most often write about.

5. A Place’s Name

Include the place’s name in your query string if you want to find any mention of a place in stories.

The techniques above can help you narrow in. If you want to make sure you don’t miss anything, you need to include the place’s name in your search query. This is especially helpful when you are searching at the city level. Of course, the problem is that you will probably get a lot of stories that mention similarly named places! For instance, searching for a place by name will include other similarly named places around the world. This approach increases “false positives” (stories that come back as matches but aren’t what you intended), but minimizes “false negatives” (stories that should have come back as matches but didn’t). If your place name is fairly unique, then the false negatives aren’t likely to be a problem. But if it is a generic or widely-used place name, then that will probably be a problem.

Other Notes

As you can see, there are a variety of approaches here to choose from based on what you are doing. This is, of course, both a strength of our platform — because you can tailor it to your needs — and a weakness — because it is confusing! Here are a few other things to keep in mind when you are doing research on news media from or about a place in Media Cloud:

• You can search by story language as well, adding “language:en” to search English language articles or “language:hi” to search Hindi language news articles.
• We also tag sources with the language they most often publish stories in, so you can search sources that most-often publish in a language. You can do this very similar to how you use a “custom collection” to use for media sources tagged as published in a place as shown above.
• Another research angle would be to search sources that are most read in a place, which of course could be a different list than the sources published in a country.

Thanks to Dennis Jen, Emily Ndulue, and Fernando Bermejo for comments and edits.

Approaches to Search by Place in Media Cloud was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.

By Aashka Dave (Tutorial videos by Rahul Bhargava, Orestis Papakyriakopoulos, and Aashka Dave)

Back in June, we hosted a tutorial session on Media Cloud at ICWSM 2020. Since that event took place online, we recorded the tutorial sessions and are sharing the videos as a resource for anyone interested in learning more about Media Cloud research and our latest updates. You can find the entire playlist of videos on YouTube. If you’d prefer to choose which videos to watch and get a quick summary of each one, you can follow the embedded links in this post.

Video #1: An Introduction to Media Cloud

This video provides an overview of why our team created Media Cloud and goes over what Media Cloud’s three tools — Explorer, Topic Mapper, and Source Manager — can be used for.

Video #2: An Introduction to Explorer

This video takes users through Media Cloud’s Explorer tool, which helps quickly study media attention, language and entities in global news reporting online.

Video #3: Explorer Case Study

In this video, we go over a Media Cloud research project looking at the language used to cover immigration in U.S.-based newspapers from 2014–2018. You can read an overview of that study here, and download the full report here.

Video #4: An Introduction to Topic Mapper

In this video, we take users through the latest version of Topic Mapper, which now supports cross-platform research. You can also find a written guide to cross-platform research in Topic Mapper here.

Video #5: Topic Mapper Case Study

This video takes viewers through our process for conducting cross-platform research using Topic Mapper. We present research conducted with the German Marshall Fund on mis/disinformative narratives about Greta Thunberg ahead of the global climate strikes in September 2019. This research project was used to inform how we designed and built cross-platform integrations into Topic Mapper.

Video #6: Using the Media Cloud API

Our last video provides an introduction to using the Media Cloud API. We take you through a set of Python3 Jupyter notebooks to help you understand how to use the API for your own media research and analysis.

We hope these tutorial videos are helpful for you, whether you’re just starting out with Media Cloud or you’ve been using the tools for years. If you have any other questions, feel free to visit our support page or contact us at support@mediacloud.org.

New Media Cloud Video Tutorials was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.

We’re excited to share our initial phase of support for cross-platform analysis within Media Cloud’s Topic Mapper tool. Starting today, you’ll notice new features and capabilities within Topic Mapper, namely the ability to ingest content to discover links shared on the open web, Reddit, Google search results, and from a self-uploaded CSV (that can also contain social media posts).

This post summarizes the tech changes and new data Media Cloud now provides and discusses how to use our enhanced version of Topic Mapper.

Since Media Cloud started in 2011, our tools have evolved to reflect new research questions, methodologies and technical abilities. So far, our research and technology has focused on news on the open web, tracking stories using RSS feeds and link networks. This latest update to Topic Mapper helps us further Media Cloud’s work in that space while also growing our tools to incorporate finding stories in different ways, from different places.

For many years now, mainstream news media has not been the only place where individuals can find information on the internet. This is especially relevant to the domain of research on mis/disinformation and hate speech, where we have been conducting more work lately. To that effect, we have been working for some time now on enhancing Media Cloud so it can incorporate links shared beyond the open web.

The need for this capability was underscored for us at the most recent Exploring Media Ecosystems Conference, hosted in early 2020 by Media Cloud’s non-profit arm, the Media Ecosystems Analysis Group. Though there are many researchers working on cross-platform questions, most technology in the space (including our own) is still built for one-off projects.

We took this first-pass at cross-platform topics to begin addressing those concerns and to provide a solid foundation upon which we — and others — could build. Leveraging our pre-existing strength in open web analysis, we decided to start by ingesting content from multiple platforms to extract URLs people are sharing online (in other words, we’re not studying the content itself). We used a plugin-based approach to discovering links shared on different platforms to build out cross-platform support.

Topic Mapper already provides three distinct advantages: (1) increasing a subject’s corpus of study by spidering to find more content, (2) letting you slice and dice along various parameters to analyze subtopics, and (3) and incorporating Facebook share counts and media inlink counts to support further analysis by source influence.

Those features still exist in the cross-platform version of Topic Mapper, but they have been augmented by the ability to increase your corpus through new platforms and data sources, and by the ability to evaluate additional influence metrics (see the table further in the post). The cross-platform version of Topic Mapper also allows for an increased number of research questions studying how links are shared across platforms and related patterns and trends.

How to Run a Cross-Platform Topic

This tutorial video, presented at the 2020 ICWSM Conference, takes Media Cloud users through the process of creating their own cross-platform topics:

If you’d prefer a written walkthrough of creating a cross-platform topic, keep reading. If you’re more interested in key features and our next steps, skip this section.

To create a topic, click on the “create a topic” button in Topic Mapper.

You’ll then be asked to enter basic information about your topic: its name, a short description and a date range. This is the date range that will be used in your entire topic unless you create a new version later on.

The “advanced settings” tab will let you select the number of rounds of spidering you wish to use for your topic; this can range from zero rounds of spidering (which means you’ll just use stories Media Cloud has already found for a given source or collection) to fifteen rounds of spidering (which means we’ll follow links within stories 15 times to see if any additional stories match your search parameters). Like before, public users of Media Cloud are limited to 100,000 stories/topic in order to avoid overloading the system. If you have questions about this limit or structuring your search query, email support@mediacloud.org.

At this point, you’ll be able to add platforms to your topic. Every topic must have an open web component, which is essentially the legacy functionality of Topic Mapper; topics do not have to use additional platforms as data sources. After you have added open web search parameters to your topic, you can use the following interface to add other platforms to your topic.

The initial platforms supported in topic mapper are listed in the table below:

After you have added open web search parameters (and any other platforms you’d like to include) to your topic, you’ll be able to generate the first version of your topic.

At this point, your topic will start running. Your topic will take more or less time to run depending on your search parameters, including the timespan of discovery, number of collections and platforms used, and the complexity of your query.

Your topic may also take more time to run if there is high user interest in Media Cloud at any given time (this usually happens when particularly newsworthy events are taking place). In order to give yourself time to have your topic run, we suggest budgeting anywhere from two days to two weeks depending on the complexity of your topic parameters. If you are waiting on a topic that hasn’t finished running after two weeks have passed, we recommend emailing support@mediacloud.org.

Key Features and Notes

Links found through any of the platforms mentioned above are then checked against your query terms and added to the corpus for spidering. Stories found linked to on Reddit are automatically put into a Reddit subtopic. In other words, if Media Cloud finds URL A and URL B in Reddit submissions, and URL C and URL D on the open web, all four URLs will be available on the homescreen for your topic, but URLs A and B will also be in a “Reddit” subtopic.

For any platform incorporated into a topic, metrics are calculated and aggregated within the corpus, which means you get a number for “relevant” shares rather than total shares within your platform’s subtopic. For instance, if the Centers for Disease Control and Prevention (CDC) are a major source of URLs within your topic, you might see a lot of Facebook shares for the CDC — but not all of those shares are relevant to your topic specifically. Instead of giving you a Facebook share count for everything from the CDC, Media Cloud returns the number of times CDC URLs were shared within your topic corpus.

Subtopics are a way for you to slice and dice your data within a topic for further analysis and evaluation. Subtopics are created for any platform added to a topic, but they can also be run along the lines of source partisanship in the United States, keyword searches within stories, and by media type, theme, or top countries within coverage. Keep in mind that searches by media type are dependent on the quality of metadata available for a given source or collection within Media Cloud (which you can evaluate using our Source Manager tool), and that subtopics based on themes and top countries are applicable only to English-language stories.

Versions are a way for you to track progress in your research and expand your corpus as your work on a given topic continues. When you create a new version in Topic Mapper, you will see three options, allowing you to expand your date range or increase the number of rounds of spidering in your topic, add or modify subtopics as detailed in the previous paragraph, and add or modify platforms, which is also how you would create platform-specific subtopics. Keep in mind that you can only create versions that expand your topic. You can’t remove data from a topic, though you can filter your topic’s data using subtopics, or revisit the data stored in a previous version.

To better understand how data is treated within a topic on Media Cloud, we recommend studying this topic dataflow diagram. (You can find a high-res PDF here.)

Conclusion and Next Steps

Cross-platform topics have been over a year in the making, thanks to extensive coding work from our technical team and careful thought from our research team to guide how to best support more research in Media Cloud. This work would not have been possible without funding from the Knight Foundation, and we are grateful for their support.

Going forward, we plan to add more platform plugins (as their APIs and such allow for). We are specifically planning to add data from verified Twitter accounts using PushShift.io and YouTube data via their API. We welcome input about our work so far, and hope that you find cross-platform topics as exciting as we do!

Links from table:

1. Media Cloud Query Guide
2. PushShift Query Guide
3. Standard Date Formats

Supporting Cross-Platform Research in Media Cloud was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.

Working on the open web is hard — getting the raw content out of a webpage is tricky, the content of stories posted online can change often, the same “story” content often shows up at different URLs, and stories can become unavailable if you try to see them a few months later (“link-rot”). These problems stimy anyone trying to research online news at scale, including ourselves. Various research projects try to work around these problems by employing techniques referred to as “normalizing”, “merging”, or “deduplicating” content. Whatever you call it, Media Cloud takes a multi-pronged approach. This blog post explains some of the problems, and the techniques we employ to address them in both Explorer and when you create a Topic in our Topic Mapper tool.

What is the real “content” of a story?

If you download a story from a newspaper’s website around 80% of the HTML content is not really part of what a reader would call the “story”. Things like javascript code, formatting, ads, and navigation controls make up the bulk of most news stories online. For our research, we just want to store the actual news content, not this cruft. Getting to that content can be tricky.

Like most other projects, we use a piece of 3rd party software to get the “news content” out of the raw HTML — we use version 0.7 of readability-lxml. This does a great job of removing all the non-content information. Every story we ingest is passed through readability-lxml.

However, we found that there was a specific way readability-lxml failed that was impacting our research. A common pattern is for news sites to include links to other stories. A few years ago we started seeing a design pattern where more and more of these links were long snippets, sometimes including the entire lede of the linked-to story. We saw that readability-lxml was including those links as part of the story text, because it was so hard to distinguish from the news content. These are very harmful to include as part of the story text, because they often are long links to the most popular “top” stories, so they can significantly impact our attention results.

Unfortunately the solution to this is quite complicated — it relies on the idea that these embedded links would appear across lots of stories on the same site. We added a layer of code that checks, for each calendar week, within a given media source, whether we see the exact same sentence repeating across stories. If we do, then for regular stories that you see in Explorer search results, we eliminate the sentence from all the stories except the first we found that includes it. The idea is that we are removing this redundant content, which is not actually part of the news story. We’ve found that this works well. It does have one weakness we noticed: it can sometimes get rid of repeated quotes. However, overall this approach helps far more than it hurts. When you create a topic we go another level and eliminate that first occurrence of the duplicated sentence as well, because we have had topics in the past that have been overwhelmed when we did not remove it.

What distinguishes one “story” from another?

Another issue is that we very often run into multiple copies of the same story — it can be in an RSS feed, linked to from another story, shared in a Tweet, etc. We take two approaches to trying to merge the many copies of a story we find into the same entry in our database — URL-based matching and title-based matching. Each of these steps is described below.

One norm of the open-web is that the same news story shows up at multiple URLs. For instance, the URL we find for a story in an RSS feed might be different than the one linked on their homepage; and one or both of those can sometimes be a “redirect” that takes you to a third URL for the story, and so on. As an example, here is a list of a dozen different URLs our system encountered for one single story from Reuters:

• https://www.reuters.com/article/us-usa-immigration/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-idUSKCN1TY1A5?feedType=RSS&feedName=domesticNews
• http://reuters.com/article/us-usa-immigration/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-iduskcn1ty1a5?feedname=domesticnews&feedtype=rss
• http://feeds.reuters.com/~r/reuters/domesticnews/~3/ndpydylrcnq/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-iduskcn1ty1a5
• http://feeds.reuters.com/~r/Reuters/domesticNews/~3/NdpYdYLrCnQ/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-idUSKCN1TY1A5
• https://www.reuters.com/article/us-usa-immigration/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-idUSKCN1TY1A5?feedType=RSS&feedName=worldNews
• http://reuters.com/article/us-usa-immigration/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-iduskcn1ty1a5?feedname=worldnews&feedtype=rss
• http://feeds.reuters.com/~r/reuters/worldnews/~3/zf78qrw6uzc/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-iduskcn1ty1a5
• http://feeds.reuters.com/~r/Reuters/worldNews/~3/ZF78QrW6uzc/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-idUSKCN1TY1A5
• https://www.reuters.com/article/us-usa-immigration/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-idUSKCN1TY1A5?feedType=RSS&feedName=topNews
• http://reuters.com/article/us-usa-immigration/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-iduskcn1ty1a5?feedname=topnews&feedtype=rss
• http://feeds.reuters.com/~r/reuters/topnews/~3/nachplxyqso/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-iduskcn1ty1a5
• http://feeds.reuters.com/~r/reuters/topNews/~3/NacHplXYqSo/u-s-probes-border-patrol-as-criticism-of-detention-centers-rises-idUSKCN1TY1A5

We begin to try to solve this like many other researchers, aggressively trying to “normalize” each URL to determine its most basic unique form. Interestingly, our approach to this often yields URLs for a story that don’t even resolve to a real web address (you can browse our source code for this “lossy normalization” process). The idea is to get down to the bare minimum amount of information in a URL that makes it unique. Once we have that, we save that “normalized” URL and connect it to the original story and URL. When a new URL comes in, we compare it to all the other normalized URLs from the same media source to look for matches. If there is no match, we add it to our system as a new story. We find this helps our research significantly, because we worry far less than we used to about whether we are seeing duplicates showing up. They still do, but at a much smaller percentage.

The second approach we take to matching stories with the same content is to use the title as a clue. Our software starts by getting rid of boilerplate information that is very often in article titles. Many publishers prefix, or postfix, titles with their website or newspaper name. For instance, the Boston Globe article pictured earlier has a page title of “Harvard, MIT sue to block Trump move to bar foreign students from US if classes are online — The Boston Globe”. Our code strips these away and also removes punctuation and converts the title to lowercase (you can review the source code for normalizing titles if you are curious). With that “normalized title” in hand, we then check it against all the other titles from the same media source on the same day. If there is a match, we add an entry so the URLs resolve to the same story ID in our database.

It is worth mentioning one of the things we do not do: we don’t refetch stories to track edits or changes. We fetch an article and save that version of it as the one true version. Our system doesn’t track an article over time to look for any changes or to try to get the latest copy of it. This limits us from some types of research questions, for instance digging into corrections or SEO optimizations, but doesn’t hinder our main body of work at all.

So how much does it help?

It is hard to come up with a rigorous quantification for how much these approaches help. Ideally you’d compare a “raw” dataset of stories to a manually reviewed and curated one. However at the scale we operate at that’d be very hard to do — our content and research just covers too many languages and technology norms to allow for a robust manual evaluation of how much each of these approaches mentioned here helps reduce duplicate content. In addition there are often qualitative decisions about how much two stories are actually the same one — sometimes it can be hard for a set of humans to agree on that. That said, over the years we have found that these approaches have identified about 25% of the stories within a given set as duplicates.

We hope this post has helped illuminate how we approach deduplicating our open web news content in Media Cloud, and why. If you have more questions or suggestions, do drop us a line at support@mediacloud.org.

Tech Brief: How We Deduplicate Content in Media Cloud was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.

From inaccurate self-diagnosing methods to false promises of a magical cure, COVID-19 mis- and disinformation grows daily and spreads rampantly, as people are desperate for the most up-to-date information about an ever-changing situation. The magnitude of pandemic-related mis- and disinformation has led to fact-checking sites like Snopes becoming overwhelmed with the quantity of requests. Large platforms like Facebook, Twitter, and Reddit released a joint statement discussing how they are “combating fraud and misinformation about the virus.”

This struggle to combat mis- and disinformation in global health was a focus of the 2020 Media Ecosystems Conference, hosted in March by the Media Ecosystems Analysis Group, the non-profit arm of the Media Cloud project based at the Center for Civic Media at MIT and the Berkman Klein Center at Harvard. The two-day conference, which was hosted once before in 2018, brought together various researchers, journalists, academics, and other stakeholders in the current media landscape.

The first day of the conference discussed disinformation across platforms and across the world, while the second day had a specific focus about these topics in the context of global healthcare. Presentations included research about India, the Philippines, Malaysia, France, Italy, the US, Nigeria, and the MENA region, and researchers from institutions in France, Italy, the US, UK, India, Nigeria, Canada, Spain, and Germany took the stage. In addition, and in keeping with the ecosystems approach, the research presented covered a wide range of platforms (WhatsApp, Facebook, Twitter, Reddit, Open Web, 4chan) and methodologies (from quantitative to qualitative, from interpretative to computer science approaches). These conversations are especially important in light of our current global situation, and this post highlights some of the great speakers and presentations made during the conference.

The conference opened with remarks from Ethan Zuckerman, the director of the Center for Civic Media. He discussed the purpose of the conference, posing a central question for the audience:”Can we build social media that is plural in purpose, public in spirit, and participatory in governance?”

This question was a good transition into the rest of the sessions on the conference’s first day, which focused on political mis- and disinformation. Session topics ranged from deepfakes and misinformation on WhatsApp in India all the way to the emergence of various narratives of disinformation on Greta Thunberg.

One talk in particular I really enjoyed was Alexei Abraham’s talk about the use of influencers in spreading pro-authoritarian discourse in the Middle East. His research at the Citizen Lab looked at trending hashtags to see if inauthentic or manufactured Tweets came from automated bot accounts or human-driven “influencer” accounts. His results challenge the narrative that bots are heavily used to spread pro-authoritarian discussion on Twitter. Instead, it provokes the hypothesis that members with heavy social media influence engage in a hierarchical space, where they are encouraged, incentivized, or potentially even coerced to guide public opinion. His talk resonated with me because it reminded me that constantly analyzing and questioning assumptions is central to researching the media landscape.

The second day of the conference opened with a keynote by Lisa Nakamura, a scholar of race and gender in internet studies. Her keynote focused on how empathy is commonly proposed as a solution to the racism and sexism that occurs on digital platforms, but this solution does not hold anyone accountable. Nakamura emphasized that while companies know how to create influential platforms, they don’t know how to be accountable for what occurs on them. The rest of the day of the day focused on mis- and disinformation in healthcare and their effects.

Jonathan Ong’s presentation, “How Racist Speech and Coronavirus Conspiracy Theory Evade Fact-Checkers and Platforms in the Philippines,” was particularly timely. His presentation focused on the rise of the anti-Chinese sentiment that emerged in the Philippines when President Rodrigo Duterte shifted the country’s foreign policy to ally with China. Ong spoke about how during Duterte’s re-election campaign, those politically opposed to Duterte would circulate anti-China extreme speech with paid trolls. And when Duterte was slow to impose travel bans on China as COVID-19 began to spread, influencers and those opposing him began circulating conspiracies about China and the origins of the virus. Ong discussed how circulation of these messages has led to racism against Chinese people in the Philippines, citing examples like ride-sharing drivers refusing service to Chinese customers.

Today, we see examples of racism emerging across the world based on harmful narratives about the origins of COVID-19. And while misinformation in public health is not a novel problem, its effects have been highlighted and exacerbated in today’s time. Just as healthcare workers and researchers are racing to develop vaccines and treatments, media researchers, scholars, and companies should be racing to develop better methods for researching, preventing, and correcting mis- and disinformation about global healthcare topics. When we consider issues like health misinformation in the time of a pandemic, we remember that these challenges in the media ecosystem cut across borders, platforms, and subject matter. By sharing our learnings about these topics on a global scale more frequently, and learning from media research in fields ranging from politics to healthcare, we can more effectively address and combat mis- and disinformation .

To learn more about this year’s Exploring Media Ecosystems Conference, visit this page.

Takeaways from the 2nd Exploring Media Ecosystems Conference was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.

Google News has me figured out and sporadically presents me with breaking sourdough news, including Scientist Says Bread He Baked From 4,500 Year-Old Yeast Is Delicious. I covet. The past few weeks, I’ve received a steady stream of sourdough stories, far different from the trickle I was used to getting. This coincided with my interest in using Media Cloud, a media analysis platform, so I decided to get acquainted with the tooling by diving into a topic of personal interest.

I started by getting a pulse check using Explorer based on US News sources. Is sourdough trending? How does it compare against baking in general? The results follow:

Although these results confirm my hunch, they also raise a few questions: what do the peaks leading up to April indicate? Many of the peaks occur in November and July, where Thanksgiving and Independence Day fall. I intuit that some of these molehills can be attributed to holiday baking projects.

My driving question is: in what way is sourdough spoken about and how has it changed over time? To answer this, I used Topic Mapper to gather and analyze stories related to sourdough. The topic is seeded by the term “sourdough” with articles collected between November 2019 through April 2020.

Topic Mapper allowed me to slice the topic into different timespans. Looking at a word cloud for November 2019, I unsurprisingly see many words (e.g. turkey, thanksgiving, stuffing, pie) related to the US Thanksgiving, which takes place at the end of the month. During this month, sourdough is often used as an ingredient in recipes (e.g. sourdough stuffing).

Toggling to different months gave me a sense of how sourdough was used; however, to understand how the topic evolved, I wanted to see the top terms in one view. I download the term frequency data (used to power the word clouds) for each month of my dataset. I then aggregated the 20 most frequently used words per month and created a parallel coordinates visualization, a go-to technique I use to visualize high dimensional datasets.

I’ll briefly describe this technique for those unfamiliar. Each axis corresponds to a month. A term (e.g. stuffing, flour) is represented by a vertical line. Where the line intersects the axis indicates how frequently the word was used in articles collected that month, where high frequency intersects at the top of the axis. Read more at the parallel coordinates wikipedia page.

To illustrate this, let’s visualize the term “sourdough” in this way:

“Sourdough” is frequently mentioned; however, the term isn’t always the most frequently mentioned, and is, in fact, lowest in November 2019. Articles about Thanksgiving dominate during this month, which I’ll illustrate using the visualization to draw out distinct words used during this month by isolating terms used frequently during November. In the following figure, I do this by selecting the lines that intersect at the top of the November axis (they are frequently used) and at the bottom of all other axes (they are not used during these months).

“Turkey,” “stuffing,” “thanksgiving,” and “pie” all evoke images of a traditional Thanksgiving dinner. It’s a heartwarming image in this time of isolation, but more importantly for my investigation, it also gives me confidence in my technique.

Similar to how I isolated words in November, I selected frequently used words in both March and April of 2020 to see the following results:

“Sourdough,” “flour,” “baking,” “starter,” and “dough” suggest the stories are about making bread from your own sourdough starter. Why might this be the case? Using Media Cloud, I can view the most shared articles on Facebook. A syndicated article with the title Experts confirm: Baking really is the best way to alleviate stress appears throughout the top listings. Anxiety baking resonates with me. Easy Recipes to Cook While You’re Self-Quarantined suggests there’s no time like social isolation to master sourdough.

This was a really fun exercise for me, and I’d like to conclude with two treats:

• Communal Tins of Sourdough Starter Are Popping Up On Trees in San Francisco on Eater. It’s a delightful little article about sharing sourdough starters. I remain grateful to Julia for sharing her starter, but part of me yearns to tell about finding my “start” in a little jar of goop tied to a tree.
• My latest bake:

The Rise of Sourdough was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.

By Fernando Bermejo, Luigi Reggi, Simona Tiribelli, Ethan Zuckerman

In US media, virtually every national or international news story is seen through a partisan lens. It’s reasonable to hypothesize that the novel coronavirus and COVID-19, the disease it causes, would follow a similar pattern. In particular, given President Trump’s attempts to minimize the impacts of the disease, we might expect to see less attention from right wing media than from more mainstream sources popular with the political center and center-left.

We used research from Benkler et. al., that classified a large set of US news sources into left, center-left, center, center-right and right, based on who retweeted which stories during the 2016 election. People who retweeted Donald Trump were very likely to retweet Fox News and Breitbart, for instance, while people who supported Hillary Clinton were more likely to retweet the New York Times or CNN. Media Cloud has media collections based on this research preloaded, so this is a very easy search to set up.

Searching US media on Media Cloud using “coronavirus OR COVID OR wuhan” as search terms, from January 1 to March 25, using the five partisan media collections from the Benkler research, we see the following pattern:

The chart clearly exhibits the three-step pattern Fernando Bermejo described — a low rumble of stories (under 10% of total stories) while the novel Coronavirus was a story primarily in China; a second plateau under 20% of stories as the virus began to spread in the US, and a sharp jump in early March, as coronavirus became the dominant story in the US. Unexpectedly, the curves move in near lockstep — while center-left (which includes journalistic powerhouses like the New York Times and the Washington Post) leads the pack with more than 2/3rds of stories focused on coronavirus, highly partisan media on both the left and right are not far behind, both clocking in at over 60%. Center and center-right media are the exceptions, crossing the 50% threshold recently, but continuing to focus roughly half their coverage on other topics.

A likely explanation for this phenomenon is that local newspapers and television stations are well represented in the center and center-right collections, and those new organizations continue to report local news, even news without coronavirus connections. Moving to specific sources makes the divergence between left and right media significantly clearer.

Beginning in late January, CNN — the network Trump often belittles as “fake news” — began covering the coronavirus story in detail, while it got significantly less play on Fox News, the President’s preferred news outlet. One exception was on February 15, when US planes evacuated Americans living in China to US airbases, a story that portrayed Trump as taking decisive action on coronavirus. It’s not until March 12, a day before Trump declared a national emergency and the day most major sporting leagues in the US announced indefinite closures, that Fox News began reporting the coronavirus as heavily as CNN. (MSNBC’s pattern of reporting is close to CNN’s.)

In other words, during the first and second “acts” in Bermejo’s model, CNN covered coronavirus more attentively than Fox News, perhaps reflecting Trump’s position that the virus was under control and shouldn’t be a primary focus for the US. In the third act, where the virus’s impact became unmistakable, Fox News began avidly covering the story.

Comparing word clouds in the “second act” of the story — February 12 — March 12 — shows a significant difference in tone between CNN and Fox News. Fox News distances the story from US shores, with common terms including “princess” and “cruise” (the Diamond Princess cruise ship, quarantined in Japan), “Iran” and “Korea”. CNN instead featured Anthony Fauci, a leading immunologist who’s recently become the target of death threats and received criticism from right-wing commentators over his support for social distancing at the expense of rapidly “restarting” the economy. CNN’s coverage in this act also included discussions of (social) “distancing” and possible “lockdown” as response strategies. By the third act, the distinctions between CNN and Fox’s coverage are not as clear, suggesting that as the crisis increases in intensity, the divergence in narrative may lessen.

UK media has a sharply different structure than US media. While US newspapers advertise neutrality, and US cable networks are more comfortable expressing partisan sentiment, the opposite is true in the UK. The BBC is prohibited from taking partisan stances, while newspapers are often unapologetically partisan. In considering UK media, we do not have the luxury of Benkler et al’s research on partisan media preferences to rely on — we build a much simpler study with three categories: newspapers that endorsed labor in 2019, newspapers that endorsed the conservatives in 2019 and the BBC.

It’s even harder here to find gaps between the left, right and center than in the US. The BBC’s pattern is spikier, in part because it publishes fewer stories per day than the aggregated newspapers. Close examination suggests that the BBC has an internationalist take, as compared to both Labor and Tory newspapers, and is more likely to mention Wuha, China, or Italy than the papers which have a more domestic focus. Even trying the trick of picking out extreme publications (The Guardian and The Sun) doesn’t show much differentiation between Labor, Tory or BBC views — the three seem to cover the story with equal intensity throughout. If anything, in contrast to in the US, the right-leaning Sun devotes slightly more coverage than the left-leaning Guardian to the story.

And what of the non-English speaking world?

In the case of Spain we looked at four different newspapers, which cover a wide ideological range and can be categorized as left (Publico), center-left (El Pais), center-right (El Mundo) and right (ABC). The evolution of the coverage in terms of percentage of attention is quite similar across all outlets, and corresponds to the pattern we’ve observed in other countries. The only noticeable difference across publications is that El Pais seems to increase attention to coronavirus less suddenly than the rest and remains slightly below in the intensity of coverage. (We are further researching to see if there’s a data quality issue, as we show El Pais publishing fewer stories in recent weeks. This could be due to an error in our system in counting those stories, though at first glance, it looks like they may simply be publishing less..)

Overall there seems to be no attempt at downplaying or exaggerating the relevance of the pandemic. Given the magnitude of the coronavirus crisis, most of the coverage focuses on the spread of the pandemic, the attempts at fighting it, and its impact on the economy. However, political issues are part of the media coverage, with right wing media more critical about the handling of the crisis and left wing media more supportive of the government measures. The political angle and its controversies seem to be gaining traction as the shock of the arrival of the virus gives way not only to public health measures but also to political decisions to control and alleviate its consequences.

In Italy, partisanship is a common feature of the media, but heavy political fragmentation makes US-style binary polarization just impossible. The sharply defined U.S. bipartisan system gives way to an increasingly fluid Italian multi-party political scenario. Currently, the government is supported by a coalition formed in parliament and composed of several small political entities, most of which fought fiercely against each other during the election campaign.

For this reason, going beyond the right-left dialectics, the study considers the main Italian media sources on the basis of their stated alignment with the current coalition government (Democratic Party — 5 Star Movement). Specifically, the graph includes four media sources considered close to the coalition supporting the government (Repubblica, Il Messaggero, La Stampa, and Il Fatto Quotidiano), plus four other sources traditionally closer to parties at the opposition (Il Giornale, La Verità, Il Resto del Carlino and TGCom24). These sources were chosen among those currently available on Media Cloud.**

Looking at the trend over time, the attention to the spread of the virus is very similar to that in other European nations.. Media attention basically followed the spread, with the main spike occurring earlier than in other countries. While the virus was spreading in Wuhan, most newspapers and TVs considered it as a potential, but distant danger. Some significant attention was only given when two positive Chinese tourists were hospitalized in Rome on January 30. Then, after about 2 weeks, the first case of an Italian person near Venice was reported, catalyzing national media attention. Since then, the virus has been dominating the news with incredibly high percentages of total attention.

In fact, while some individual politicians from different movements (both right and left-oriented) have tried to minimize the problem at the beginning of the spread, no party has substantially put the economy first when dealing with the virus. The political debate has been focusing more on the solutions to stop the spread than on the very existence of a serious threat coming from China. According to governmental statements in the press, every decision was based on hard evidence provided by a scientific committee and public health institutions.

Around the world, we see the same basic pattern of media attention to the novel coronavirus: a distant, lightly covered story; a situation that’s come to our shores; a crisis that consumes all our attention. We see the same basic structure of attention to the story across four countries — the US, the UK, Spain and Italy — but when we dig into the story in the US looking at opposing media sources, we see some divergence. Fox is slower to cover the story than CNN, and initially treats it as a problem occurring elsewhere, not as a problem coming home. As the situation emerged as a clear global pandemic, this disparity in coverage dissipated.

Spanish media suggests one possible future: as coronavirus becomes a fact of life, parties will position themselves in terms of advocating different strategies to combat the disease. It’s possible that another outcome, similar to what we’re seeing in Italy and the UK emerges, where a common crisis leads to a temporary deprioritization of politics.

We will be continuing to study coverage of COVID-19 around the world — if you have a question you want us to try to answer, or if you want to try one of these experiments in your country, let us know in the comments.

Methodological notes:

* for the present study we have used as research key-terms “coronavirus”, “COVID-19”, “Wuhan”, and “epidemia” as the main terms used by Italian media to refer to the epidemiologic emergency caused by the novel coronavirus “SARS-CoV-2”.

** The analysis has been totally carried out via Media Cloud (some sources have not been considered inasmuch as currently unavailable on the platform, e.g. “Corriere della Sera”).

*** The analysis does not pretend to be completely exhaustive.

Coverage of COVID-19 and Political Partisanship — comparing across nations was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Intercept has released an excellent story on the report and its key findings, but here’s an overview from our lab:

There has been a massive rise in immigration coverage in American media between 2014–2018. Along with that rise, we have found a significant rise in denigrating speech — including terms like “illegal aliens” — across media, from the far right to the left.

In four key news outlets, we found a significant rise in absolute use of denigrating speech, though denigrating speech actually shrinks in percentage terms — that’s because there is such a huge growth in immigration coverage overall. This absolute rise in denigrating speech is accompanied by a growth of the use of these terms in quotes — we see lots of evidence that news organizations are usually using these terms either in “scare quotes” or when they are quoting a public figure.

We also noted that there are three ways a news organization can deal with denigrating speech:

1. they can normalize it, adopting it as part of their official or unofficial style;
2. they can distance themselves from it, keeping it within quotes;
3. or they can gatekeep it, choosing not to showcase it, even within quotes.

We also saw a sharp increase in citation of three extreme anti-immigration think tanks. Using hand-coding, we found that these organizations are rarely cited with the context necessary to acknowledge that their views are extreme.

We hope that this report will encourage journalism organizations to take a close look at their coverage using tools like Media Cloud. If you’re interested in comparing your news organization to another, or in conducting the same searches we used to generate the report, visit this link. You’ll need to have a Media Cloud account set up, but with that you’ll be able to run queries to explore this type of speech in thousands of sources.

Beyond replicating this work, we hope that news organizations will take this report as a prompt to consider other aspects of their coverage that they should monitor, quantitatively, over time: How is your immigration coverage changing over time? Who are you citing and who’s not getting featured?

Media Cloud is both free to use, and free as in open source software, so you can conduct studies like this one yourself, whether you’re within a media organization or an external scholar trying to understand them. And if you ever have any questions about using our tools, feel free to reach out to us.

Lastly, we want to recognize the contributions of Emily Ndulue, who did the primary research and writing for this report, and Susan Benesch of the Dangerous Speech Project. Our report builds heavily on Susan’s work. Emily is currently on maternity leave with her new daughter, Chinyelu, and Susan is currently on maternity leave with her new daughter, Aya. This report also owes its existence to Jen Humke at the MacArthur Foundation, who brought the Media Cloud and Define American teams together at a critical summit on Civic Media.

Study: How major U.S. news sources cover immigration was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.

In an effort to make Explorer faster and more flexible for our users, we updated our Media Picker interface to allow for more versatility in source selection. Until now, users could use Media Picker to select specific source collections and add various sources to their query by searching for those sources individually.

Our updated version of Media Picker is more robust, allowing users the opportunity for more customization and creativity. Users can still search for and add sources by name, collections, favorites, or geography, but now they can also search for sources along certain parameters: publication country, language, publication state, media type, or “about place,” the country which is the subject of the media coverage.

Moreover, groups of different source types can now be saved as something we’re calling a “custom collection.” For instance, in the screengrab below I’ve created a custom collection for the first states to hold Democratic primaries in 2020: Iowa, New Hampshire, Nevada and South Carolina. The filter will surface all the media sources tagged in our system as being from any of those four states. (The same logic would also apply to a filter by country or language: if a source is tagged in our system with any of the five filter parameters you’ve selected, then it will appear in the search you conduct in Media Picker.)

After using the filters to find the four states I was interested in, I used the “Add Custom Collection” button in the bottom right of the above image to create my custom collection. I then clicked “OK” in the left-side panel to add the custom collection to my Explorer query.

I then went through and set up a comparative Explorer query so that I would be able to study the media coverage of the current debate-qualifying primary candidates in the four primary states selected above. That comparative query looks like this:

From there, I can use Explorer to study the primary candidates and their media coverage by analyzing attention, language and entities. If I want to save my search for further use later, I can click on the “Save Search” button to create a saved search for later; by saving my query in its entirety, I am also saving my custom collection for future use.

You can also set multiple filters at the same time, so that you’re querying against a more refined custom collection, or so that you’re including other collections in your query. In the example below, I’m still looking at primary candidates, but I’ve added a second filter for Spanish-language coverage:

We’re excited about the new, improved Media Picker and hope that it will be a helpful way for you to expand your usage of Media Cloud. As always, feel free to get in touch with us if you have any questions, comments or feedback. And for more information about using Explorer, you can also watch our introductory webinar.

Create custom collections in Explorer with Media Picker was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.

Research by Anushka Shah and Aashka Dave

In a 2016 report, the McKinsey Global Institute selected five developmental opportunities in India including higher living standards, sustainable urbanization, Indian-based manufacturing, harnessing technological growth, and empowering Indian women.

Since then, the Indian economy — currently the fastest growing economy in the world — has outpaced its peers. A recent report from the Organization for Economic Cooperation and Development (OECD) highlighted how India’s economic growth exceeds that of every other G20 economy.

This unparalleled growth, in a country with a population of over 1.3 billion people, underscores the developmental opportunities the country faces; such economic growth enhances the need for development of key wellbeing indicators and resources.

To better understand the media discourse surrounding development topics in India, we investigated how the mainstream media in India covers development topics across a number of sectors in the country.

With support from the Bill & Melinda Gates foundation, our team of researchers at the MIT Media Lab and Media Ecosystems Analysis Group set out to explore how development coverage takes shape. We used Media Cloud, an open-source media analysis platform, to investigate mainstream Indian media news coverage of 11 development topics over the 18-month period between January 1, 2017 and July 1, 2018.

Our Methods

The topics investigated were:

• State programs supported by the Indian government
• Access to digital financial services
• Women-led community organizations
• Women’s economic empowerment
• Social determinants of health and wellness
• Family planning, maternal and neonatal health, and children’s healthcare
• Vaccination
• Nutrition
• Water, sanitation and hygiene
• Diseases and conditions, including enteric and diarrheal diseases, HIV/AIDS, malaria, neglected infectious diseases, pneumonia, tuberculosis and polio
• Agricultural growth development

We created robust queries for the above topics and searched Media Cloud’s collection of national Indian news sources for each development topic. This collection is comprised of over 130 English-language media outlets, including major daily print publications such as the Times of India and Indian Express, digital-native news outlets such as The Wire and Scroll.in, and broadcast outlets such as NDTV and Times Now.

We sought to analyze attention, influence and language for each of these topics in order to better understand (1) the volume of coverage for development topics, (2) the influence that various publications carry in regards to development coverage, and (3) key frames or narratives for development topics.

Our Findings

Attention

Our research revealed a marked increase in coverage of development topics in Indian news media over the 18-month period. This was measured as percentage of stories dedicated to the topic over all the stories published by the media collection in that time period. Between January 2017 and July 2018, each individual development topic received increased media attention. In other words, the volume of news coverage rose for each development topic. This increase was measured in six-month increments using the total number of stories found for each topic in our source collection.

This finding demonstrates that development topics in India are receiving increased news attention. The development topics that attracted the most attention in total were water, sanitation and hygiene; government-supported state programs; and family planning, maternal and neonatal health, and children’s healthcare.

Influence

The three most influential Indian news sources were: (1) The Times of India, (2) Indian Express, and (3) The Better India. We reached this conclusion using a three-part methodology based on factors including the attention given to our selected development topics, digital audience size for each news outlet, and Facebook shares for selected topics.

To assess attention given to specific development topics, we used Media Cloud to determine the percentage of stories written about each development topic by each news source in our collection. Audience size was based on average monthly visits to that source using data from SimilarWeb. Facebook shares refers to the total number of Facebook shares received by a source for all stories on a given topic. We then aggregated the ranks across all three parameters to produce a list of the ten most influential media sources related to development coverage.

Framing and Narratives

We found that media coverage of development topics is driven by three factors: political attention, research studies, and lifestyle and wellness frames.*

Political attention was defined as quotes about politicians commenting on development topics, development initiatives led by the government, and legislation that affected the selected development topics.

Research study-based coverage was comprised of reports on new research findings relevant to development topics. These were often the driving factor for editorial and feature pieces across publications. More specifically, reports were often used to cover family planning, child and maternal healthcare, vaccination coverage and coverage of various diseases.

Lifestyle and wellness frames were most often found when publications covered topics related to nutrition and state programs. Based on other Media Cloud research our team has conducted, lifestyle and wellness frames also trend more frequently on social media than the other frames and narratives found during this project.

Conclusion

These findings demonstrate that a variety of development topics are receiving attention in the mainstream Indian media. In addition, they highlight potential avenues for growth of development coverage and provide valuable framing insight that could be used to inform further news coverage and the communications strategy of development initiatives.

As the Indian economy continues its growth trajectory and development in the country continues, we expect development reporting to grow and expand accordingly. This research helps inform media and communications strategy going forward and will serve as the basis for further research on development coverage in India. Feel free to reach out with questions and comments on this project!

*We selected a random sample of 100 stories for each development topic. We then categorized the stories into a general news type category which covered the types of coverage (ex: straight/hard news, feature story, editorial) and a more specific subcategory which flagged types of stories within those formats (ex: news stories, profiles, opinions).

Evaluating attention and influence for development topics in India was originally published in Media Cloud Project on Medium, where people are continuing the conversation by highlighting and responding to this story.