<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Deepnews.ai - Medium]]></title>
        <description><![CDATA[Deepnews.ai uses its algorithm to spotlight quality articles and talks to experts trying to bring people better news - Medium]]></description>
        <link>https://medium.com/deepnews-ai?source=rss----9d9c163d2dcb---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Deepnews.ai - Medium</title>
            <link>https://medium.com/deepnews-ai?source=rss----9d9c163d2dcb---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 04:53:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/deepnews-ai" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Will “data poisoning” be a particularly dangerous type of computer-made misinformation?]]></title>
            <link>https://medium.com/deepnews-ai/will-data-poisoning-be-a-particularly-dangerous-type-of-computer-made-misinformation-50a1fb7a7993?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/50a1fb7a7993</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[journalism]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Wed, 16 Jun 2021 13:05:19 GMT</pubDate>
            <atom:updated>2021-06-16T13:05:19.429Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Looking at the work of Priyanka Ranade, Dr. Anupam Joshi and Dr. Tim Finin of UMBC</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ERofSpjARdwArcqYCYfQLw.jpeg" /></figure><p>A lot of the time when we think about misinformation, we think of it as a sort of toxic substance that one can encounter when one is roaming around cyberspace. Some bad actor puts it out there for whatever reason, maybe supports it with bots, and it spreads out into the corners of the internet.</p><p>Our last post on a <a href="https://medium.com/deepnews-ai/can-gpt-3-make-misinformation-yup-it-sure-can-988bd1ab2f19">Georgetown study of algorithmically generated misinformation with GPT-3</a> looks at things this way, and how different combinations of human and machine are necessary to make convincing text that either a) looks like a garbage tweet you see on Twitter or b) looks a bit more quality and could potentially influence more people.</p><p>But when misinformation can get to a higher level of quality text, there is another issue. It can cause problems not just on the wide public squares of the internet, but also be specifically designed to infiltrate the sort of walled gardens of information that experts use to make decisions.</p><p>Recently I came across work done at University of Maryland Baltimore County, where the team of Priyanka Ranade, Dr. Anupam Joshi and Dr. Tim Finin looked at the ability of machines (in this case a fine-tuned version of GPT-2)<a href="https://theconversation.com/study-shows-ai-generated-fake-reports-fool-experts-160909"> to fool cybersecurity analysts</a>.</p><p>I am not an expert on cybersecurity, though the team explained that many analysts are getting information about potential security breaches and vulnerabilities, everything from the SolarWinds hack to less prominent ones, from systems that crawl the web for intelligence.</p><p>The idea was then to see if GPT-2, trained on a bunch of cybersecurity texts, could create fake cyber intelligence that could trick the people tasked with guarding our computer systems, the sort of trick that would then allow hackers to get a leg up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*t2meH-_aFeFHwgmOcL-86A.png" /><figcaption>Priyanka Ranade</figcaption></figure><p>“I used to work in a cyber security operations center, and the analysts do use a lot of the cyber threat intelligence that’s out there on a daily basis,” Ranade said.</p><p>“I was reading <a href="https://krebsonsecurity.com/">Krebs on Security</a> one day and I thought ‘How does he get all this information?’ He has to do research and probably has people working under him doing research, but what if they come across something that is misinformation that gets spread on these popular blogs. You have all these analysts reading it and they trust these sources.”</p><p>It turns out that current methods of generating fake news with a machine, even using a model that pre-dates recent advances like GPT-3, Google’s <a href="https://blog.google/technology/ai/lamda/">LaMDA</a> or <a href="https://www.politico.eu/article/meet-wu-dao-2-0-the-chinese-ai-model-making-the-west-sweat/">Wu Dao</a> in China, are <a href="https://www.researchgate.net/publication/350794232_Generating_Fake_Cyber_Threat_Intelligence_Using_Transformer-Based_Models">good enough to fool cybersecurity analysts who had worked in the field for years.</a> They were able to identify only around 21% of the fake reports as fake, worse than chance. The researchers said that this batch of correctly identified fake reports also happened to be the ones that had more “linguistic deficiencies” than the ones that analysts thought were true.</p><p>The results raise worries about how easily someone with advanced technological skills and resources could conduct “data poisoning” where the toxic information doesn’t just exist on the open web but seeps into the places they want to harm. They can deliberately put false information online that they know will get pulled up into systems that are not just looking for viral posts, but are designed to help people make important decisions.</p><p>“Right now if you look at the (cybersecurity) pipelines that are out there, they really aren’t engineered to find misinformation. All they do is look for this data … Let’s just collect, collect, collect. Put it in the pipeline, and there you go,” Ranade said.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*UAjGBDiKc9LwDq6fKiYKYQ.png" /><figcaption>Dr. Tim Finin</figcaption></figure><p>Whether the information is coming from a trustworthy source is always a good clue in terms of gauging whether something could be misinformation or not, and in our conversation Dr. Finin highlighted that work in fields such as cybersecurity relies on experts who are rational and likely to investigate further if something seems off. However, the work also “tends to be open source and it’s community driven.”</p><p>“So it is possible for false information to be injected from what looks like a plausible source —or some university could be corrupted — and then some process starts to generate data about attacks.”</p><p>This reminds me of the importance of getting more metrics about what is written online beyond just the easy ones like those centered on engagement, especially for any sort of field where they are trying to crawl or monitor the web to get news as quickly as possible. Limiting the sort of information you read to established sources that you already know has its downsides for understanding what is going on in the world, so for new ones it is important to get as much information as possible on what is being written, its characteristics and whether it is similar to misinformation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*cJdkJpw0oa4lBKIOQTOocQ.png" /><figcaption>Dr. Anupam Joshi</figcaption></figure><p>Dr. Joshi says that while fields like cybersecurity may be particularly vulnerable, he also worries about other areas such as medicine where preprint papers put up online can attract attention in times of crisis.</p><p>“We started training on a little bit of COVID-19 articles to produce abstracts. You know I’m not a medical doctor, but when I read it it sounds like a medical abstract, it has all the right terms,” he said.</p><p>One of Joshi’s upcoming plans is to see if computer generated misinformation such as a medical abstract can be detected by those trained in the medical field.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50a1fb7a7993" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/will-data-poisoning-be-a-particularly-dangerous-type-of-computer-made-misinformation-50a1fb7a7993">Will “data poisoning” be a particularly dangerous type of computer-made misinformation?</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can GPT-3 write misinformation? Yup, it sure can]]></title>
            <link>https://medium.com/deepnews-ai/can-gpt-3-make-misinformation-yup-it-sure-can-988bd1ab2f19?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/988bd1ab2f19</guid>
            <category><![CDATA[misinformation]]></category>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[gpt-3]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Thu, 03 Jun 2021 08:48:57 GMT</pubDate>
            <atom:updated>2021-06-03T19:30:17.133Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Looking at the “Truth, Lies and Automation” report from Georgetown University’s Center for Security and Emerging Technology</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*xX0svtnCO9CYORhNkT1AgQ.png" /></figure><p>It’s the worry that creeps in whenever people write about GPT-3: could this be used for bad?</p><p>We’ve covered the t<a href="https://medium.com/deepnews-ai/machines-arguing-with-humans-and-ibms-project-debater-94b27755d9a2">echnological advances in AI text generation</a> like those from OpenAI <a href="https://www.deepnews.ai/bot-answered-reddit-questions-at-one-per-minute-this-engineer-discovered-it-was-gpt-3/">a lot.</a> There are always the “oohs” and “aah” about what it can do (write a<a href="https://www.deepnews.ai/college-student-used-gpt-3-to-create-self-help-blog-but-what-about-quality/"> self-help blog</a>, for instance) and then others pointing out what it can’t.</p><p>But in the background is the question of how the ability to instantaneously “write” large amounts of text based on certain prompts could change the internet in unstoppable ways. Can artificial intelligence like GPT-3 be used for something like misinformation?</p><p>The answer, according to the report <a href="https://cset.georgetown.edu/publication/truth-lies-and-automation/">“Truth, Lies and Automation” </a>from the Center for Security and Emerging Technology at Georgetown University, is yes, it sure can be.</p><p>This week I spoke to Drew Lohn, one of the authors of the study that looks at how the AI system ranks on different types of misinformation tasks: <strong>“narrative reiteration”</strong> (repeating messages on Twitter, for example), <strong>“narrative elaboration</strong>” (writing an article with the desired message), <strong>“narrative manipulation”</strong> (rewriting an article to fit a certain slant), <strong>“narrative seeding” </strong>(creating new narratives that could become a part of a conspiracy like QAnon), <strong>“narrative wedging”</strong> (targeting members of specific groups to increase divisions in society) and “<strong>narrative persuasion”</strong> (changing people’s minds).</p><h3><strong>What GPT-3 can do well right now</strong></h3><p>As you can see in the results from the study’s table below, there are some places where advanced AI systems can already be used to push misinformation, like in “reiteration.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*annNTYynwUwOjfusAyshkA.png" /><figcaption>Courtesy of the Center for Security and Emerging Technology</figcaption></figure><p>In one test, Lohn and his fellow researchers input a set of tweets and then let the machine do the rest, creating its own set of tweets promoting the denial of climate change. Even though each message is small, this is one aspect what people worry about when they worry about the <a href="https://mondaynote.com/the-internet-is-not-ready-for-the-flood-of-ai-generated-text-a082976c6186">“flood of AI-generated text”:</a> that outside walled gardens of content that you need to pay for, the free version of the internet will be overrun with endless blurbs of text written by robots whose aims may be political, commercial or otherwise. It’s where automation can meet ideas like Steve Bannon’s “flood the zone with shit” or Chomsky’s flak.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*i7jJr8Ajj51QSuKQMYetxw.png" /><figcaption>Drew Lohn</figcaption></figure><p>Other tasks where the amount of text that the machine had to generate wasn’t very long also saw good results. It was able to create strange, nonsensical QAnon-type posts for “narrative seeding” and was able to create at least a limited amount of “narrative persuasion.”</p><p>One of the more alarming aptitudes is GPT-3’s ability to create divisive content for “narrative wedging.” It was aided by a human who helped engineer the prompts, but could then write to target different groups, including the use of racial slurs. More extreme content also appears to be where GPT-3 is most useful now. One of the overarching lessons mentioned in the Georgetown report is that “neutral headlines were often more varied and less consistent with one another, another reminder of the system’s probabilistic approach to ambiguity. Extremism, at least in the form of headlines, is a more effective way of controlling the machine.”</p><h3><strong>Where it needs help</strong></h3><p>GPT-3 shows more complicated results on other tasks where it was required to write a longer story, such as “narrative elaboration” and “narrative manipulation.” That is, at least when there are no humans involved.</p><p>For “elaboration” the machine was prompted with headlines from different outlets, The Epoch Times, Global Times and New York Times, and asked to replicate an article in the style of a given outlet. The model did not have an overwhelming accuracy at fitting the different styles, particularly for the NY Times at only 52%. Performance increased significantly when the researchers “fine-tuned” a new version (using GPT-2 rather than 3).</p><p>This to me raises questions about the ability of AI to generate text at higher levels of quality (such as in a NY Times article) instead of just generating some sort of text to fill a space. Another problem not addressed at length in the paper is the lack of logical flow that is a hallmark of GPT-3 output when it extends beyond a few paragraphs. The connection between the paragraphs it is writing can break down, which feels odd to a careful human reader.</p><p>Lohn is unsure how many misinformation actors would want to create longform pieces, or the exact number of those pieces they would really need for their aims. However, I think that one of the true risks of automated misinformation is not just the generation of a flood of crappy tweets, which a person with a certain degree of media literacy will ignore, but the generation of different sorts of content from tweets to fake “in-depth” articles that pass the smell test of a reader, sucking them into a desired worldview making it easier to feed them more misinformation. One way toward this goal would also be “narrative manipulation,” where GPT-3 was asked to summarize and rewrite an Associated Press story with different slants. However, to me at least, the output is not so much a repurposing of the information in the original story as a newly generated story (often vague) on a similar theme.</p><p>Lohn did not have an answer about the potential limitation of GPT-3 for generating quality news copy and cautioned that the distinction between the different outlets in this study was judged using CSET’s own classifier. In addition to “fine-tuning,” for more sophisticated misinformation operations, I imagine that the use of other metrics to measure the quality of an article (such as Deepnews) could create a sort of filter on the output of the model. OpenAI has already created such a system called <a href="https://openai.com/blog/clip/">CLIP</a> to filter through the answers that its image creator <a href="https://openai.com/blog/dall-e/">DALL-E</a> draws.</p><p>Another issue with GPT-3 that Lohn sees is the errors it makes when it may not fully understand the task ahead of it because of its training data. The data was compiled before the pandemic, for example, and so it does not have a firm grasp of “COVID.” Lohn also speculates that when the model is presented with a prompt that it does not immediately categorize into a “news” context, it may then create a text that is more fictional, including factually wrong information. Exactly how the categorizations happen remains unclear.</p><p>“I tried to get it to do super verifiable facts, science stuff, and it could. It could continue on with the three laws of thermodynamics, no problem. But when I tried to give it word problems to solve, for some of them it would do okay and for some of them it would frame the problem right and then get the answer wrong,” Lohn said.</p><p><strong>What happens now?</strong></p><p>The Georgetown study is one of the first in-depth looks we will have into how machines can start impacting the information we see and read, and I imagine that we will see other models be put through the paces of similar tests on the different “narrative” tasks.</p><p>But the other question about GPT-3 and misinformation is not just what is possible now, but who is going to be able to use these abilities going forward.</p><p>Lohn and his colleagues’ work focuses largely on the abilities of nation states, such as U.S. adversaries China and Russia, to use GPT-3 for misinformation. They would have the sort of resources to develop something, fine-tune it to certain needs, and hire people to put it into action for the tasks when it will need human intervention.</p><p>But the kind of technology that we are getting a glimpse of now is also going to become much more available. Microsoft bought exclusive access to GPT-3’s architecture and <a href="https://blogs.microsoft.com/ai/from-conversation-to-code-microsoft-introduces-its-first-product-features-powered-by-gpt-3/">recently unveiled its first features</a>. More commercial applications are on the way and Lohn believes that it will be difficult to create filters on exactly what sort of content someone creates once they have access to the technology.</p><p>He also cautions that there is likely to soon be open source knowledge of large language models available to those who want it. Training such a large language model is currently exceedingly difficult for most because it requires technologically sophisticated “parallelization splitting.” However, Chinese researchers are saying they will open up their tools for that splitting.</p><p>”So if you’ve got the model and the parallelization all open source, after a couple of months or years people make it easy to use, And then, anybody, well not quite anybody, but almost anybody could be able to play with it,” Lohn said.</p><p>The sort of inevitability of these tools casts a harsh spotlight on the infrastructure we have in place to share information right now and whether it needs changes to avoid helping create harm.</p><p>Text generation means that the bad guys won’t just write a few random messages and A-B test which ones get more play. They will create thousands of messages and test all of them until they are optimized for causing impact. It is up to us to use all the tools we can, from models that can spot AI-generated text to metrics for journalistic quality, to limit that impact. At the core is <a href="https://mondaynote.com/the-internet-is-not-ready-for-the-flood-of-ai-generated-text-a082976c6186">rethinking an internet where “relevance” is based on the idea of engagement and virality.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=988bd1ab2f19" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/can-gpt-3-make-misinformation-yup-it-sure-can-988bd1ab2f19">Can GPT-3 write misinformation? Yup, it sure can</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Outlets are trying to “publish less, publish better”]]></title>
            <link>https://medium.com/deepnews-ai/outlets-are-trying-to-publish-less-publish-better-e02f561bfca?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/e02f561bfca</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[subscription]]></category>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[local-news]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Wed, 26 May 2021 08:26:20 GMT</pubDate>
            <atom:updated>2021-05-26T08:26:20.567Z</atom:updated>
            <content:encoded><![CDATA[<p>Discussing what local news outlets in Europe are doing with Dr. Joy Jenkins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*I5juvGJCX2_s--O_CPt-zg.png" /></figure><p>This week’s post builds off of some of the ideas that I spoke about with Paul Cheung of the Knight Foundation <a href="https://medium.com/deepnews-ai/can-ai-help-local-news-figure-itself-out-201fef12d9b5">last week</a>, and focuses on how some local news has already been changing to survive, and potentially thrive.</p><p>Our talk, and honestly, a lot of the talk around local news, focused on what local news should be doing, and was hopeful about a future where it can leverage technology to offer something new and interesting for readers, advertisers, donators, etc.</p><p>But there is also the question about what is actually happening on the ground, beyond the doom and gloom that often dominates our understanding (and often is only focused on papers in the U.S). Around the world there are editors, reporters and managers trying to make the whole thing work as they give up a model focused on traffic and “pivot to paid.”</p><p>That is one of the findings of work from Dr. Joy Jenkins, a professor at the University of Tennessee who has looked at local news in Europe and published findings with the Reuters Institute at Oxford <a href="https://reutersinstitute.politics.ox.ac.uk/publish-less-publish-better-pivoting-paid-local-news">(Publish less, Publish better) </a>as well as in <a href="https://www.tandfonline.com/eprint/AAVDUHX5AWEQATC8PPJI/full?target=10.1080/17512786.2021.1913626">an academic paper.</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*4qAXj8I32cWfXW43S6Xdmw.jpeg" /><figcaption>Dr. Joy Jenkins</figcaption></figure><p>“We recognize that we need to give people not just what they want but when they need it we need to ‘publish less but publish better,’ which was a direct quote from a French journalist,” she told me recently.</p><p>“Although we can get a lot of traffic for certain types of stories that might not be the best option for us looking forward. Let’s look at analytics and let’s look at all this information we can garner in a really different way and put it in a different type of context than has been done before.”</p><p>That gets into using different types of measurements to find the stories that will turn casual readers into subscribers and members of a local newspaper’s community. Data analyzed by Deepnews has shown is often the in-depth articles<a href="https://www.deepnews.ai/our-scores-explained/"> (5 out of 5 on our scale)</a> that really explore the city or region where people live rather than smaller, snack-sized updates.</p><p>One of the big takeaways from Jenkins’s work is that readers don’t want to pay for something that’s not distinctive, that’s not original. The days of every local news outlet in a country doing the same article about a menu change at a fast food restaurant, which would generate clicks but didn’t really provide any added value, is perhaps over. While the number of journalists has been going down, I’d be interested in seeing data about the amount of real original reporting that is being done per journalist, now that resources are less often being used to aggregate stories with the hope that random clicks will rain down on your little patch of internet rather than someone else’s.</p><p>“What’s really getting [readers] to stay is this quality, public interest, in-depth, local journalism and this recognition of needing to offer something distinctive, something memorable and something that they can only get from these news organizations,” Jenkins said.</p><p>That has implications beyond just what stories are written, but where those stories go. Jenkins’s discussions with European newspapers actually are the second round of conversations she’s had with the same group, the first round starting in 2017. One of the things that is most striking to her coming back for a second time is how much the thinking has moved away from where it was only a few years ago.</p><p>“I think almost every newsroom I talked to, they had kind of scaled back what they were putting on Facebook and really trying to be more kind of thoughtful about who those Facebook readers are and whether those who actually want to become subscribers,” Jenkins said.</p><p>As I hinted at last week, I think that a refocusing on what local news already possesses in terms of data (and more information it can generate) means a new direction for making a sustainable place to do good journalism, through things like subscriptions, non-profit models as well as the advertising market and our bright beautiful cookieless future. The question now is whether now that outlets are moving away from the platforms as a technological crutch, they can grasp further opportunities.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e02f561bfca" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/outlets-are-trying-to-publish-less-publish-better-e02f561bfca">Outlets are trying to “publish less, publish better”</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can AI help local news figure itself out?]]></title>
            <link>https://medium.com/deepnews-ai/can-ai-help-local-news-figure-itself-out-201fef12d9b5?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/201fef12d9b5</guid>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[local-news]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Wed, 19 May 2021 11:55:25 GMT</pubDate>
            <atom:updated>2021-05-19T12:02:41.894Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Talking to Paul Cheung of the Knight Foundation about the “present and the potential” of AI for journalism.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*m5saG8Tg9oFrYU0aNWPPSQ.png" /><figcaption>A “Save Local News” rally, held around the U.S. last weekend. Photo from the Sentinel’s <a href="https://twitter.com/GabrielleRusson">Gabrielle Russon</a></figcaption></figure><p>The coming wave of artificial intelligence is often thought about in a threatening way, as if truckloads of robots will come to take jobs from grocery store clerks to journalists.</p><p>But journalism, with many outlets in dire straits before those real or imagined waves of machine replacements arrive, is also a place where AI can be used to turn things around, from pulling information out of documents to tools like Deepnews.</p><p>Outlets such as the New York Times already have their own machine learning teams, but some such as Paul Cheung, Director for Journalism and Technology Innovation at the Knight Foundation, are looking to make sure that advances can help those who are struggling in addition to those doing well.</p><p>“You know, from an editorial standpoint, no newsroom is getting tons and tons of investment and resources. It continues to be a challenge, and pressures,” Cheung, whose organization is <a href="https://knightfoundation.org/articles/ai-for-local-news-advancing-business-sustainability-in-newsrooms/">giving out $3 million in funds</a> around the issue, told me this week.</p><p>“I think if they are able to leverage technology properly, they will be able to do more with less.”</p><p>Accompanying Knight’s announcement is a report on the <a href="https://knightfoundation.org/articles/the-present-and-potential-of-ai-in-journalism/">“present and potential” </a>of AI for journalism. Among its findings is that of the AI projects being put to use in newsrooms, almost half were focused on helping reporters report their stories.</p><p>This makes sense considering that the projects surveyed were mainly from bigger organizations, though shows that there is a gap of using AI to address the most pressing problem for many outlets, which is not that they aren’t doing good journalism but that the business model around their <a href="https://www.deepnews.ai/deepnewsdigest/">excellent content </a>is no longer working. The issue has become part of mainstream discussion as local papers and journalists <a href="https://www.chicagotribune.com/business/ct-biz-tribune-rally-alden-shareholder-vote-20210516-oupxhdudvnf75i62zmk4sadhku-story.html">worry about owners that don’t want to invest in their work</a> but squeeze out any possible remaining profits while selling off pieces.</p><p>Initiatives focused around local news can help find solutions including tech to change the industry, though Cheung also warns of the “copycat syndrome” that can make editors glom onto one tactic as a cure for all their ailments.</p><p>“Just because the AP uses NLP to do an automated story, it doesn’t mean you should do that. Just because New York Times is using AI for commenting, it doesn’t mean that it would work for your community,” he said.</p><p>I am generally in agreement with Cheung, though for me, using AI well in newsrooms, and technology more generally, boils down to having and using the right data, and this is something that applies to all newsrooms big or small. Tools like Deepnews quality scores and other AI metrics could be useful in conjunction with the mountains of data that local newspapers already have about what people are reading, what leads to subscriptions, etc.</p><p>Especially as the internet approaches it bright, beautiful, <a href="https://www.google.com/search?q=cookieless+future&amp;newwindow=1&amp;safe=off&amp;sxsrf=ALeKk03QGwTs3U7rx4ESBmg3Il0x-FEq6w:1621422815748&amp;source=lnms&amp;sa=X&amp;ved=0ahUKEwjU-NnXztXwAhURPewKHRJWDCAQ_AUIDigA&amp;biw=1443&amp;bih=747&amp;dpr=1.6">cookieless future </a>online, there will be a refiguring out how to (and who, besides the platforms, can) make money from writing online. It’s a place where local news can also reinvigorate itself to become a bigger part of the conversation by offering subscriptions to people who want something different than news on Facebook or through search.</p><p>Personalization was one of the areas where Cheung pointed out how AI could help readers have a better experience with local news, with a newspaper site perhaps acting more like a Netflix queue or Spotify recommendations in that they can provide people stories based on more complex factors than just category, such as serving someone another business story if they are reading a business story.</p><p>Another trend is discussions of data and the ethics around it, which gets into questions about who has your data, what they are using it for, and who you are comfortable having your data (perhaps you’d prefer a local newspaper rather than a bigger company or a data broker).</p><p>“One of the hopes is that people will be a lot more educated, the general public will be a lot more educated about this technology,” Cheung said.</p><p>“And I think journalism to be sort of a leader in shaping that conversation and the ethical use of it.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=201fef12d9b5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/can-ai-help-local-news-figure-itself-out-201fef12d9b5">Can AI help local news figure itself out?</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The social media “prism” and how to build things differently]]></title>
            <link>https://medium.com/deepnews-ai/the-social-media-prism-and-how-to-build-things-differently-91f8f359887b?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/91f8f359887b</guid>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[social-science]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Wed, 05 May 2021 16:18:59 GMT</pubDate>
            <atom:updated>2021-05-05T16:26:24.972Z</atom:updated>
            <content:encoded><![CDATA[<p>I chatted with Prof. Chris Bail about his new book “Breaking the Social Media Prism: How to Make Our Platforms Less Polarizing.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*ulprq343QrtJN2xtgEPglQ.png" /></figure><p><strong>If you are interested in receiving more posts like this in your inbox each week, as well as our Friday Digest of stories that received high Deepnews scores, </strong><a href="https://www.deepnews.ai/deepnewsdigest/"><strong>please sign up here.</strong></a></p><p>Social media is made up of a lot of different parts, some readily seen on the facade, and others deeper in the architecture.</p><p>With billions of users, the data they generate, the algorithms that use that data and the ads that pay for it all, the platforms are mammoth constructions. But on closer inspection they are more commotion than exquisitely crafted cathedrals, with designs that aren’t exactly aiming all our eyes up towards beautiful rose windows.</p><p>Prof. Chris Bail, of Duke University’s Polarization Lab, is one of the people looking at the way that social media is constructed and who it can be better. He joined me and other guests on the Clubhouse show that Deepnews sometimes puts onto discuss his new book, “<a href="https://press.princeton.edu/books/hardcover/9780691203423/breaking-the-social-media-prism">Breaking the Social Media Prism: How to Make Our Platforms Less Polarizing</a>.”</p><p>The book itself is a good (and quick) read, where Bail moves from his previous research on polarization to how it plays out online, including experiments, such as one that looked at social media users who were shown posts from popular accounts on the opposite end of the political spectrum. While the popular imagination might believe that this would cause the users to moderate their views, the reverse happened and they became more polarized, some repulsed by the perceived extremity of what they were seeing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*lwXRYJbp3AXzEanmMLJyFg.jpeg" /><figcaption>Prof. Chris Bail</figcaption></figure><p>“We risk confusing extremists on the other side, with moderates on the other side. And that makes us all feel more polarized. And so this is why I introduced this idea of the social media prism to try to capture the way that social media distorts our offline lives in this in this online manner,” he said.</p><p>Part of this has to do with social media algorithms that prioritize engagement to sell more ads that can be shown to users, but Bail’s book also does a good job at exploring other aspects such as the way that users are driven to accrue “status,” which may be easiest with polarizing content.</p><p>These complexities are important to consider when looking at how to re-build social media, another subject of the book, either with some renovations at the giants or from the ground up in new ventures.</p><p>To revamp the idea of status, for example, the Polarization Lab worked on a “bipartisanship leaderboard” that measures social media success across the political aisle. They also put forward the benefits of anonymity, which Bail says “allows us to focus on the content of each other’s ideas instead of that ‘us vs them’ focus that we all have when we log on to talk about politics.”</p><p>He also mentions that, because of the business models of the biggest social media companies, that real change may come from smaller efforts aimed at changing the metrics for specific areas of the internet. It reminds me a lot of what Deepnews is doing with its quality scores, essentially adding new data that can isolate a different dimension of online discourse, the depth and quality of an article.</p><p>“Pinterest for hobbies. Instagram for photos. So why not another platform for politics?” Bail said.</p><p>“You wouldn’t get broad scale uptake. It wouldn’t be a competitor to Facebook. Instead, it would be for those people, probably about 5% to 6% of the population, who want to engage in productive debate and need to solve problems. Can you create a kind of status for them, where you’re rewarded for problem solving in the political space?</p><p>Getting more granular on specific social networks for specific uses also raises bigger questions about what the goal of a social network is, with the likes of Twitter and Facebook possibly being viewed as not having a very clear purpose. Even if you say that the purpose is “connectivity,” we have the technical capabilities to aim at a certain type of connectivity that doesn’t have some of the negative social and psychological effects of current social networks.</p><p>“Why not platforms with a purpose? Why not try to bake it into the design of our platforms, some principles that will guide behavior? Not just content, moderation, trying to plug the dam after its bust open in a million different ways. Asking those questions of what’s driving the water, where is the energy coming from? Those are the questions that I think we’re not thinking enough about,” Bail said.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=91f8f359887b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/the-social-media-prism-and-how-to-build-things-differently-91f8f359887b">The social media “prism” and how to build things differently</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Big AIs made with bad data]]></title>
            <link>https://medium.com/deepnews-ai/big-ais-made-with-bad-data-aec0f9f9f2ba?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/aec0f9f9f2ba</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Wed, 21 Apr 2021 08:42:24 GMT</pubDate>
            <atom:updated>2021-04-21T10:41:21.372Z</atom:updated>
            <content:encoded><![CDATA[<h3>Big AIs made with the help of bad data</h3><p>Looking at the work of MIT’s Curtis Northcutt on large amounts of errors in popular datasets</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*_CmBIwrHjSQOQMeuaGTwaA.png" /><figcaption>Labelerrors.com shows places where labels for AI test sets are wrong.</figcaption></figure><p>Those working in AI, including at Deepnews, <a href="https://mondaynote.com/deepnews-has-transformed-5746a0fba2e1">love to announce their brand new shiny models</a>, highlighting how innovative and efficient they are at accomplishing tasks and helping people.</p><p>However, the raw material that helps those models is data, and there is an increasing call by some in the community, such as entrepreneur Andrew Ng, to move from a <a href="https://www.youtube.com/watch?v=06-AZXmwHjo">“model-centric” view of developing artificial intelligence to something more “data-centric.”</a> This means that when researchers want to improve something, they look more at what they are putting in to their model, rather than the code that makes it up.</p><p>Recent years have seen closer looks at what exactly is being fed into machines that help them perform, with implications for everything from highlighting quality news to making suggestions for law enforcement that can have terrible consequences. While questions of bias because of a lack of comprehensive data on certain groups have gotten a lot of deserved attention, there is also the question of whether some data being used in the development of AI is just not accurate.</p><p>Last week I had the pleasure to speak with Curtis Northcutt, a researcher at MIT <a href="https://arxiv.org/pdf/2103.14749.pdf">who has led a look into data that are mislabelled in major test sets</a>. These test sets are used to measure how good a new model is. For example, ImageNet is tens of thousands of images that have been labelled with classes (a picture of a baseball being labelled a baseball) so that when someone creates a new AI aimed at “seeing” the creators have something to check against to make sure their model is doing a good job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*GL7S_da3uiGCuzCyy7xhhw.jpeg" /><figcaption>Curtis Northcutt</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/361/1*SZgKnCJAhUdQjv5mw03xvA.png" /></figure><p>What he found was that significant percentages, 5.8% in the case of ImageNet and higher for others, of the data in those test sets were mislabelled. The errors ran the gamut from misunderstanding the image completely, misidentifying the type of dog or failing to account for multiple labels, such as a bucket full of baseballs that was labelled “bucket.” If a model said that the image contained “baseballs” during a test then it would be called inaccurate, which impacts how useful we think it is. Northcutt likens the test sets to an answer key for a high school exam, through which a teacher determines who gets high scores and is then offered more opportunities.</p><p><strong>“</strong>If those tests are wrong then the benchmarks are wrong and we have no idea how machine learning is progressing,” he said (making an appearance on Deepnews’s occasional Friday Clubhouse show, which you’ll be alerted to if you<a href="https://mondaynote.com/deepnews-has-transformed-5746a0fba2e1"> follow my account)</a>.</p><p>It is of course difficult and expensive to have humans go through every bit of thousands of rows of data and make sure that they are all correct, though Northcutt has been part of a push to help find out where there may be mistakes in a more systematic way. Through a process called “confident learning” he and his team were able to find the labels that were likely to be errors by looking at the places where a model was also confident that the image was something else (“baseballs” instead of just “bucket.”)</p><p>Those errors were then checked by human workers, which is how Northcutt’s paper came to its eye-popping final number of mislabelled data. You can find some interesting examples at<a href="https://labelerrors.com/"> Labelerrors.com</a>.</p><p>The fact that there are so many mistakes is interesting in and of itself, but Northcutt’s team also went further and tested well-known AI models on the data both with and without the errors. What he found was that some less advanced, less complex models performed better on the corrected test sets, which of course means they would perform better in the “real world.”</p><p>“Simpler models, when there’s a lot of noise, don’t fit to that noise,” Northcutt said.</p><p>The problem appears to be that some of the more advanced models have been learning to perform better on the mislabelled data. In the education metaphor, this is the teacher “teaching to the test,” but the test itself is wrong. And these mislabelled tests are being used for AI models that get a lot of the attention. For example <a href="https://openai.com/blog/clip/">OpenAI’s CLIP</a>, which I<a href="https://www.deepnews.ai/artificial-intelligence-avocados-and-abstraction/"> have written about before</a>, tested itself on ImageNet.</p><p>So what do we do about all these errors? For his part, Northcutt, a strong supporter of keeping knowledge open source, has made cleaned versions of test sets available through <a href="https://github.com/cgnorthcutt/cleanlab">his GitHub page</a>.</p><p>Beyond that, there is probably something to Ng and other’s push to change mindsets to focus on data that can improve outcomes, rather than just announcing a model with nice top-line figures. At Deepnews, we developed a test set for our English language model based on various news categories (local news, national magazines, etc) by hand with human-labelled scores of 1, 2, 3, 4, or 5. We are doing the same thing now for French with similar categories. This makes sense because we are doing something new with AI and there is not any sort of publicly available test set we could use, but this process, though time consuming, also helps us be confident in where our model can deliver for clients and where it may need improvement.</p><p>Work like Northcutt’s is also increasingly important as we learn better how to create AI models that are effective and reliable. For those who want to dig deeper on the idea of “confident learning” and related topics, Northcutt also recommends <a href="https://cseweb.ucsd.edu/~elkan/posonly.pdf">this paper from Charles Elkan and Keith Noto at UC San Diego</a> as well as work done by <a href="https://link.springer.com/article/10.1007/s00778-019-00552-1">Snorkel</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aec0f9f9f2ba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/big-ais-made-with-bad-data-aec0f9f9f2ba">Big AIs made with bad data</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How do you teach someone to write in a world of robot authors?]]></title>
            <link>https://medium.com/deepnews-ai/how-do-you-teach-someone-to-write-in-a-world-of-robot-authors-a74238e519c1?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/a74238e519c1</guid>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[blogging]]></category>
            <category><![CDATA[writing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Wed, 14 Apr 2021 11:40:33 GMT</pubDate>
            <atom:updated>2021-04-14T11:40:33.784Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*FwGNpc-FF5SBS6fj1njTzA.png" /></figure><p>Writing, since the Mesopotamians carved cuneiform onto clay, has seen authors use the tools available to them to communicate and express themselves. While the image of a “writer” sometimes seems stuck in the amber of whisky and typewriters, those tools have progressed from quill pens to spell check to Google Docs.</p><p>But what does it mean when AI gets more involved in the actual creation of the words themselves? And how do we produce and highlight quality writing when we face <a href="https://mondaynote.com/the-internet-is-not-ready-for-the-flood-of-ai-generated-text-a082976c6186">an incoming tsunami wave of robot-generated text?</a></p><p>It’s a pet subject of mine and of Deepnews (which cares about quality content), which we’ve explored through the lens of <a href="https://www.deepnews.ai/bot-answered-reddit-questions-at-one-per-minute-this-engineer-discovered-it-was-gpt-3/">misinformation</a>, <a href="https://www.deepnews.ai/college-student-used-gpt-3-to-create-self-help-blog-but-what-about-quality/">marketing</a> and more. One important part of writing we haven’t covered, however, is learning to write, which is the focus of Dr. Lucinda McKnight, a lecturer in Pedagogy and Curriculum at Deakin University in Australia.</p><p>McKnight has written a couple pieces over at <a href="https://theconversation.com/profiles/lucinda-mcknight-324350/articles">The Conversation</a> about how we should teach students to write when in the future (and to some extent the present) there are machines capable of producing passable text. The rise of tools such as GPT-3 has led her from a focus on the effects of testing, which can push people towards bad, adjective-laden writing, to considering what students’ interactions with AI may be.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/290/1*YslhScIyBz7cT4chc-y7hA.jpeg" /><figcaption>Dr. Lucinda McKnight</figcaption></figure><p>“It was interesting to see how our high-stakes testing environment had narrowed the teaching of writing down to something very formulaic, that teachers were just trying to really tick boxes to get the students to comply with things like formulas for paragraphs, formulas for sentences, formulas for whole essays. They were doing very little writing that actually connected with the real world, that was anything like what a journalist might do or even someone maintaining their own blog,” she told me this week.</p><p>“One of the key things that I was thinking was that if school writing is becoming so formulaic, this formulaic kind of writing is exactly what machines can do so well and are already doing.”</p><p>So what kind of writing is the “human” kind? The elaborate and literary? Will the focus of teaching writing in the future be in creating purple prose so swollen with complexity that the sentences stick out like a sore, undeniably human, thumb?</p><p>McKnight doesn’t think so. She says that in a world of AI text, the skills that students need “ will be much more aligned with what we would describe as a ‘process writing’ approach, drafting, editing, collaborating, sharing and developing those kinds of soft skills that are going to be the skills that are useful in the future.”</p><p>“The emphasis, I think, is going to shift from the sort of humanist individual as writer to a more dispersed, networked notion of writing.”</p><p>Beyond the tools for generating text, in the context of education there is also the assessment of text, and whether machines can be used to help those on the teacher side. There are already<a href="https://www.turnitin.com/blog/artificial-intelligence-and-machine-learning-at-turnitin"> tools for plagiarism detection </a>that go beyond just measuring a students’ input against text online, and measure a students’ text against work that they have already submitted to see if there are abrupt changes in the patterns of writing that they have used previously.</p><p>Deepnews is of course interested in the evaluation of text, which we do through our quality score to judge its level of depth. Right now even the most advanced systems like GPT-3 have difficult generating high quality text in the Deepnews sense, though it is easy to imagine scores like ours needing to evolve to make sure that we are ahead of the curve of highlighting the added value that a human puts into a article (including the soft skills that McKnight talks about) as text generation advances.</p><p>Part of that work, in the world of technology as well as the world of education, is helping determine exactly where that human work occurs and how to make the most of it.</p><p>“Hopefully there will be a whole resurgence of creativity as something that humans can do that machines can’t do, or they can’t do yet. But I don’t think it will be a pure thing where there’s a binary division between humans and machines. I think that it will be something that is collaborative in the sense of working together,” McKnight said.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a74238e519c1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/how-do-you-teach-someone-to-write-in-a-world-of-robot-authors-a74238e519c1">How do you teach someone to write in a world of robot authors?</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Looking for a good time (on page)]]></title>
            <link>https://medium.com/deepnews-ai/looking-for-a-good-time-on-page-17be8610862f?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/17be8610862f</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[content-analytics]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Wed, 07 Apr 2021 09:19:07 GMT</pubDate>
            <atom:updated>2021-04-07T09:40:42.683Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*rytgGKv7CYTInK05JXBdQQ.png" /></figure><p><strong>This is one of the occasional posts we do about topics that interest us. If you are seeing this and you aren’t signed up to our email lists, you can receive blog posts like this, as well as our Friday Digest, </strong><a href="https://www.deepnews.ai/deepnewsdigest/"><strong>by signing up here</strong></a><strong>.</strong></p><p>Where does the time go? It’s a tough one to answer, though over the course of the last year, the response has been easier: time is spent online, our eyes flitting about on one page or another.</p><p>We normally don’t think about it that much , but on our screens our time is also getting quantified into seconds. That is then aggregated into the average “time on page” for whatever article, like this one, we and thousands of other people are reading. But what are those numbers actually good for?</p><p>This blog has in the past written a lot about page views, or clicks. That is mostly because I find that they are the easiest thing to compare against the Deepnews score. On large parts of the internet the idea of <a href="https://nickclegg.medium.com/you-and-the-algorithm-it-takes-two-to-tango-7722b19aa1c2">“relevance” is often mistakenly conflated with the idea of engagement</a>, whereas Deepnews is trying to do something different and provide a measurement to an article that is about the text itself.</p><p>But beyond clicks there are other measurements, such as time on page, that newsrooms and analytics companies use. Some have even stated explicitly that they are trying to optimize for time spent, which was the subject of <a href="https://www.tandfonline.com/doi/full/10.1080/21670811.2021.1890170">research from Jonathan Hendrickx </a>at Vrije Universiteit Brussel in Belgium.</p><p>Hendrickx spent a large amount of time with journalists from Mediahuis, a major newspaper publisher behind titles such as De Standaard and Het Nieuwsblad as well as more local papers. As he explains in his paper, a lack of time spent was particularly vexing for Mediahuis given that some of the advertising (and revenue) was sitting further down the page, unseen.</p><p>The piece goes on to explore that efforts by reporters to increase time on page purposefully (through things like adding videos) were not overwhelmingly successful. However, another aspect that Hendrickx looks at is how the journalists reacted, which varied from the more accepting local reporters to the more prestigious De Standaard where older veterans were much more reticent to change anything with their practices.</p><p>“I think important at any company is, if you want to change anything, to involve the people that you want to enact this change. Don’t go for a one size fits all solution,” Hendrickx told me.</p><p>The split between writers who are producing articles and their coworkers in audience development who want to measure those articles is something I have seen time after time. There is a certain unfortunate sentiment among many journalists, even younger ones, that numbers are distasteful and they would have majored in something like statistics during their undergrad years if they had wanted to deal with them.</p><p>Part of the reason why I think that Hendrickx’s work is interesting for Deepnews is that Mediahuis seemed to have been treating time on page not just as a business metric but also as a sort of editorial metric.</p><p>The Deepnews quality score is an editorial metric. It gives a quality score from 1–5 to a text which, if that same score was given by a human, would be given by an editor. If a newsroom were to choose to try to produce as many 5/5 Deepnews scores as possible, it would be a business decision but also a decision about their editorial priorities.</p><p>A combination of editorial metrics with more traditional engagement metrics (including things like subscription numbers) can also help different outlets better become the sort of outlets they want to become. Having the right sort of metrics as a guide can help a newsroom produce a unique sort of coverage for the unique community that supports it with subscription or donation dollars.</p><p>This may sound obvious, but it is a massive departure from the early and mid 2010s, when news outlets were all essentially chasing one metric, clicks. This led to a bunch of newsrooms all becoming interested in the same sort of stories, chucking them up online with little original reporting or added value and hoping that their article got more views than all the other, near identical versions.</p><p>That model has largely failed, and in some way, the path away from clicks may help bridge the gap between the editorial side and the analytics side of newspapers. It may head towards a place where the two sides work more closely together to talk about which of the many metrics they want to target as they differentiate themselves from everybody else. We may already be seeing this as some newsrooms look more closely at things like time on page vs. clicks.</p><p>Clicks can still be very useful for some outlets and for gauging immediate interest, Hendrickx says, though “Time-spent, then, is relevant as it enables knowledge on so-called tipping points where people stop consuming an article based on, for instance, average reading paces. This can then again inform journalists and editors on users’ behavior towards (quality) news content.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/0*CJ-tyk1_eYEkcvVb" /></figure><p>With the advances in machine learning where algorithms “read” the actual text, we can also easily get even more information about the articles that journalists are creating and use them together with other metrics to make something like time on page more useful . For example, the graphic above shows how Deepnews scores predict higher time on page in our study of 17,000 articles in the US. But the possibilities for AI to understand text extend far beyond just quality. Researchers we have spoken to for this blog have mentioned possible metrics such as sentiment, political bias, logic of a text or how deliberative they are.</p><p>This sort of combination will help us not only find out what sort of things we can do to increase time on page, as in the case of Mediahuis, but also give readers the sort of coverage they sign up for instead of just what makes them engage. In that case, everyone will be having a better time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17be8610862f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/looking-for-a-good-time-on-page-17be8610862f">Looking for a good time (on page)</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machines Arguing with Humans and IBM’s Project Debater]]></title>
            <link>https://medium.com/deepnews-ai/machines-arguing-with-humans-and-ibms-project-debater-94b27755d9a2?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/94b27755d9a2</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[journalism]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[debate]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Wed, 31 Mar 2021 11:29:08 GMT</pubDate>
            <atom:updated>2021-03-31T15:21:20.188Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*VmnteNODLpy318tPOsPFiA.png" /><figcaption>Project Debater and debate champion Harish Natarajan.</figcaption></figure><p><strong><em>Editor’s note: This is one of the occasional posts we do speaking to someone with something to say on topics that we find interesting. If you are seeing this and are not yet signed up to Deepnews, </em></strong><a href="https://www.deepnews.ai/deepnewsdigest/"><strong><em>sign up here</em></strong></a><strong><em> to start receiving our blog posts every week, and a Digest of quality news on an important subject every Friday.</em></strong></p><p>At Deepnews we love a good argument. Part of looking for quality writing and content is looking for quality arguments, implicit or explicit, made with solid supporting evidence.</p><p>Our model of course uses machine learning to search through thousands of sources to find the articles that we highlight, though others are looking to use the advances in technology to not find but *create* excellent debate.</p><p>The team at IBM is doing exactly that, and a couple weeks ago published a paper in <a href="https://www.nature.com/articles/s41586-021-03215-w#MOESM1">Nature</a> that presents the results of their Project Debater, which created a robot that was capable of debating an expert human in a formal debate setting. You may have <a href="https://www.youtube.com/watch?v=-d4Uj9ViP9o">gotten a glimpse of the work</a> led by Noam Slonim back in 2019 when it debuted, though the recent paper explains its system architecture and shows that the machine’s statements were more highly rated than those written by non-expert humans (but less highly rated than expert debaters).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*MHrNNNA0bRvdBJTpeLt4HA.png" /><figcaption>Graphic taken from the Nature article on Project Debater</figcaption></figure><p>It is all very impressive, but what exactly is Project Debater doing that is new? I spoke to Prof. Chris Reed, the director of the <a href="https://www.arg.tech/">Centre for Argument Technology </a>at University of Dundee, who wrote <a href="https://www.nature.com/articles/d41586-021-00539-5">a commentary piece in Nature alongside the IBM paper</a> and is an expert in the field of argument mining.</p><p>“I think that the thing that’s actually rather underplayed, even by them, is the feat of engineering o getting all the pieces to work together,” Reed said.</p><p>“Being able to go from the audio of listening to somebody give an opening speech. And then trawling through 400 million news articles to identify snippets of text that are appropriately relevant. And then editing and splicing effectively those bits of text together to render grammatically correct sentences. And then organizing those grammatically correct sentences into something that bears some kind of resemblance to a coherent narrative flow. That is astonishingly difficult.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*jPV_1imeUg8YhDvAF14tvw.png" /><figcaption>Prof. Chris Reed</figcaption></figure><p>Part of the core of this work is argument mining, where the machine finds relevant arguments and counter arguments from within its massive corpus of text. In a textbook an example argument to be found would be something like “Socrates is man. All men are mortal. Therefore Socrates is mortal.”</p><p>“Except that’s not what arguments look like. They look like: ‘Let’s go and get a beer now,’ ‘No, I’m a bit tired this evening.’ ‘Can we do it tomorrow?’ ‘Okay, let’s do it tomorrow because they’re going to have a special,’” Reed says.</p><p>So what does it mean that Project Debater can parse through text for arguments and then use them? What does being able to debate, a classic example of reasoning in action, show? On this blog we previously talked about reason when discussing t<a href="https://medium.com/deepnews-ai/language-reason-and-building-a-machine-that-can-spot-fallacies-d012870158f0">he work of Dr. Henry Shevlin</a>, who has worked on comparing machines to intelligences from animals to humans.</p><p>Reed says that Project Debater is still nowhere even close to human cognition. One example of this is evidenced in Project Debater’s “rebuttal” to the arguments put forward by its debate opponent. It takes the text spoken by the opponent, translates it into text, and then compares that text to arguments it has pulled from its corpus, the knowledge base it has been fed, and a specific database full of debate topics.</p><p>“It’s absolutely not creating a structure of the opponent’s argument, and then looking at that structure and reasoning about where the weak points are in it to formulate an argument. There is none of that,” Reed says.</p><p>In the same way that we need to avoid thinking about GPT-3 as intelligent, it is easier to understand what machines like Project Debater are doing when we focus on the work that they are doing, thinking back to the origin of the word “robot” as coming from words for labor. GPT-3 is able to generate passable text and Project Debater is able to accomplish the impressive task of debating by bringing together different systems. Both tasks involved language, which is how intelligence is often expressed and where AI has seen many recent advances.</p><p>By focusing on the broader task of parsing through language, Project Debater is doing something different from its IBM ancestor Deep Blue, which played Garry Kasparov in chess with the narrow goal of reaching checkmate. Slonim’s team points out that debate is harder to judge, with more subjectivity, and that pushing a machine to use more advanced forms of human language, even if just pulling from material it has been fed, pushes artificial intelligence outside of its “comfort zone.”</p><p>The question then is not about intelligence but about how machines working with more advanced human language can be useful, particularly as humans are confronted <a href="https://mondaynote.com/the-internet-is-not-ready-for-the-flood-of-ai-generated-text-a082976c6186">with the flood of text on the internet</a>.</p><p>One of those ways that a focus on argument may be useful in that we as humans can start understanding what is inside of text, rather than just the number of clicks it gets, much faster and on a much larger scale. That’s something that Deepnews is working on in terms of our quality metric, and our technology shows the benefit of letting a machine help categorize content.</p><p>Being able to engage with individual arguments within an article at scale would be another step forward. Reed’s team a couple years ago <a href="https://www.bbc.co.uk/taster/pilots/evidence-toolkit-moral-maze">teamed up with the BBC to look at the challenge of fake news</a> and use argument mining to help aid human students’ reasoning. If applied on a broad scale, this can help increase the quality of debate.</p><p>“What we’re starting to think about is ways in which AI systems are able to understand the arguments of humans and contribute to the debates as a whole. Teams can be expanded to have some human members, and some AI members, and the team as a whole can then produce better quality decisions,” Reed said.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=94b27755d9a2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/machines-arguing-with-humans-and-ibms-project-debater-94b27755d9a2">Machines Arguing with Humans and IBM’s Project Debater</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What do I have to say to get you to subscribe?]]></title>
            <link>https://medium.com/deepnews-ai/what-do-i-have-to-say-to-get-you-to-subscribe-fe802785f22?source=rss----9d9c163d2dcb---4</link>
            <guid isPermaLink="false">https://medium.com/p/fe802785f22</guid>
            <category><![CDATA[local-news]]></category>
            <category><![CDATA[subscription]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[journalism]]></category>
            <dc:creator><![CDATA[Christopher Brennan]]></dc:creator>
            <pubDate>Wed, 17 Mar 2021 10:45:41 GMT</pubDate>
            <atom:updated>2021-03-17T10:45:41.386Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Speaking to Yujin Kim of UT Austin about her work on subscriptions</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*UeT2tNt9qTDdzS6kTGWwiA.png" /></figure><p><strong><em>Editor’s note: This is one of the occasional posts we do speaking to someone with something to say on topics that we find interesting. If you are seeing this and are not yet signed up to Deepnews, </em></strong><a href="https://www.deepnews.ai/deepnewsdigest/"><strong><em>sign up here</em></strong></a><strong><em> to start receiving our blog posts every week, and a Digest of quality news on an important subject every Friday.</em></strong></p><p>Over the last several years, the decline of smaller newspapers has firmly established itself as an unstoppable narrative, with the headlines that get the most attention from reporters on social media almost always including words like “layoffs” or “closure.”</p><p>Beyond the doom and gloom, some of those articles move from the problem to the solution, and mention the possibilities to save local news going forward. There is the non-profit model on one hand or, what is increasingly becoming the go-to option for papers big and small, subscriptions.</p><p>Putting up a paywall is not that difficult, but local papers also need people to actually input their credit card numbers and become part of their communities. So the question remains: why does someone do that? Last week I had a chat with Yujin Kim, a researcher at University of Texas at Austin’s Center for Media Engagement, who set out to clarify one angle, the messages that newspapers use to push people to subscribe.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*pu0IUwrNCEFmxvNO2vUECw.jpeg" /><figcaption>Yujin Kim</figcaption></figure><p>In <a href="https://www.tandfonline.com/doi/full/10.1080/21670811.2021.1873812">her recent paper</a>, Kim worked with local newspapers in the U.S. to test which messages were more effective on different channels like social media or email: a) those that framed subscriptions in terms of a gain to readers or b) those that framed things as a loss to readers if they didn’t. Examples she cites are the NY Times gain-framed slogan “Truth. It needs your support” versus the Washington Post’s loss-framed “Democracy Dies in the Darkness.”</p><p>Kim worked with the local papers to A/B test different messages and see how they performed on different channels.</p><p>“One significant finding was on the Facebook versus email models. Basically, the findings are that the gain frame works better on email than on Facebook,” she said.</p><p>This perhaps tells us a bit <a href="https://www.deepnews.ai/politics-on-facebook-and-local-information-infrastructure/?_thumbnail_id=7983">about the way that Facebook is used for news</a> and the sort of messages that get more attention there, but also hints at why the people who do sign up for newspapers may hit subscribe.</p><p>While there was limitations to the significance of the results in other tests, Kim also said that in none of the tests did the messages about losing out outperform the messages about gaining from the value of news and being informed.</p><p>I was interested in Kim’s study in terms of Deepnews because the data we have on how our higher quality scores (4s and 5s out of 5) predict subscriptions suggest something similar. In our analysis of 17,000 local news articles, the percentage of articles with a 5 that led to at least one person subscribing immediately after reading was a staggering 35%, far higher than for articles with lesser scores. Articles highlighted by our higher scores also had a much higher likelihood of converting a reader within 30 days, as readers go through multiple pieces of all different types that the newspaper offers before deciding to commit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q6SyiBAUCVZgIt-L80GpJQ.png" /></figure><p>The move to subscriptions is happening at the same time that local newspapers have realized that they need to use data better, though for too many outlets the focus on metrics raises fears that clicks will be prioritized. Reporters have spoken up against being pushed towards writing about <a href="https://www.docdroid.com/n8pxTOD/sac-bee-union-letter-re-performance-metrics-pdf">menu changes at Taco Bell </a>rather than covering their communities. At the same time as there has been a focus on clickbait in some corners, I think that the <a href="https://www.deepnews.ai/deepnewsdigest/">Deepnews Digest</a>, a different sort of metric, shows that there are local newspapers, with subscription-only, non-profit and other models, who are doing important, in-depth work that scores 5 or 4 in our model.</p><p>Kim’s study also mentions previous literature on people who subscribe to a newspaper to promote their membership in some sort of sub-group, potentially the sort of personality that wants to be part of the bigger conversations that are made possible by well done journalism. I think that metrics like the quality score can be part of changing the narrative around local journalism from one of decline and fear of collapse to one that says “you are the kind of person that is interested in excellent stories, here are ours, come and be part of the conversation.”</p><p>If what this subscriber sub-group wants is depth, then it seems like the path to subscriptions, and with it profitability, may lie with providing higher quality content rather than hoping that your outlet gets lucky and has page views rain down on your fluff article rather than everyone else’s fluff article.. The next step is being able to find and use that excellent journalism as efficiently as possible, and accompany it with the right sort of message.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe802785f22" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepnews-ai/what-do-i-have-to-say-to-get-you-to-subscribe-fe802785f22">What do I have to say to get you to subscribe?</a> was originally published in <a href="https://medium.com/deepnews-ai">Deepnews.ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>