<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yves Bergquist on Medium]]></title>
        <description><![CDATA[Stories by Yves Bergquist on Medium]]></description>
        <link>https://medium.com/@ybergquist?source=rss-1600f8fa2199------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*BtbIRI4p6lkQXxg4quh7TQ.jpeg</url>
            <title>Stories by Yves Bergquist on Medium</title>
            <link>https://medium.com/@ybergquist?source=rss-1600f8fa2199------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 12:58:19 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ybergquist/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How Big Media Companies Can Survive in an AI-Driven World]]></title>
            <link>https://medium.com/@ybergquist/how-big-media-companies-can-survive-in-an-ai-driven-world-b13a84dbe248?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/b13a84dbe248</guid>
            <category><![CDATA[hollywood]]></category>
            <category><![CDATA[medium]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Tue, 30 May 2023 18:01:03 GMT</pubDate>
            <atom:updated>2023-05-30T18:01:03.670Z</atom:updated>
            <content:encoded><![CDATA[<h3>How Big Media Companies Can Thrive in an AI-Driven World</h3><p>This post is a review by NAB Amplify’s Adrian Pennington of my latest AI White Paper looking at how AI affects the Hollywood studios’ competitive position in the media market.</p><p>TLDR:</p><ul><li><em>A new report from studio-funded thinktank ETC, “AI &amp; Competitive Advantage in Media,” authored by the org’s resident data scientist and AI expert, Yves Bergquist, explores the impact of generative AI on the media.</em></li><li><em>Bergquist argues that generative AI impacts the media industry less by disrupting its traditional content creation process and more by giving social content creators the tools to make large amounts of truly cinematic, studio-like content.</em></li><li><em>Social media’s strength is its enormous library of content, which allows it to conduct millions of content market-fit experiments per minute.</em></li><li><em>There’s still a place for deliberate, curated, data-driven development decisions about what audiences want to see, but studio-style content needs to be integrated and immersive.</em></li></ul><p>The single biggest impact of generative AI for large content producers and distributors isn’t about disrupting the media-making process. It’s that it gives its fiercest competitors — content creators on YouTube and TikTok — more tools to eat further into everyone’s daily video consumption that the media industry is battling for.</p><p>According to a fresh report by <a href="https://www.etcenter.org/">studio-funded thinktank ETC</a>, “AI and Competitive Advantage in Media,” generative AI “potentially disrupts the already unfortunate economics of the media business: stable demand (never more than 24 hours in a day) and exploding supply.”</p><p>In the report, <a href="https://amplify.nabshow.com/?s=Yves+Bergquist">Yves Bergquist</a>, ETC’s resident data scientist and AI expert, argues that what’s happening in the media industry is proximate to what already happened in manufacturing: automation of the craft of making a product (i.e., making the product computable).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TGoj3u4SEsDwAX5zv23FnQ.png" /><figcaption>Generative AI is the latest chapter in a long-standing trend to make all of society and business computable, starting with knowledge (which the Internet made computable), social relations (computable via social media)</figcaption></figure><p>By computable they mean that content is produced in volume and is “machine readable” in terms of every aspect of its creation to distribution to feedback from audiences being data and therefore available for dissection</p><p>Traditional media companies currently are not “computable” in the sense that they produce products linearly, one at a time. It is scarce, whole, long-form (not conducive to being sliced and diced by an online audience) and unstructured (its narrative DNA is not yet machine-readable).</p><p>This is going to have to change if studios and streamers want to part of the bigger picture in a few years’ time.</p><p>ETC divides the creative process into three parts. Bergquist dubs the ideation part, where creatives “sense” what an audience wants to see, “zeitgeist intelligence.”</p><p>Then there’s the core of the creative process, where creatives define their voices and make strategic decisions about what product will be crafted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KV2z9P9pBNXLJ07Abaspzw.png" /><figcaption>Generative AI automates the craft of media-making, thus putting more emphasis on the two other aspects of creative media-making: Zeitgeist intelligence and creative decision-making</figcaption></figure><p>Finally, the product is made.</p><p>AI’s immediate impact is on that final phase. But by automating production, “Generative AI not only puts more emphasis on Zeitgeist-sensing and creative decision-making, it gives creative decision-makers tools to quickly and cheaply tinker, experiment, and prototype.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u1SHS_weymZyuQAIiEzgeg.png" /><figcaption>Does knowing the craft of media-making make one a better creative decision-maker?</figcaption></figure><p>At the same time, traditional media companies “risk losing their monopoly on the craft of high-quality content.”</p><p>Generative AI empowers social creatives to quickly and cheaply craft “studio quality” content threatening the status of traditional media. They can do this because their knowledge of what the audience wants is crowdsources by links, likes and recommendation algorithms. The content produced is computable in the sense that it can all be digitally mined. And the scale of content production means there’s enough supply to fit cater for every audience whim.</p><p>But ETC spots a weakness. Social media platforms and content creators reliant on those platforms lack any real understanding of their audience, claims ETC. It is just “basic content match-making”.</p><p>Instead, studios and especially streamers, can strike back against pure AI content generators by using the data they have at their disposal more intelligently.</p><p>“Programmatic content distributors like TikTok match content with audiences without any semantic understanding of why this content resonates. It’s just a programmatic marketplace that computes the content de facto.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F0xjYwLGyXU05cfii0ErAQ.png" /><figcaption>Generative AI’s biggest impact on traditional media players is that it intensifies the competition for eyeballs with the Creator Economy</figcaption></figure><p>With generative AI bringing high production value tools to social creators, we can expect a new category of “short-form linear content” to emerge on social platforms.</p><p>Studios, on the other hand, “have the longest experience and the largest dataset available to not only develop an intelligence go their audiences, but to draw them into a deep relationship with their franchises.”</p><p>Media organizations, “especially those with a streaming service,” have both the data and a unique capability to understand the cultural zeitgeist. They can use AI to better “know” what audiences want, Bergquist says.</p><p>ETC also suggests that it’s the large media organizations that have the financial backbone “to create highly integrated and replicable AI-driven virtual production workflows.”</p><p>It contends that traditional media players will need to differentiate through immersive, multi-platform, world-building franchises, a trend they are already pursuing of course.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DpMrCjD1cu-Tj5UYrjtswQ.png" /><figcaption>In a world where “near-Hollywood” content creation tools are int he hands of virtually everybody, large traditional media companies can compete by strengthening a data-driven “relationship” with their audience via deeply immersive multi-platform content creation.</figcaption></figure><p>This, says ETC, “is the greatest opportunity for large media organizations to leverage virtual production and generative AI together to quicken and cheapen the cost of producing these multi-format immersive pieces. This new form of computable content will run on game engines.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bvt_uTbH4hAlPxRi02dDTg.png" /><figcaption>Hollywood’s biggest asset is its relationship with audiences</figcaption></figure><p>In so doing, this “revolutionizes the way stories are told,” with integrated narratives spun across linear and immersive media products.</p><p>There are warnings, though.</p><p>“Media organizations don’t have a software culture, nor can they support large AI R&amp;D assets. They could partner with (or acquire) key AI research organizations to leverage their data to create their own proprietary content and audience intelligence models, but this is a heavy lift.”</p><p>ETC also identifies a need for intuitive “human-ready” and “business-ready” interfaces for AI models, which continues to be the greatest bottleneck for AI in enterprise. Too often, says Bergquist, organizations can’t connect models and business needs.</p><p>“Whoever can redesign their organizations and workforce needs to best create a ‘culture’ of AI and data will move faster than its competitors.”</p><p>Education, insists ETC, is the largest opportunity in AI today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u-AcaujGPY6FqtBrFb9cHQ.png" /><figcaption>The 5 places where organizations can develop competitive advantage in AI today</figcaption></figure><p>While everyone seems to agree AI represents a big financial opportunity to automate some production and postproduction workflow it begs a question: Does taking knowledge of the craft out of creative work affect creative decisions and creative output overall? Or, put another way, does knowing the craft make a creative a better decision-maker? ETC has no answers for this, and perhaps we’ll only find out in time.</p><p>More globally, what the media industry needs right now is a distinct and actionable AI vision.</p><p>Written by Adrian Pennington and published through NAB Amplify.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b13a84dbe248" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Generative AI ‘s “War of the Worlds” Moment]]></title>
            <link>https://medium.com/@ybergquist/generative-ai-s-war-of-the-worlds-moment-ab30bfd0ffae?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab30bfd0ffae</guid>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[hollywood]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Wed, 11 Jan 2023 02:09:54 GMT</pubDate>
            <atom:updated>2023-01-12T19:52:58.882Z</atom:updated>
            <content:encoded><![CDATA[<p>ChatGPT came too late (end of November) to make a significant impact on CES this year, but the cacophony of opinions about the generative AI model definitely made its way to Vegas. The timing was perfect. Just as the crypto crash left the hype industry paralyzed, OpenAI launched ChatGPT in what now feels like a nerdy and frustrating tech version of the Rolling Stones’ Altamont concert in 69 (with computer scientists as the Hells Angels).</p><p>Make no mistake: this is a landmark achievement in machine learning. Perhaps the single greatest since the 2006 paper by Hinton, Salakhutdinov, Osindero and Teh on backpropagation in deep neural networks.</p><p><strong>The LD on GPT</strong></p><p>ChatGPT is a version of OpenAI’s latest Large Language Model (LLM), GPT-3.5, optimized for dialogue with humans. As far as OpenAI has been willing to share publicly (quite a bit more than previously, actually) ChatGPT uses similar training data as GPT-3 (45TB of text), and has the same initial number of parameters (175 Billion). That’s around 3x less than its 2 closest competitors, Turing NLG (Microsoft, 530 Billion) and PaLM (Google, 540 Billion), and 100x more than GPT-2.</p><p>Like all large language models, ChatGPT does one critical thing very well: using its gigantic training set to guess the probability that one word, or part of word (a token in NLP), will come after another. This is a traditional natural language processing method called an N-Gram model, but on T-Rex steroids. What makes ChatGPT unique — and powerful- is the amount of manual (human) fine-tuning that this probabilistic model went through. OpenAI used a method called Reinforcement Learning through Human Feedback. In simplistic terms, RLHF is a method where human analysts prompt GPT-3 and manually “reward” (or not) the response based on how “aligned” it is with the human intent. Like teaching a puppy how to play soccer with a pocket full of treats. Another human team then ranks the rewarded responses from most to least useful, creating a new class of labels, which is then used to re-train the model. Rinse repeat. The human in the loop’s ranking greatly improves the model’s “alignment”, which in machine learning means that it’s both accurate <em>and</em> relevant to the prompt, an area where GPT-3 was seriously lacking. It also reduces the size of the model (1.3 Billion parameters for the RLHF-powered InstructGPT, close precursor of ChatGPT). Finally, this manual scoring does a much better job at weeding out toxicity in the output text.</p><p>The result is impressive. ChatGPT can produce coherent and often insightful text in many very specific styles, and perform many natural language tasks at human level. When it’s good (often) the machine-output text is indistinguishable from human output, which is probably why the education industry is (justifiably) freaking out. When it’s bad (also often), it betrays inescapable yet fundamental flaws in OpenAI’s methodology: it’s hyperscale parroting, not intelligence per se. But hey, it works.</p><p>Just like its parent, GPT-3.5, ChatGPT is a mix of different sub-models, each capable of handling specific tasks and types of human knowledge. This approach greatly improves the quality of the outputted text for these use cases in these domains. But it also means that it leaves out many other domains, tasks and types of knowledge, where the model hasn’t been trained. The lack of proper reasoning modules means that ChatGPT fails at simple arithmetic or logic, for example (“10 pounds of feathers are lighter than 10 pounds of steel”). It often answers questions in ways that are grammatically correct (high accuracy) but don’t make any sense in the real world (low alignment). See Gary Marcus’ blog post for examples: <a href="https://garymarcus.substack.com/p/how-come-gpt-can-seem-so-brilliant">https://garymarcus.substack.com/p/how-come-gpt-can-seem-so-brilliant</a>. This is because the model has been fine-tuned on tens of thousands of human contexts, but far from all. So when a prompter hits the model with something it’s been trained to “understand”, it can <em>pastiche</em> at a level unseen before. But when in unknown territory, it fails spectacularly. More importantly, ChatGPT is still an extension of the same broad methodology, deep learning, that covers only one aspect of intelligence: learning on training data.</p><p><strong>“Everybody Please Calm Down about ChatGPT” — Vice</strong></p><p>So yes, it’s been fun seeing the techno-optimists and techno-pessimists agree on <em>something</em>. But behind the attention-grabbing hyperbole about ChatGPT is a fundamental reality of the tech industry: keep your eyes on the builders, not the commentators. There’s no tech without products. And there are many less high-tech products on our tables than in our podcasts. In many ways, the past 6 weeks of the tech press have felt like an extended version of Orson Welles’ 1938 “War of the Worlds” broadcast, minus the street mobs. No, ChatGPT is not Artificial General Intelligence (not even close). No, this isn’t the end of Google (they’re doing well with AI thank you). No, as we heard at CES, 90% of content won’t be machine-generated in two years. No, it’s not the end of writing. As often we find the truth by following the money. And in the attention economy there’s big money in hysteria for tech-commentators and their book agents, not to mention OpenAI’s lucky stock holders.</p><p>Here’s what’s likely to happen, besides hours wasted on senseless blog posts:</p><p>1. <strong>A handful of creators will leverage Generative AI models to take their wotkflows to the next level</strong>. Our most precious commodity is time. Those creators who can leverage ChatGPT into their workflow to laser-focus their time on the core craft of exploring higher-level human narratives will win the era of augmented content creation.</p><p>2. <strong>Some cool apps will emerge</strong>. Most of our short-form communication is made of commoditized, pro-forma replies that we do mechanically for the performance of politeness. This will be automated, thankfully. And ultimately written politeness will disappear altogether when it’s clear that machines, not people, have been thanking you for years.</p><p>3. <strong>The cat and mouse game between digital marketers and content distributors will become nuclear warfare</strong>. We’ll see a dramatic amplification of the already overwhelming abundance of high calorie, low-nutrient digital content (bad tweets, biased product reviews, boring Instagram posts) designed to game the search and content recommendation platforms. ChatGPT will make this problem exponentially worse. And yes, some clever junk content publishers will make a lot of money for a little while. And yes, search engines will need to apply enormous efforts to combat them.</p><p>4. <strong>Education will get seriously disrupted, Halleluiah</strong>. By automating regurgitation, ChatGPT will force the education sector into the modern age of teaching kids how to digest oceans of public information to solve complex, systems-level problems.</p><p>5. <strong>Curation will continue to rule. And Hollywood will as well.</strong> In an ocean of content, the value lies with curation and personalization. Time being the world’s most precious commodity, exhausted digital denizens will pay a high premium for a service that can deliver them the exact content that they need or that inspires them. And nobody is better at sorting this signal from the noise than the a giant talent-filtering algorithm called Hollywood, which also knows something Silicon Valley keeps ignoring: people actually hate technology.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab30bfd0ffae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Ethics: A Framework for the Media Industry]]></title>
            <link>https://medium.com/@ybergquist/ai-ethics-a-framework-for-the-media-industry-2a5cf2719e13?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a5cf2719e13</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-ethics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ethics-in-tech]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Mon, 09 May 2022 18:53:49 GMT</pubDate>
            <atom:updated>2022-05-09T18:53:49.253Z</atom:updated>
            <content:encoded><![CDATA[<p>By Yves Bergquist for the Entertainment Technology Center.</p><p>IN THE SUMMER OF 2020, the Entertainment Technology Center (ETC) at USC and the Society of Motion Pictures and Television Engineers (SMPTE) joined forces to create the Joint Task Force for Artificial Intelligence in Media.</p><p>THE TASK FORCE’S MANDATE is to research opportunities to deploy Artificial Intelligence throughout the media industry, and write a report for SMPTE’s standards committee on what standards should be created to accelerate this process. ETHICS WAS IDENTIFIED EARLY by Task Force members as one of the major areas of focus for its work. This was confirmed by data scientists and media executives during several industry-wide roundtables on AI organized by ETC.</p><p>THIS PAPER AIMS TO REFLECT the ideas and suggestions of the group as a whole about what ethical considerations and best practices need to be made in the development and deployment of AI systems in the media industry.</p><p>THIS DOCUMENT AIMS TO GUIDE media executives and organizations involved in developing or deploying AI applications. It aims to provide a starting point for an industry-wide conversation about computational ethics as a whole. Many people provided review and feedback to this paper, and we are grateful for their help. SMPTE AND ETC welcome open debate about one of the most potent technologies of our time.</p><p>This can be done through ETC’s AI Roundtables, which are held remotely every 4 to 6 weeks, and are open to ETC and SMPTE members. Contact yves@etcusc.org to participate.</p><p>For information about guest participation in SMPTE Standards activities, contact dir.standards@smpte.org.</p><h3>I— Introduction</h3><p><em>( a ) What is AI ethics?</em></p><p>AI ethics is the set of ethical considerations involved in the design, development, and deployment of artificial intelligence systems. Because they are hand-built, by humans for humans, AI architectures encode organizational and human biases at all stages of the data science pipeline, from data collection to model deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IzDE5TtkYSNqNkeP" /><figcaption>Especially when used in decision-making contexts, AI and machine learning technology (and all computational tech, really) can give bias enormous scale. Source: ZDNet.</figcaption></figure><p>As such, ethics are concerned with virtually all aspects of AI development: ethical intent in the design phase, respect for privacy and minority representation in data collection, racial, gender, and cultural bias in training data, inclusivity in data science teams, biases in machine models, the need for transparency and explainability, and of course benevolence of end-uses. AI ethics is a complex and evolving ecosystem of practices, systems, and goals. First and foremost, it’s an organizational mindset.</p><p>In the media industry, this means putting ethics on the agenda of data science and product teams. It means making deliberate and ethically conscious decisions regarding such concerns as data privacy, the use of synthetic media, and content recommendations. It means hiring culturally diverse engineering teams. Communicating transparently with consumers about how their data will be used and decisions about them made. It means training data science teams on identifying cultural biases in data and models, as well as -perhaps most importantly- how to talk about them with business stakeholders.</p><p>Direct-to-consumer business models and the considerable opportunity presented by the emergence of Multiverse environments are pushing the industry farther into the arms of machine models, and thus creating even stronger ethical requirements.</p><p>As computational intelligence spreads throughout media-making decisions and workflows, it’s important for organizations to develop the confidence that their datasets and models are consciously built and fully known, warts and all. This means they are transparent and auditable, trusted, rid of invisible and unwanted biases, and do not result in privacy violations or discriminatory outcomes. Computer vision models, large language models, and generative models, have all crossed a threshold of performance that carries opportunity and ethical risk. “Deepfakes” arise from a powerful but ethically dangerous technology.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CS7GK-LXAHOpO8vW.jpg" /><figcaption>The 2020 HBO Documentary “Welcome to Chechnya” used “deepfake” technology to mask the faces of the Chechen LGBTQ activists it profiled, to protect their identity — and safety. This was the first reported use of the technology in documentary filmmaking.</figcaption></figure><p>Similarly, unbound synthetic characters and agents (chatbots) pose considerable ethical dilemmas. Ethics is altogether a practice, a mindset, and a conversation. It’s an emerging and uncertain, yet essential field. And not just because of ethical risk. Robust, conscious, and transparent data science helps everyone.</p><p><em>( b ) Why should ethics be useful in AI development?</em></p><ol><li><strong>BECAUSE AI IS EARLY, CRITICAL, AND MISUNDERSTOOD</strong>. AI is at the same time disruptive, vague, complex, experimental … and a great story. It’s hard to understand and easy to load up with fears and fantasies. This is a dangerous combination.</li></ol><p>The convergence of corporate hype, fledgling methods, incomplete and biased datasets, and the urgency to productize, are all a fertile ground for failure. Failure in tech is good, except when when models are put in a position to make decisions about such areas as policing, hiring, synthetic conversations, or even content recommendation and personalization. Then failure comes at a high human cost.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*JvrrpscNf-_V9f69VQQ9Ug.png" /><figcaption>Potential biases in the use of AI/ML methods in hiring is of particular concern. See Bloomberg Law’s: <a href="https://news.bloomberglaw.com/business-and-practice/law-firms-turn-to-ai-to-vet-recruits-despite-bias-concerns">https://news.bloomberglaw.com/business-and-practice/law-firms-turn-to-ai-to-vet-recruits-despite-bias-concerns</a></figcaption></figure><p>The time to talk about ethical considerations in AI is now, while the field is still nascent, teams are being built, products roadmapped, and minds made up.</p><p><strong>2. BECAUSE IT’S THE LAW</strong>. According to the United Nations Conference on Trade and Development, 77% of all UN member states already have data privacy laws or have pending legislation. GDPR (European Union), and the joint CCPA/CPRA in California have already alerted the private sector on how much attention regulators are paying to consumer data and artificial intelligence-driven decisions.</p><p>Around the corner, EU’s proposed Digital Markets Act makes ethical requirements and data privacy even more ominous. Closer to us, the City of Los Angeles recently took legal action against IBM for misappropriation of user data for the latter’s weather app. Goldman Sachs has been investigated for discrimination against women in some credit card applications. The list goes on, and it will unfortunately get much longer.</p><p><strong>3. BECAUSE FAILING IS EXPENSIVE</strong>. As seen above, AI development is no longer just a technical issue, it is increasingly becoming a risk factor. Because AI is altogether experimental, impactful, and expensive, organizations need to examine the downside risk of deploying underperforming and unethical AI systems. Especially because, in most cases, ethical and technical requirements are the same. Unseen bias is as bad for model performance as it is discriminatory, for example. Model transparency isn’t just an ethical consideration: it’s a trust-building instrument for organizations still viewing AI with suspicion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*mqBfCAZuZq2v42vxZf9C5w.png" /><figcaption>A 2019 study published in Nature broke new ground in identifying algorithmic bias in health care: <a href="https://www.nature.com/articles/d41586-019-03228-6">https://www.nature.com/articles/d41586-019-03228-6</a></figcaption></figure><p>In 2017, Amazon had to famously scrap a costly machine-driven job applicant processing piece of software because it discriminated against women (it was trained on an an overwhelmingly male dataset). This cost the company in 3 ways: (1) the obvious reputation hit for a technology leader, (2) the cost of developing, then scraping, a faulty application, but most importantly (3) the opportunity cost of making bad decisions based on biased machine learning models. The next year, an autonomous vehicle tested by Uber killed a pedestrian in Arizona, in part because its model had not been properly trained on jaywalking samples. If we want intelligent machines to make big decisions at scale, we must recognize — and mitigate- the costs of failure.</p><p><strong>4. BECAUSE MEDIA NEEDS ITS OWN VOICE.</strong> The media and entertainment industry is a tech industry. As such, it has its own voice, its own culture, and nearly 150 years of success marrying human and technological genius. It also holds a substantial and powerful place in our society as the mass distributor of human narratives and social norms.</p><p>Media needs to bring this unique voice and hybrid human/machine culture both to AI development and the debate on AI ethics. And as the industry starts developing and deploying AI applications from development to distribution, there is a need to approach this issue at the industry level first.</p><p>Media and entertainment companies collect and process large amounts of consumer data, for example. Increasingly this means that they need to comply with a growing list of legal regimes and data governance requirements. Similarly, there’s a substantial opportunity to use computer vision in the production (virtual production) and post-production processes (color correction, translation and localization, and of course vfx work).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tqv86aCwHPGJxkY8Avwalw.png" /><figcaption>The 2021 Anthony Bourdain documentary “Roadrunner” was met with controversy after it appeared that it had artificially recreated Bourdain’s voice for some segments. <a href="https://www.newyorker.com/culture/annals-of-gastronomy/the-ethics-of-a-deepfake-anthony-bourdain-voice">https://www.newyorker.com/culture/annals-of-gastronomy/the-ethics-of-a-deepfake-anthony-bourdain-voice</a></figcaption></figure><p>The quality and diversity of training sets, how color correction can affect representation of minorities, and of course the use of “deepfake” technology, are all critical areas where ethical considerations are paramount. The media industry’s history of sophisticated legal practice around likeness rights and participations is a substantial advantage in navigating issues related to computational derivatives of image and content.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*60HWeNoxmQRx7uhP.jpg" /><figcaption>As mentioned previously, “deepfake” technology is one area of application of machine learning methods in media that bears both promise and risk. See: <a href="https://www.scientificamerican.com/article/detecting-deepfakes1/">https://www.scientificamerican.com/article/detecting-deepfakes1/</a></figcaption></figure><p>At a minimum, the requirements of data and model transparency would go a long way towards reinforcing trust in computational methods and help convert those in the industry still reluctant to use statistical learning to optimize human processes. Around the corner, the development of conversational agents (chatbots) creates serious ethical risks, especially as the industry looks to create highly immersive and personalized experiences in the multiverse.</p><p><strong>5. BECAUSE IT’S FUNDAMENTAL.</strong> As mentioned, technical and ethical standards in AI are overwhelmingly one and the same. Bias is the model-killer. Black box algorithms are inscrutable and can lead to serious unseen bias or underperformance. Intellectual and cultural diversity is critical to high performance in dat science teams. It’s healthy for product teams to broaden their system view and consider ethical and societal applications of their work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z6p3GZH3O_iAupZSJY1P7g.png" /><figcaption>Fairness use cases extend beyond the “traditional” realm. Source: Dr Kush Varshney, “Trustworthy Machine Learning”.</figcaption></figure><p>Entangling the ethical implications of AI goes a long way towards deepening our collective understanding of the field. And thinking about AI ethics forces us into systems thinking, which is an almost Darwinian imperative in all areas of contemporary business, technology, and society.</p><p><em>( c ) Some core principles</em></p><p>In his excellent book”Trustworthy Machine Learning” (available for free at <a href="http://www.trustworthymachinelearning.com),/">http://www.trustworthymachinelearning.com),</a> IBM researcher Kush Varshney compares the challenges of trustworthiness in AI and machine learning to those pf processed foods. In the early XXth century, processed food companies like Heinz had to gain the trust of consumers and regulators through “unadulterated ingredients, transparent containers, sanitary food preparation, factory tours, labels, and tamper-resistant packaging”. Inspired by this effort to build trust in an essential component of society (food), here are some core principles of AI ethics for the media industry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*BkAWF_pVW8R7X4slNB5Mmg.png" /><figcaption>Pioneering work on Trustworthy Machine Learning by IBM researcher DR Kush Varshney: <a href="http://www.trustworthymachinelearning.com/">http://www.trustworthymachinelearning.com/</a></figcaption></figure><ol><li><strong>BROADNESS</strong></li></ol><p>AI ethics need to be viewed in the larger context of computational ethics. Whether or not they are build upon AI or ML architectures (increasingly they are), systems with impact on an organization, a business model, a revenue stream, a key life decisions (such as hiring or getting a mortgage), a minority group, or medical treatment, are subject to bias. As such, they need to be understood, built transparently, and audited regularly. Computational bias, AI or not, means bias on a massive scale. Ethical considerations should be a systematic part of all aspects of digital product design, development, and QA. This seeding of ethics at the product level is essential to look at bias as a complex ecosystem of inputs, features, models, outputs … and outcomes.</p><p><strong>2. FIT</strong></p><p>We are what we build. Any organization’s output, products, and decisions (deliberate or not) inherently fits its culture and values. This is why AI ethics is high stakes: it deploys an organization’s culture and values on a large scale. Because they shape society at scale and have a history of taking the public interest seriously, media companies have a distinct responsibility to move forward in their AI ambitions in full awareness of these applications’ ethical considerations. They should ensure that all aspects of their development (including data collection), deployment, and end-uses, support the law as well as their own values regarding privacy, justice, tolerance, and human rights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ikz9aGxtWMRkKdCjRZrnVw.png" /><figcaption>What are the most often-cited attributes of a “trustworthy” machine learning application. Source: Dr Kush Varshney, “Trustworthy Machine Learning”.</figcaption></figure><p><strong>3. INCLUSIVITY</strong></p><p>Gender, racial, social, intellectual, and cultural diversity of all kinds are critical to maintaining a richness of voices, societal experiences, and cultures when developing AI systems. It’s not just the right thing to do, it’s the smart thing to do. AI and machine learning is extremely hard. It touches upon a large number of technical, academic, cultural and intellectual domains.The more diverse voices, the more viewpoints, the more creative solutions, the more chances of success. Diversity creates richness in products and organizations, and is a critical factor in the performance of data science teams. It’s also a good remedy against confirmation bias, which can be costly once enshrined in organizational processes and systems.</p><p><strong>4. TRANSPARENCY AND TRUST</strong></p><p>The entire value chain of AI development, from product design to data collection to model deployment, should be secure, transparent, explainable, and auditable. Black box machine learning frameworks are both ethically and statistically dicey. They foster sloppiness in data science teams and mistrust for those already suspicious of machine models. Sure, it’s hard to audit the feature representations of each layer of a deep neural net, but that’s an audit problem, not a transparency problem. What can’t be explained should not be deployed in a decision-making environment. Only secure, transparent, explainable, and auditable machine models can scale in organizations that are often too suspicious or too enthusiastic. Additionally, all stakeholders deserve transparency, each in their own language, across different points of view and technical sophistication. Ethics should be part of Quality Assurance for any and all computational systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/919/1*d4o2dZis4JtwEuVZa8kzAw.png" /><figcaption>Dr. Kush Varshney’s “Cycle of Trust”. Source: <a href="http://www.trustworthymachinelearning.com/">http://www.trustworthymachinelearning.com/</a></figcaption></figure><p><strong>5. OPENNESS</strong></p><p>AI is still a technical Wild West. Everything around it, from roadmapping to modeling to seeding in company culture, is hard, and will remain so for a long time. Mistakes will happen. Organizations need to communicate comprehensively and with humility about their journey to approach and implement processes around ethical AI. For the benefit of all. After all, we’re all trying to make ethical something that even experts still can’t fully understand. Transparency will help regulators, senior business executives, and the general public understand that artificial intelligence is the exact opposite of “magic”. It’s either blood, sweat, and tears … or it’s not AI. Just like technical and organizational implementation, ethical considerations in the development and deployment of AI are complex and laborious. Being open and didactic will not only feed the public debate about AI with realistic and trustworthy narratives (as opposed to noxious hype), but will create a collective mindshare for organizations to learn from each other’s successes and failures.</p><h3>II — THE AI ETHICS PIPELINE</h3><p>Implementing the previous principles is where most of the challenge lies. It’s harder (and expensive) to audit machine learning models than to build them. Besides, the AI field is still early, and AI ethics is an almost entirely blank slate. Examples of successful, organization-wide implementation of machine learning transparency and trustworthiness are extremely rare. Nonetheless, some early experimentation in the pharmaceutical and financial services industries have suggested some best practices.</p><p><em>( a ) Organization</em></p><p>This is perhaps the most critical step in laying out an AI ethics strategy, because nothing is more impactful — and difficult to build- than organizational systems, incentives, and mechanisms. This is where AI ethics get enshrined. Here are a few principles borrowed from successful experiments deploying AI governance in a corporate environment:</p><ol><li><strong>Set clear goals but flexible roadmaps.</strong> AI ethics is a nascent and uncertain practice that touches upon virtually every business process. It needs flexibility to experiment and diverse buy-in to flourish. It’s a good idea to have open and transparent conversations at all levels about expectations prior to setting a roadmap. In media, it means that virtually all sectors across marketing, development, and technical implementation have a piece of the puzzle and a role to play in setting expectations for an AI ethics initiative. Also, the AI ethics work is never done, this will be a perennial trial and error process.</li></ol><p><strong>2. Inventory organizational resources already available to seed an AI ethics program</strong>. Chances are some foundations of an ethics practice already exist within the organization. Set up an executive committee inclusive of all voices, business units, technical backgrounds, and cultures. Promote the initiative (and the group) internally. Educate and train to create a level playing field.</p><p><strong>3. Create clear lines of responsibility and accountability.</strong> Ethics is funded, incentivized and supervised at the corporate level, but its implementation needs to be bespoke to the needs, resources and priorities of each business unit. Product managers in each business unit should be front and center in leading the deployment of ethics policies and practices.</p><p><strong>4. Foster cultural and intellectual variety.</strong> Because of the multifaceted nature of AI ethics, working groups should include a wide variety of stakeholders. This obviously means gender, racial and cultural diversity, but not only. Consumer research teams, product managers, legal and compliance teams, and data scientists all bring a different perspective on balancing the requirements of ethics with that of performance and customer experience. Ethics should not be the exclusive domain of those preoccupied with governance and risk.</p><p><strong>5. Communicate with senior stakeholders.</strong> Learn how to talk about AI and ethics with senior business executives. C Suites and legal executives looking for clear and certain ROI in their AI efforts — including ethics- ignore how experimental the tech still is.</p><p><strong>6. Make the process as transparent and measurable as possible.</strong> Measurement is important in any trial and error process. So is transparency about mistakes and lessons learned with regards to building ethical and trustworthy AI applications. It’s a completely new domain related to a completely new technology. Failure will happen.</p><p><em>( b ) Product design</em></p><p>Because computational ethics (not just in AI) need to be implemented as closely as possible to those with responsibility over the use case behind AI applications, two roles within organizations take a pivotal role in AI ethics implementations: insights leaders and product managers. The first one (internal facing, focused on processes) is straightforward: it is the responsibility of the head of insights to ensure that they are collected, processed, and outputted transparently and ethically. Data and model integrity are a core responsibility.</p><p>Product managers could also take a central role in leading external-facing (focused on products) AI ethics considerations. Because they are by nature systems thinkers, care about the customers at least as much (if not more) as about the company, and are ultimately accountable for an organization’s raison d’être (its products), product leaders are essential “quarterbacks” of computational ethics. They are best positioned to weight all considerations of transparency, integrity, and functionality. Plainly put, they sit at the intersection of product and users and can best weigh user experience vs ethical requirements.</p><p>Whoever is given the lead to examine ethical considerations in computational systems should start by laying out requirements of “AI trustworthiness and transparency” that are specific to each stakeholder. A customer will have different needs, and speak a different language, than a marketing executive, or a data scientist. And ethical considerations even vary within customer types. For example, audiences must trust an AI-driven recommendation algorithm to “know” their specific tastes, while intelligently expanding their creative horizon. Marketing executives and analysts must trust that a sentiment analysis engine correctly classifies positive from negative sentiment in most cases (deeply semantic sentiment domains like sarcasm are still difficult to measure). A digital product manager must trust that her virtual character won’t stray into inappropriate conversations with users. Listing all stakeholders and analyzing their various cultures and needs is a useful initial step in the AI ethics pipeline. Performance and transparency are also big components of trustworthiness. It’s critical to label outputs of AI and ML models with their performance.</p><p>Identify clearly where technical, business and ethical goals are aligned, and where they are not. Start with he former, and promote quick wins with low hanging fruits (see the hierarchy of needs pyramid). Model performance and sample bias are examples of ethical issues where technical, business, and ethical goals are aligned. Other issues, such as the decision of whether or not to use “deepfake” technology in the VFX process, may require more extensive consultations with C-level executives. Some products may perform better with more intrusive consumer data collection. This could be seen as a user experience problem, where users will have the choice between an enhanced experience that collects data more aggressively and a limited experience that protects privacy.</p><p>Finally, an effective AI ethics program will make checking for bias, trustworthiness, and transparency a function of quality assurance. This is a critical part of the ethics pipeline, as it ensures balance between those requirements and the needs of user experience and product performance</p><p><em>( c ) Data collection</em></p><p>The data collection process is very much at the heart of the AI practice as a whole, and of ethical considerations in particular. “Garbage in, garbage out” is the Golden Rule of data science: models are only as good as the data they are trained on. Identifying biases in data collection (and monitoring how bias might increase over time) is at the heart of the AI ethics practice.</p><ol><li><strong>KNOW THY DATA</strong></li></ol><p>This is perhaps the most important part of the AI ethics pipeline. It’s also a major area where statistical and ethical requirements are one and the same.</p><p>Data is the raw material of data science, and it is data scientists’ first and foremost responsibility to know their dataset, its strengths and weaknesses, inside and out, to be able to map issues with a skewed output (the model) back to skewed inputs (the data). Use representative datasets that fully take into accounts gender, race, culture, etc. This is not just the right thing to do, it’s the statistically sound thing to do.</p><p>Sample bias is a primary source of poor ethical outcomes in AI. For example, facial recognition applications have been notoriously underperforming in the detection of both darker skin tones and females (see Dr. Joy Buolamwini’s and Dr. Timnit Gebru’s “Gender Shades” study), due to substantial under-representation of darker skinned samples in computer vision training sets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*c5VlHmPkfGwf6FaF.jpg" /><figcaption>The debate over error rates in identifying darker-skinned males and females rages, with different studies confirming that it is critical to deeply understand one’s training sets. See: <a href="https://physicsworld.com/a/fighting-algorithmic-bias-in-artificial-intelligence/">https://physicsworld.com/a/fighting-algorithmic-bias-in-artificial-intelligence/</a></figcaption></figure><p>In their 2018 paper, Gebru and Buolamwini noticed that two of the most prominent training sets of faces at the time, IJB-A and Adience, are composed of 79.6% (IJB-A) and 86.2% (Adience) of lighter skinned faces. They found that the maximum error rates for lighter skinned males in these models was 0.8%, vs 34.7% for darker-skinned females (<a href="http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf">http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf</a>). A subsequent 2018 study (https://arxiv.org/abs/1812.00099) dove further into other standard datasets and found a different explanation, which illustrates the need to be intimately familiar with one’s data.</p><p>Similarly, the bias in Amazon’s hiring software came from the fact that it had been trained on resumes received by the company over a 10-year period. An overwhelming majority of these resumes were from men. As a result, it penalized resumes that included the word “women’s”, as in “women’s basketball team”.</p><p>This is a complex and very multi-layered issue to tackle. For example, even very subtle nuances in how data is collected (the way a question is asked, or how incentives to contribute one’s data are structured) that can dramatically affect the resulting dataset. Knowing how the data has been collected, and what biases may lie within that process, is increasingly a core responsibility of data scientists.</p><p><strong>2. KNOW THY PROBLEM</strong></p><p>AI and machine learning are used to solve real world problems by using data to represent and model those problems in their larger context. This is identical to how the human brain functions: we use data to model (summarize) an infinitely complex world, and use those models to act upon the world.</p><p>Knowing intimately the problems, systems, behavior, phenomena they are trying to model, and in this case, what biases may be inherent in them, is a key strength of great data scientists. Some parts of our world are simple, but most are extremely complex (not least of which human behavior). When modeling a real-world system (such as audience decisions) data scientists need to understand is the set of variables they are analyzing accurately represents (e.g., sums up) the larger system they are trying to generalize their findings about. They also need to identify if and when the inequities or biases of the system itself will get passed on to the model.</p><p>Often times the set of variables available to the system is too small and partial for the model to generalize to a much more complex (and fast evolving) real-world system. Over and under-fitting are the statistical manifestations of this issue, as is encoding real-world biases in machine models.</p><p>This is especially important when modeling human behavior, specifically during audience research. Posting a thread on Reddit or retweeting a tweet on Twitter or liking a post on Facebook are 3 radically different kinds of social behavior expressed by different genders, age groups, races, and sub-cultures. And models about them generalize differently, which is why it’s critical, in the practice of AI ethics, to maintain intimate knowledge of how underlying social, cultural or behavioral biases may impact data collection. It’s critical for data scientists to understand the underlying biases not only in the input data but in the systems they are trying to model.</p><p>This is best done as a collective process, since confirmation biases (the tendency to look for, cherry pick, or interpret insights according to one’s preexisting beliefs) are also present in data science teams, or quantitatively-driven functions such as marketing and consumer research.</p><p><strong>3. COMMUNICATE CLEARLY ABOUT USE, BIASES, &amp; TRADEOFFS</strong></p><p>When collecting user data, opt-in is a must (it’s also increasingly the law). The best practitioners in this domain avoid legal language and use instead a simple user-facing explanation of how personal data will be used, and what the implications of opting in and out are for the user experience (for example, an opt-out of sharing data would affect personalization).</p><p>Bias often can’t be avoided, in which case it’s critical for data science teams to communicate fully and clearly to end-users about their model’s skews resulting from the bias. A simple annotation in the output can be very powerful in building transparency and trust, without which no culture of data (let alone AI) can be successfully scaled in media organizations.</p><p><em>( d ) Modeling</em></p><p>Traditional measures of performance In ethically compliant AI or machine learning, building trust is both critical and labor-intensive. There are two parts of this: creating explainable statistical models (model transparency) and effectively reporting key features of the model, so it can be quickly audited by endusers for bias and potential performance variations.</p><ol><li><strong>MODEL TRANSPARENCY</strong></li></ol><p>Transparency means building simple models to explain complex ones, to give data scientists a window into often complex and inter-locking machine learning architectures. This is increasingly a challenge, as neural net architectures become deeper and more integrated with other types of models. Luckily, the past few years have seen a flurry of development of AI transparency tools.</p><p>All providers of cloud-based machine learning have started offering tools to interpret and understand their models (for example, AWS’s Sagemaker model explainability feature — based on SHAP-, Microsoft’s InterpretML toolkit, or Google AutoML’s feature scoring tool). These are useful because variables in a mode are hierarchical (some are more powerful than others within the model), and surfacing that structure is a key step in understanding how the power of certain variables related to gender, race, or culture, for example, may perpetuate inequalities.</p><p>Here are some good and popular explainable AI tools:</p><p><strong>SHAP (SHapley Additive ExPlanations):</strong> this is perhaps one of the most widely known AI transparency tools, because it works across a wide range of models, from linear regression to deep learning, and covers a wide variety of domains including computer vision and NLP. SHAP uses a game theoretic approach to rank features (for example, words in a sentiment analysis model) by order of importance in predicting the output. This is the kind of hierarchical view that is very helpful not just in the context of AI ethics, but for data scientists to QA their own models. Transparency one of many areas serving both ethics and model performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nfJpaUSH3iLXpP8c.png" /><figcaption>Example output of SHAP showing a hierarchical view of variables in a model</figcaption></figure><p><strong>LIME (Local Interpretable Model-Agnostic Explanations):</strong> similar to SHAP, but more computationally effective (quicker). LIME also ranks features by ow much it contributes to the output (in an image classifier it can produce a heat map of an image with “useful” features in green and “not useful” features in red — SHAP can do this as well). This is very popular for Python’s scikit-learn users because of its built-in integrations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y5BXYVHU2MQzVO58.png" /><figcaption>Simplified explanation of how LIME works</figcaption></figure><p><strong>ELI5:</strong> this works similarly to SHAP and LIME but is perhaps the most popular transparency packages in Python, because of its integrations across the board with scikit-learn, XGBoost, Keras, and a few more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/0*kYW9JKcXONfB0QmX.png" /><figcaption>Feature ranking in Eli5</figcaption></figure><p><strong>AIX360:</strong> developed by IBM Research but still open source, this toolkit has extensive functionality and isn’t dissimilar from Google’s “what if” tool, but can be used outside of the Google CloudAI environment, although it is not for beginners.</p><p><strong>Google’s “What-if Tool”:</strong> this allows data scientists to test a model’s performance under a variety of different situations. It helps understand the impact of various variables (such as race or gender) on the model itself. This is an excellent and intuitive tool for beginners using the Google Cloud AI infrastructure, as it has a very good visual interface and can be run easily (and with minimal code) from platforms such as Jupyter Notebooks, Google Colab, and even Tensor Flow’s TensorBoard dashboard. It can be used at various stages of the data science workflow. It can support TensorFlow models out of the box. Works with tabular, image, and text data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3gHevgRiTOxAMCL9.png" /><figcaption>Screenshot of Google’s “What If Tool” dashboard</figcaption></figure><p>Transparency is not just key: it’s a perennial concern. The world changes, the problem changes, the data changes, and model performance is affected. There is no longer a fit between the model and the system, or behavior, it’s representing. “Model drift”, as it’s called in data science circles, impacts ethical outcomes, because what may be ethical in January may no longer be in June. Only transparent and auditable models can catch model drift before it causes damage.</p><p><strong>2. MODEL REPORTING: MODEL CARDS</strong></p><p>Too often, machine models are released with incomplete documentation. As a result, they are applied to contexts in which they do not perform well or are not appropriate to deploy in. Or they exert substantial yet hidden influence on decisions that affect us as digital citizens. This is a rising concern in public policy, where local and national governments are starting to expose computational decisions and how they are made. Recently, the cities of Helsinki of Amsterdam have created a public online registry laying out in detail the algorithms, models, and data used to make specific public decisions. It’s only a matter of time until this becomes a widespread rule in the public sector. There’s an interesting framework for this.</p><p>Created by Dr. Margaret Mitchell and Dr. Timnit Gebru’s team at Google in 2018, “model cards” are standardized documentation laying out all the information necessary to evaluate a model and benchmark its performance in a variety of contexts. Sure, libraries and models often come with documentation, but it’s often incomplete, too long, and generally could use standardization. Model cards are standardized “food labels” for data science that also -ideally- benchmark a model’s performance in a variety of contexts and use cases, some related to inclusion and bias. Per the Mitchell/Gebru/team paper (“Model Cards for Reporting”: <a href="https://arxiv.org/pdf/1810.03993.pdf):">https://arxiv.org/pdf/1810.03993.pdf):</a></p><p>“<em>Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information</em>”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RNUWn7sHZidD6gPk.png" /><figcaption>Example of a Model Card. Source: <a href="https://ai.googleblog.com/2020/07/introducing-model-card-toolkit-for.html">https://ai.googleblog.com/2020/07/introducing-model-card-toolkit-for.html</a></figcaption></figure><p>As stated consistently in these pages, there’s no real playbook for AI ethics. The technology is early, and its ethical considerations another area where the hype has vastly preceded the reality. Even regulators have, by and large, limited their interest to data privacy — for now.</p><p>But because the technology is so complex and increasingly important, and its presence is so ubiquitous, it needs to be approached and roadmapped through multifaceted systems thinking. There are simply too many components in any serious artificial intelligence effort to avoid considering its requirements and ramifications — especially ethics- as anything but a system.</p><p>The most successful organizations in building and scaling AI internally are the ones that think about it the most thoroughly and systematically. And nothing forces an organization to think deeply — and in systems- more than ethics. Far from window-dressing or virtue signaling, putting ethics front and center will bring about the modes of operation, intellectual rigor, and organizational culture necessary to excel in building AI systems.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a5cf2719e13" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using AI for Consumer Insights in the “Indefinite Normal”​: A Case Study]]></title>
            <link>https://medium.com/@ybergquist/using-ai-for-consumer-insights-in-the-indefinite-normal-a-case-study-2e5b68d84636?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e5b68d84636</guid>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[branding]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <category><![CDATA[analytics]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Wed, 03 Jun 2020 17:25:58 GMT</pubDate>
            <atom:updated>2020-06-03T17:25:58.614Z</atom:updated>
            <content:encoded><![CDATA[<p>By Yves Bergquist (yves@corto.ai) and Eric Holdener (eric@kinestry.io)</p><p>Whoever said on Twitter that they “no longer want to live in interesting times” probably captured the mindset of all brand marketers right now. There’s crises and there’s … “which way is up?”.</p><p>With the triple reckoning of health, social, and political crises, brands are facing a paradigm shift of unprecedented depth and scale. It’s no longer the message that has to change, it’s the DNA of brands’ connection with audiences. It’s not the words, it’s the moral psychology, the social contract, the core values, the identity, and of course the stories. The New Normal was so May 2020. We’re beyond that now: we’re entering a moment of deep profundity and mystery. The Indefinite Normal.</p><p>In this period of complex and high-velocity change, the new brand titans will be those who can create a true relationship with their audiences. Those who can read between the lines, dig deep, talk authentically, contribute meaningfully to their customers’ lives and mindsets. Those who can courageously try, fail, rinse, repeat transparently.</p><p>Scary, huh?</p><p>Luckily, AI research labs all over the world have given those who know where to look, and how to build, tools and methods so powerful that make even the most cutting-edge consumer analytics tools currently available look like Victorian-era automata.</p><p>For the past few years, Corto and Kinestry have been working to leverage frontier AI methods like information theory and probabilistic graph models, to try to take digital consumer insights to the next level. They have built a powerful Beta AI application, CortoY, which for the past 6 months has been quietly tested by a handful of international brands and film studios, with substantial success.</p><p>We recently had the opportunity to leverage CortoY as part of a proof of concept with a major apparel brand, so we wanted to share our findings and start a conversation.</p><p>We looked at 4 aspects:</p><p>Brand identity: who are you to your consumers? What values, needs, stories are expressed in relation to your band?</p><p>Here, using CortoY’s natural language processing stack on social media conversations across multiple sources, the AI engine was able to detect a very strong presence of symbols conveying values such as generosity, individuality, humility, freedom, creativity, empathy and introspection. The brand’s audience was a community of consumers that expressed a strong sense of quiet individualism (even introversion at times) and humility, but within a tight community with a unique identity centered around certain sports.</p><p>Brand psychographics: what emotions are you triggering, and why? How is this evolving during the Covid crisis?</p><p>The traditional Positive/Negative polarity breakdown of social sentiment no longer captures the level of granularity needed to deeply understand how a message will resonate. In our case, the apparel brand wanted to use Corto’s “emotional tonality” engine to precisely capture how emotions about the brand had shifted throughout the COVID crisis.</p><p>The result was stark: not only “active”, “cheerful”, and “adventurous” tonalities had lost about 75% of their strength between the week of February 24 and that of April 6, tonalities such as “depression”, “caution”, “empathy”, and “cooperation” had risen dramatically, as did a focus around family values … and food.</p><p>Content genomics: what topics and emotions in your own branded content generate the best performance?</p><p>One of the most unique features of Corto’s platform, even in limited beta, is that it’s able to analyze features of the brand’s content (narrative structure, topics, emotions, etc.) and audience conversations revolving around that content at the same time. This allowed our brand partner to optimize their content editorial to drives excitement and performance, which we could measure in real-time, for transparency and accountability.</p><p>For example, the tool analyzed all the videos produced for social content by the brand and the ones with an editorial heavy on aggressive and slightly depressed or anxious tones performed 3x better than ones that sounded more agreeable and empathetic. Needless to say, this was pure gold for the brand’s agency.</p><p>Zeigeist Fitness: how does your identity fit the “Zeigeist” of your audience? Are you aligned on the topics, needs, values, stories, emotions that are driving the conversation? If not, how far are you?</p><p>The application was also able to calculate — as precisely as possible — how the needs, values, and emotional tonalities of the conversation about the brand matched the overall Zeitgeist — the essence/mood of our time — across all of these three dimensions.</p><p>In this case, it was eye-opening for our client to see that conversations about them were most aligned with the general Zeigeist in the areas of integrity, empathy, community and social responsibility, and the need for belonging while missing the mark on honesty and the expression of physiological safety, self-esteem and self-actualization.</p><p>These are the types of insights that brands will need in the New Normal. Our ability to deliver deep, genomics-level insights on audiences empower brands to navigate as closely as possible to their customers. Our ability to statistically link elements of messaging to content performance is critical is an era of tightening budgets. Our ability to deliver all of this in real-time was critical in an era of high-velocity conversations and fast-paced change.</p><p>Yves Bergquist</p><p>Yves is an AI researcher and the Director of the “AI &amp; Neuroscience in Media” Project at USC’s Entertainment Technology Center. A think-tank and research lab of the School of Cinematic Arts, ETC is funded by all 5 Hollywood studios, as well as large technology companies such as Microsoft, Cisco, and Verizon. Yves is also the founder and CEO of Corto, an AI startup developing computational tools to help media makers and marketers apply deep science to understanding the genomics of their content, their audiences, and the cognitive relationship between the two. Corto is currently in Private Beta with a handful of studios and international brands.</p><p>Eric Holdener</p><p>Eric is an innovation strategist. He spent over 12 years with Nestle, a Fortune 100 company, in numerous functions both in Switzerland and the USA. His experience includes driving business and technology transformation initiatives, launching products, conducting large-scale LEAN initiatives, and partnering with executive leadership teams on strategic projects to deliver organic growth. He introduced design thinking and developed innovative tools to help organizations visualize value and improve their performance.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e5b68d84636" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI is a Revolution in What and How we KNOW]]></title>
            <link>https://medium.com/vantage/ai-is-a-revolution-in-what-and-how-we-know-14acdebf9816?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/14acdebf9816</guid>
            <category><![CDATA[graph-theory]]></category>
            <category><![CDATA[systems-thinking]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Sat, 18 Aug 2018 20:21:24 GMT</pubDate>
            <atom:updated>2018-08-21T06:02:09.320Z</atom:updated>
            <content:encoded><![CDATA[<p>Most of what you read or watch about artificial intelligence, including from some very prominent field experts, is completely wrong. One of the most frustrating downsides of the decline of institutions and the Age of Individual Empowerment that we are going through is the incredibly low signal to noise ratio in any kind of expertise. The dominant voice on any topic is often just the loudest, or the savviest promoted, and not the most substantive. This is especially disappointing for topics as complex and critical as AI, where commentary ranges from silly and utterly misinformed fear-mongering (looking at you, Elon) to a more educated but extremely partial view of what it is and what it means. Lots of prominent AI researcher like Yann LeCun or Geoff Hinton, for example, are too all-in on scaling computational models up to multi-system and multi-model adaptive intelligence, which is only half of the work.</p><p>It’s especially unfortunate that computer scientists have earned somewhat of a monopoly on the field, making intelligence all about learning (which it is not) and that no prominent philosopher or mathematician (except my friend Ben Goertzel, who I sincerely believe history will remember as the true intellectual forefather of Artificial General Intelligence) has developed any kind of influential voice in the domain.</p><p>To fully grasp what artificial intelligence means, and start thinking about its impact on our society, it’s worth looking back to its philosophical First Principles.</p><p><strong>Representations and the Human Brain</strong></p><p>At a broad, philosophical level, artificial intelligence starts with the notion that, as powerful as it is, the human mind isn’t powerful enough to represent and understand the reality it lives in in its complete mathematical layout.</p><p>Representation is a key concept here (and “knowledge representation” is a critical -yet misunderstood- area of AI). There’s reality (let’s call it N, for the statistically minded) and the entirety of data about it. Humans and machines can’t measure all the position of all the atoms in the universe every single millisecond, so they have to summarize that reality. Humans can only look locally (around them) and in a very abstract manner (symbolic representation). As we’ll see, representational machine learning does almost the opposite (builds n from a lot of atomic-level data about N, as opposed to sampling N and creating symbolic representations of it). Even then, the complexity of what constitutes just my office, for example, or the physical state of my body, are both very high. So the human mind (and even machines) need to compress that reality into a simpler model, or represent N as a simpler statistical model that — hopefully- is a high fidelity representation (often it is not) or N. While reality is the function N, a representation is a compressed, sampled version of N called n.</p><p>A representation n can have high or low fidelity to N, just like the summary of a book can be a good, high fidelity representation of the book, or a poor representation of the book.</p><p>n, in this case, is our human knowledge (basically a bunch of statistical models) about itself and its environment. And that knowledge is a very compressed representation of reality, for which we never have a lot of feedback on, except when we make decisions from it. Sometimes that knowledge is good (and then the decisions we make based on it generate intended outcomes), and sometimes it’s bad (they generate detrimental or unexpected outcomes). “Good n” leads to good decisions, “Bad n” leads to poor decisions. This is important, because both at an individual and an organizational level, evolutionary “fit” (and success, performance in a competitive environment, etc.) is essentially a higher ratio of good to bad decisions.</p><p>Now, for many reasons linked to the structure of the mammalian brain and to how Western education has been shaped by engineers for centuries, we’ve been building mental representations of reality (building a bunch of n models) that are very mechanistic. Medicine is a great example. Look at how it’s organized by arbitrary functions, like the brain, the gut, hands and feet. Is this a high fidelity representation of the human body? Of course not. The body is a much more complex system, and we’ve been going way too long without representing and analyzing it that way.</p><p><strong>Systems of Systems</strong></p><p>The human body, society, the Universe itself, is a complex system of systems. Imagine a Russian nested-doll structure of graphs. A graph is a topological structure made of nodes (entities such as people, cars, atoms, molecules) and relationships between them (called edges). A molecule is a graph (atoms and relationships between them). A cell is a graph of that graph, where molecules become nodes in a higher level graph. That graph, in turn, becomes a node in a higher level, more complex graph that can be an organ or a muscle. That graph of graphs then is a node in a higher level graph that is the human body, which itself is a node in higher level nested graphs of family, community, society, country, mankind, etc. You get the sense of how complex this is: it’s mind-bogglingly complex (remember all of these entities and the relationships between them are constantly in flux), and computing all of these relationships together to represent and predict the state of even any of these sub-systems is, for now, virtually impossible (that’s the whole challenge of AI).</p><p>This data structure is called a “hypergraph” (a graph or graphs), and it’s perhaps one of the most profound and fundamental concepts in mathematics. Because it is a direct reflection of how the Universe is organized, a hypergraph is by far the most high-fidelity type of knowledge representation, and one of the most promising areas of research in AI (as you can imagine, it’s incredibly hard to compute such a complex graph on large datasets).</p><p>To create a much higher fidelity representation of our reality, humans should be thinking in systems (N). But because it’s far too complex for us to do so (the amount of computation is completely insane), we’re thinking very mechanistically trying to build a very compressed representation of N as a bunch of simple systems interacting together in a very linear fashion (still a graph, but a very rough and inadequate one), vs a more granular look at how certain cluster of nodes in a system might impact other cluster of nodes in another.</p><p><strong>Enter AI</strong></p><p>In 1992, an IBM engineer named Gerry Tesauro created a backgammon-playing machine learning application (based on a shallow neural network) called “Neurogammon”. The application would play at and even above human level, but in a way that didn’t quite make sense to even the best players in the world (see Gerry’s paper <a href="http://www.bkgm.com/articles/tesauro/NeurogammonANeuralNetworkBackgammonProgram.pdf">here</a>). Fast forward to 2016: DeepMind (owned by Google) extended the work of Tesauro -and many others- to create a Go-playing application that performed above the level of even the best human players at the time. Here again, AlphaGo played the game in ways that often didn’t resonate with experts (a really good blog post on this is <a href="http://deeplearningskysthelimit.blogspot.com/2016/04/part-7-review-of-game-2-alphagos-new.html">here</a>). More recently, an OpenAI team created a very interesting hierarchical neural net application, called “<a href="https://blog.openai.com/openai-five-benchmark-results/">OpenAI Five</a>”, which beat some of the best players of Dota 2 at a very simplified version of the game. Once again, the application played the game in ways that were very unusual to human players.</p><p>You get the idea. In each of these examples, AI applications (thanks to clever architectures and gigantic amounts of computation, which many AI purists take issue with) we’re able to build vastly more complex representations of the system of these games than even the best human minds could. All of these applications performed what’s called “representational learning”, which is a type of machine learning (of which neural networks are a part of) where machines build their own hierarchical representations of the “system”, which is made of all the complex patterns in the data they are fed.</p><p>Representation learning is impressive specifically because it takes the entirety of the data about a system and learns to build compressed representations of it. It build n from N. This isn’t how the human mind works. Our brain has evolved over millions of years to promote efficiency, which means we don’t have the ability to be fed all the possible information about a system (except relatively simple ones) to create an optimal n representation of that system.</p><p>Instead, we have to strategically sample the data and build the best possible n representation of N, which we’ll iterate probabilistically through trial and error, and pass on these optimized models to future generations (new brains) so that — hopefully- the learning that happens in a machine in a few hours can happen over a century or more. So the human brain is not great a knowing in and of itself over a single lifetime, because it’s built to optimize its representations of the system of systems it lives in a collective manner (transfer learning) and over generations of agents and thousands of years. Sure, it’s worked pretty well so far, but it’s very slow. And for any of us, it doesn’t feel great knowing that we’re just a link in a long chain of evolutionary algorithmic optimization. Or to borrow an engineering term, just one “sprint” in a much larger development project.</p><p><strong>A Revolution in Knowledge</strong></p><p>At the most fundamental and philosophical level, AI promises to considerably accelerate this process by having powerful and -hopefully- highly intelligent machines to derive the best n from N. Armed with large datasets and such a capacity to create high-fidelity representations of complex systems, humans could become exponentially more knowledgeable about themselves and their environment.</p><p>What happens when were able to think about complex phenomena like cancer, economic development, or human performance, with hundreds of thousands of variables instead of a handful, which is how human knowledge has been developed? We’re suddenly able to create much more accurate and high-fidelity representations of these complex systems, and we’re able to develop better knowledge of them.</p><p>It’s a lot more complicated, of course. This hybrid man-machine superintelligence presupposes the existence and availability of large, properly curated, and accurate datasets (the machine learning community is awakening to how hard that is), organizations and societal normative frameworks that are ready to accept it (a whole new bag of scorpions right here), and a radically new mindset: systems thinking.</p><p>Equally as important is a giant technical step, which my team and I focus a lot of our efforts on, which is the ability to semantically and probabilistically represent all of this data across formats and ontologies.</p><p>As seen previously, representational learning is very powerful at creating high fidelity representations of whole complex systems. But that only goes so far, for several reasons: (1) it needs large and well curated datasets, (2) it needs lots of experts to tune, and (3) while it can represent very complex systems (with a large state space, like Go) it isn’t capable of scaling beyond bound, stable (they don’t evolve) and fairly coherent (from a data diversity standpoint) systems, where most systems we are trying to better understand (like the human brain, cancer, or human performance) are multi-dimensional, multi-ontological (they involve different types of data), and fast-evolving.</p><p>So while neural networks have shown impressive results in representing fairly complex systems (far above human abilities), they have not yet been able to cross into a higher level field of systems representation, which would make them very useful in helping humans understand more complex systems such as disease.</p><p>This step is where my team and I, both at ETC and Corto, have focused our efforts. We are working on extending hypergraph-like probabilistic knowledge representation applications to allow for more complex systems representations across large and heterogeneous datasets in a way that could make them more reliable, more autonomous, and more easy to compute.</p><p>After all -and we don’t hear this enough- the AI research community has for long known how to build supintelligence (probabilistic graph representation + information theoretic methods+ Bayesian/Reinforcement learning on complex adaptive systems), it just hasn’t been able to do so in a way that can be computed. 100% of the debate on AI today can be understood this way. It’s worth reminding that, a few years ago, AI researcher Marcus Hutter wrote a complete and definitive Artificial General Intelligence application made of just 50 lines of LISP code … but which, to be implemented, would need more computational power than exists in the whole Universe.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14acdebf9816" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vantage/ai-is-a-revolution-in-what-and-how-we-know-14acdebf9816">AI is a Revolution in What and How we KNOW</a> was originally published in <a href="https://medium.com/vantage">Vantage</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cognitive Hollywood: Data Reveals Box Office Economics in Turmoil]]></title>
            <link>https://medium.com/vantage/cognitive-hollywood-part-1-data-shows-box-office-economics-in-turmoil-411a4b22f858?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/411a4b22f858</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[box-office]]></category>
            <category><![CDATA[movies]]></category>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[entertainment]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Mon, 11 Sep 2017 07:29:11 GMT</pubDate>
            <atom:updated>2017-09-18T18:54:03.901Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*FN1OkAhvxkrIrNlKGNRhlw.jpeg" /><figcaption>Photo by Delphine Queme, via <a href="https://www.flickr.com/photos/delphinequeme/8079462758/in/photolist-diXmN3-r4xTeE-9xjAEz-kybwW9-e72gE2-dMDXX-6otTBy-nGk8c5-aM4GT2-57Uohe-pDT9H1-iLFFcz-X2ykFf-e95NNx-ksKd1p-aM4GwX-efLr9k-aivBW4-9suZg2-4nKsEH-8kVPk3-4VY7NN-2Hq1XJ-9d55NS-9d55Rw-rk145u-9d1ZGV-pRTuds-9d55XA-8wyLMq-99krp8-5x6WGh-dG7Wrw-ebYgE2-fEk9j9-WEXDsg-WgA3iD-UaSLok-Y9LUDL-U7rCwS-61PtAZ-5YiP2h-q7T1VK-j94QhE-9h2pLx-e1jeZN-nq1Zcn-5jSm8w-31fDRk-HuKWgy">Flickr</a></figcaption></figure><h3><strong>COGNITIVE HOLLYWOOD, Part 1: Data Shows Box Office Economics in Turmoil</strong></h3><p><em>NOTE: the opinion below is in my name only, and does not represent the views of ETC, the School of Cinematic Arts, or USC as a whole. Also, this analysis does not factor in </em>It<em>’s recent box office successes.</em></p><p>As part of my work managing the Data &amp; Analytics Project at USC’s <a href="http://www.etcenter.org/">Entertainment Technology Center</a>, I spend a lot of time analyzing data around the relationship between film stories and film audiences. Because of this, I believe the entertainment industry is entering a crisis, and much of this crisis has to do with the fact that audiences now demand increasing levels of innovation in how film and TV stories are told. They want novelty, and to satisfy them, Hollywood will need to profoundly transform how it does business. Filmmakers will need to make more data-driven decisions, and think more cognitively about their audiences.</p><h3><em>Box Office Hurricanes As Evidence of a Change in the Entertainment Climate</em></h3><p>Hollywood CFOs are closing out the Summer, and it’s a Lannister kind of party. The <a href="http://www.boxofficemojo.com/news/?id=4321">worst Labor Day Weekend performance in 17 years</a>; a bloodbath Summer box office in a near 20% retreat compared to 2016; bulletproof franchises underwater; even the mighty Dwayne “the Rock” Johnson <a href="https://www.rottentomatoes.com/m/baywatch_2017/">took a beating</a>. Summer 2017 was an Irwin Allen disaster.</p><p>It’s not that the industry is having a bad year overall — so far the median worldwide box office revenue for 2017 is around $66 million, up from $61 million in 2016, and there are still a few months left to go.</p><p>The problem goes deeper than that.</p><p>Audiences seem to be turning away from <em>some</em> movies. The hit rate hasn’t yet come down too much, but there is ample evidence that, as production budgets (and financial risks) go up, volatility in financial performance is also going up. What this means is that the Traditional Hollywood Formula of Hit-Making is coming to an end, and studio executives aren’t ready for it.</p><p>Nowhere was this more clear than in last week’s article in <em>The</em> <em>New York Times </em>by<em> </em>Brooks Barnes<em>, </em>“<a href="https://www.nytimes.com/2017/09/07/business/media/rotten-tomatoes-box-office.html?_r=0">Attacked by Rotten Tomatoes</a>”, which echoed a view often heard among senior creative executives in Hollywood that poor Rotten Tomatoes reviews are to blame for bad theatrical performance. One anonymous executive even went so far as to put the destruction of Rotten Tomatoes (which is owned by Fandango, itself co-owned by NBCUniversal and Warner Brothers) at the very top of his goals for next year.</p><h3><strong>Debunking the Rotten Tomatoes Myth, With Data</strong></h3><p>Barnes’s piece was pretty high on posturing and pretty low on actual data. So, being the resident data scientist at the ETC, I decided to see what the math said.</p><p>I collected box office return data through Box Office Mojo for all the 150 titles released in 2017 that grossed more than $1 million, plugged in Rotten Tomatoes Scores and Audience Scores for all titles, and looked at the correlation between scores and financial performance through both a basic <a href="https://en.wikipedia.org/wiki/Pearson_correlation_coefficient">Pearson Product-Moment Correlation Coefficient</a> (PMCC) analysis and some linear modeling to extract r-squares, a measurement of the strength of correlation.</p><p>PMCC measures the linear correlation between two variables X and Y. It has a value between +1 (100% positive correlation) and -1 (100% negative correlation, often called “inverse correlation”). The closer to 0 a PMCC score is, the less correlation exists between X and Y.</p><p>So has Rotten Tomatoes been spoiling box office performance? According to the data: Nope. The math shows pretty overwhelmingly that there was no correlation (positive or negative) in 2017 between Rotten Tomatoes Scores and box office returns.</p><p>The results showed a 12% PMCC correlation, and a .009 r-square, meaning there is likely no statistical relationship between the two variables. Even more surprising, the impact of Rotten Tomatoes scores on opening weekend box office seemed even lower: .08 PMCC score (a mere 8% correlation), and a -0.001 r-square.</p><p>That’s for all 2017 titles so far. What about the Summer titles, which the executives quoted by <em>The New York Times</em> complained about?</p><p>Again, nada. We found no meaningful correlation between Rotten Tomatoes scores and the total gross for summer movies running between May and Labor Day. We saw a .07 PMCC score (only 7% correlation), with a -.006 r-square, which is actually less correlation than for the entire 2017 slate.</p><p>There was even less correlation between Rotten Tomatoes Scores and opening weekend performance: -.03 PMCC score (meaning no relationship whatsoever), with an r-square of -.01.</p><p>It’s unclear how much of some creative executives’ opinions related in the<em> New York Times </em>article reflect actual belief that critics are hurting the top line, and how much they reflect the need to scapegoat Rotten Tomatoes.</p><blockquote>What is clear, looking at all film data since 2000, is that Rotten Tomatoes scores have never played a very big role in driving box office performance, either positively or negatively.</blockquote><p>Overall, Rotten Tomatoes scores for all movies grossing more than $2 million have been pretty stable since 2000. The median score was 51 during the 2000s and 53 during the 2010s so far, and it’s actually gone up quite significantly from 2015 (46.5) to 2017 (71).</p><p>For movies grossing more than $300 million worldwide, the median Rotten Tomatoes Score was 73 in 2013, then about 72 for 2014, 2015 and 2016. However, it has gone up to 77.5 in 2017, which means that — contrary to the grumble coming from Hollywood — critics are actually enjoying the slate this year.</p><h3>Does a Rotten Tide Raise All Boats?</h3><p>So Rotten Tomatoes Scores are going up, together with the median box office (adjusted for inflation), which has increased, from $26.6 million in 2015 to $66.1 million in 2017. But does the fact that Rotten Tomatoes scores and financial performance have both gone up suggest a relationship between the two variables?</p><p>Let’s — you guessed it — look at the data. Once again, using the PMCC as a measure of the strength of the correlation, and analyzing all films having grossed more than $2 million worldwide, we see the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WzI_am9yZ2nBNzBx1TjDDw.png" /></figure><p>This is actually a pretty strong correlation, which seems to be increasing, and — naturally — seems stronger with Domestic Box Office than Foreign Box Office performance (It’s unlikely that foreign markets, which are now much more profitable to Hollywood than the domestic market, pay much attention to Rotten Tomatoes scores).</p><p>What about movies grossing more than $300 million worldwide?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NH3dJjWCWIi7hO9ZeEOzoA.png" /></figure><p>The correlation seems equally strong. Clearly there’s something going on here: could the executives who complained to <em>The New York Times</em> be right in accusing Rotten Tomatoes of destroying their industry?</p><p>Well, not exactly.</p><p>There’s another, very interesting phenomenon at play, one that holds deep implications for how Hollywood makes movies:</p><blockquote><em>Audience scores and critic scores are increasingly correlated, me</em>aning that audiences are becoming expert at smelling a “bad” movie and staying away.</blockquote><p>Indeed, the data shows a very strong correlation between Rotten Tomatoes scores and audience scores (also reported on the Rotten Tomatoes website) for all films (that grossed more than $2 million worldwide) released since 2013:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zkuJ7qSk-g0zwnhjW738qw.png" /></figure><p>This chart sums up the correlations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2vGp2XfBAwhn6VmPWF8mJw.png" /></figure><p>Whoa. There’s virtually no difference between critics’ scores and audiences’ scores, and the more successful the film is at the box office, the smaller the difference. This makes it very difficult to isolate the true impact of Rotten Tomatoes scores on box office performance. When Hollywood executives complain about Rotten Tomatoes scores, they’re really complaining about their audience’s tastes, because it’s basically the same thing.</p><p>This suggests something that I’ve been thinking about for a few years, which is that it’s frankly a miracle that the entertainment industry would be making any money.</p><blockquote>Imagine running a company in a shrinking market where every product costs hundreds of millions of dollars to make, where every single one of your customers is an expert in the product, and millions of competitors are offering free, lower-quality but very accessible and compelling knockoffs.</blockquote><p>And that’s not even the worst of it. The worst part is that, as a studio executive, you know next to nothing about who these customers are, what they want, or why they like your product. That’s Hollywood.</p><p>The market for entertainment has been shrinking as the amount of time we spend on entertainment shrinks (at least until autonomous vehicles roll around). See below (thanks to UCSD’s Jim Short for the data):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ddhVR4BTkQDNiHgjSn0Xcw.png" /><figcaption>The left column represents hours spent per day</figcaption></figure><p>And no matter how old you are, you’re a film expert. How many hours of film and TV have you consumed since birth? Compare that to how many cars you have bought in your lifetime.</p><p>Meanwhile, the competition for this shrinking pie (digital and social media) is free, or near-free, and it’s exploding. Think about how many options you have to entertain yourself right now, between linear TV and film, OTT, social media, streaming music, the web…</p><p>Yes, the fact that the industry not only still exists, but is turning healthy profits overall, is a testament to the genius of those who run it (and the insanity of those who fund it).</p><h3><strong>RIP: the Good Old Days</strong></h3><p>Let’s make this as clear as possible: no Hollywood executive knows why you liked <em>War of the Planet of the Apes</em>, or why <em>Game of Thrones</em> is one of the greatest successes in the history of entertainment. Creative decisions across the entire industry are still largely made on the intuition of a handful (no more than 50) of executives who rely less on data and more on the cognitive models developed over decades of bone-crushing experience making movies.</p><p>These executives were bred over years and years of working the system from the inside. From the mailroom to the assistant’s cubicle to the VP of Production desk, they were bathed in the infinite human complexities of conceiving, creating and selling entertainment, slowly establishing their own cognitive scaffolding (i.e. biases) for what makes the ‘magic of story’ work. Their brains physically experienced what neuroscientists call “<a href="http://www.jneurosci.org/content/22/3/612">neurogenesis</a>”: a profound restructuring of the brain that can lead to a change in how one perceives the world.</p><p>By the time they would rise to the top of the studio system, executives had<em> become </em>film<em>. </em>They had<em> become </em>TV. Success was defined by the ability to bring together the basic element of success: creative talent. Bring it, pay it, and they will come.</p><blockquote>Nobody ever got promoted in Hollywood because they had better inside knowledge of what audiences wanted to see, simply because, until fairly recently, that didn’t matter. The industry had an unassailable stronghold on our leisure time, and so the factors of success were, for the most part, intrinsic (the product), rather than extrinsic (the market).</blockquote><p>This is why, as CGI opened the door to a much more immersive form of storytelling, and the landscape became more and more crowded with smaller and more nimble competitors (Miramax, Lionsgate, etc.), median production budgets surged: from $8 million (inflation adjusted) in the 1970s, to $22 million in the 2010s.</p><p>And now with even more competition from such newcomers as Netflix and Amazon (and soon Facebook) that median is rising fast: $25 million in 2016 and $30 million in 2017.</p><p>Lately the rationale has been: make more expensive films (heavy on extremely expensive visual effects and CGI), and you’ll attract larger audiences, especially in foreign markets.</p><p>Is that true? Let’s (say it together now!) look at the data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xmb1DG9VtzhxxyIQ18eNeA.png" /></figure><p>This is pretty compelling, and seems to validate the traditional Hollywood view of higher production budgets as a means of ensuring higher returns. Interestingly, we clearly see here that the introduction of CGI, which drove production budgets up, created value for audiences in the form of more spectacular and experiential content.</p><p>This is basic economics: if our time to entertain ourselves is flat (flat demand) but the number of options we have to entertain ourselves for the same amount of time is exploding (rising supply), then rational actors (us) will maximize the cognitive payoff (here, the dopamine rush that entertainment produces) by investing our scarce resources into the most cognitively rewarding (eg the most experiential) forms of entertainment possible: big, CGI-laden blockbusters and video games.</p><p>For more than 20 years, Hollywood’s bet on inflated production budgets was a good one, especially as they conquered foreign markets, here there seemed to be a particular appetite for these type of films. Until now.</p><p>After 20 years of big and loud CGI titles, there’s substantial evidence that audiences are tuning out. As CGI reaches the upper limits of photo-realistic characters and scenes, audiences are getting bored. Action-driven blockbusters high on visual effects but low on finely-crafted characters (which actually worked better to sell a film across cultures) are starting to lose money at the domestic box office with increasing frequency.</p><p>And it’s starting to show in the data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6o-psYU-3Hm0CvD-td9i-A.png" /></figure><p>Since 2013, the correlation between production budgets and domestic theatrical performance has started to come down, and it’s even looking soft on foreign markets.</p><p>What about for big box office earners?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jr10_mg5oYnwf2SWkwWVlw.png" /></figure><p>Here it seems the story is a bit more complex, but at a minimum, the correlation seems to soften, and the volatility in PMCC scores suggests that box office performance is driven by variables other than production budget, or that need to be combined with production budget to generate financial performance.</p><p>Let’s see if we can collect some insights by turning to another dimension in the relationship between audiences and film. For 20 years, rising production budgets led to the production of films that felt more and more enjoyable to audiences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ldWDtJCsaW1VAHNKhHpHiA.png" /></figure><p>Clearly the increase in CGI budgets didn’t impress the critics much. But while the impact was rather small on audiences, we see a substantial rise in how much audiences enjoyed movies during the Golden Age of CGI, from the 2000s until now.</p><p>Is this still true? Let’s turn to the data once again:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*agYTGJtl29ClTq7yKzl9PQ.png" /></figure><p>Let’s look at this another way: is there a relationship between critics and audience scores and production budgets? See below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ps137hjWjuEk7t2lD4FTAg.png" /></figure><p>Three things are apparent here:</p><ol><li><em>The New York Times</em> article notwithstanding, Hollywood is having a great year with critics and audience scores so far;</li><li>There’s definitely a lot of volatility in the scores, regardless of the production budget, meaning there are hidden variables (beyond budget) influencing scores;</li><li>It’s very clear that production budgets do have a very substantial impact on both critics and audience scores. Here again it seems Hollywood’s bet on CGI has largely paid off.</li></ol><p>However, the data above also suggests three more profound (and challenging) trends for Hollywood:</p><ol><li>Production budgets are less and less of a predictor of box office success, meaning that as financial exposure rises, so does financial risk. This is not good (for a long time it was the opposite), and is a substantial reason why Wall Street has been so tough on entertainment stocks lately;</li><li>Audiences are becoming extremely adept at predicting and judging the quality of a film;</li><li>Quality here is a very subtle measure of how much, and where, a film innovates in its story and character mechanics, and how much — and where — it doesn’t.</li></ol><p><strong>Part two of this post will go into why — and how — we’re entering a new era in Hollywood, the Golden Age of Stories, and why the industry as a whole has no choice but to think cognitively about storytelling.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=411a4b22f858" width="1" height="1" alt=""><hr><p><a href="https://medium.com/vantage/cognitive-hollywood-part-1-data-shows-box-office-economics-in-turmoil-411a4b22f858">Cognitive Hollywood: Data Reveals Box Office Economics in Turmoil</a> was originally published in <a href="https://medium.com/vantage">Vantage</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rotten Tomatoes Scores and Box Office: What the Data Says]]></title>
            <link>https://medium.com/@ybergquist/rotten-tomatoes-scores-and-box-office-what-the-data-says-5adfc49baa9f?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/5adfc49baa9f</guid>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[hollywood]]></category>
            <category><![CDATA[movies]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Fri, 08 Sep 2017 14:19:41 GMT</pubDate>
            <atom:updated>2017-09-08T15:38:51.897Z</atom:updated>
            <content:encoded><![CDATA[<p>Yesterday <em>The New York Times</em> joined the chorus of publications covering the horrendous Summer 2017 Box Office performance, echoing the long-standing Hollywood view that poor <a href="https://www.nytimes.com/2017/09/07/business/media/rotten-tomatoes-box-office.html?_r=0">Rotten Tomatoes scores have a dramatic negative impact on financial performance</a>. But a simple data analysis found no correlation.</p><p>The piece was pretty low on actual data, so I decided to head in that direction. I collected box office return data through Box Office Mojo for all the 150 titles released in 2017 that grossed more than $1 million, plugged in Rotten Tomatoes Scores and Audience Scores for all titles, and looked at correlation between scores and financial performance through both a basic Pearson Product-Moment Correlation Coefficient (PMCC) analysis as well as some linear modeling to extract r-squares (which measure the strength of the correlation).</p><p>The result? Nope. The math is pretty overwhelming in saying that, for the 150 top box office earners (all titles grossing more than $1 million at the box office) of 2017, there was no (positive or negative) correlation between Rotten Tomatoes Scores and box office returns.</p><p>Overall, the PMCC analysis reveals that, at least in 2017, there was a 65% correlation between RT scores and audience scores for all these titles, which means Rotten Tomatoes scores are a fairly good predictor of audience reviews. As you can see in the plot below, the relationship is fairly linear.</p><p>However, the data shows pretty clearly that there was little to no relationship between good or bad Rotten Tomatoes Scores and total gross: 12% PMCC correlation, and a .009 R-square (meaning there is no statistical relationship between the two variables).</p><p>Even more surprising, the impact of Rotten Tomatoes scores on opening weekend box office seemed even weaker: 0.8 PMCC score (only 8% correlation), and a -0.001 r-square.</p><p>That’s for all 2017 titles so far. What about the Summer titles?</p><p>Nope. We did not find any meaningful impact of Rotten Tomatoes scores on total gross for summer movies (May through Labor Day): .07 PMCC score (only 7% correlation), with a -.006 r-square.</p><p>There was even less correlation between Rotten Tomatoes Scores and opening weekend performance: -.03 PMCC score (3% correlation means no relationship whatsoever), with an r-square of -.01.</p><p>Obviously, this quick and dirty analysis doesn’t hold real scientific value, since it has a small sample, and only directionally accurate financial performance data (Box Office Mojo). Still, the myth — at least for 2017– seems debunked: audiences did not, in fact, get turned away by low Rotten Tomatoes Scores. A lot of people in the industry hang on to hopes that sliding theatrical financial performance might be less structural than circumstantial. Alas, the data shows that, at least for this Summer, something more profound — and more sinister- may be at play.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5adfc49baa9f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Insane World of Blockbuster Economics]]></title>
            <link>https://medium.com/@ybergquist/the-insane-world-of-blockbuster-economics-e02e57d85498?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/e02e57d85498</guid>
            <category><![CDATA[movies]]></category>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[entertainment]]></category>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[film]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Fri, 08 Sep 2017 07:29:01 GMT</pubDate>
            <atom:updated>2017-09-08T07:29:01.846Z</atom:updated>
            <content:encoded><![CDATA[<p>NOTE: This piece was published in March 2015.</p><p>Last week, Deadline.com released a small and exceptional dataset: the profit and loss statement for 2014’s Top 20 performers at the global box office, from the latest “Transformers” installment (which grossed more than $1 billion at the box office) to “Interstellar” (which clocked in at almost $673 million).</p><p>This is a rare document: these numbers are usually treated as state secrets in Hollywood, so this is the first time in recent memory that actual P&amp;L numbers have surfaced in the open.</p><p>And it’s easy to see why. The gospel of blockbuster economics is that inflating production and marketing budgets to churn out “tentpole” movies will breed higher financial returns for studios. But applying simple data analysis models on these numbers reveals a far different story, one of very strange economics and inefficiencies so massive they would never be tolerated in any other industry.</p><p>It’s important to set off with one very strong caveat: any data analysis is only as good as the data itself. “Garbage in, garbage out” as statisticians like to say: the most sophisticated math will never extract meaningful and reliable insight out of bad data.</p><p>Conversely, very simple statistical models can uncover powerful and complex patterns from good, clean data. Clearly, a P&amp;L statement is less prone to error than sampled behavioral or social media data, both of which have a lot of noise, so there should not be any bias (unless someone purposely fiddled with the numbers).</p><p>But this is single-source data, and while there is nothing to suggest that it is incorrect or biased, only senior studio executives can spot check Deadline’s dataset with their own numbers.</p><p>Also, it’s important to note this dataset only covers 20 movies, so the analysis below doesn’t generalize to all films produced in the US. It also only covers one year, so it’s unclear whether it describes a long-term trend, although there is no reason to believe studio executives would have made different budget decisions in 2014 than in 2013 or 2012.</p><p>But assuming this data is reliable, here are a few very surprising takeaways.</p><p><strong>1. Foreign markets are where Hollywood’s big production bucks get their biggest bang:</strong></p><p>On the surface, classic blockbuster economics seem to hold water: a simple linear regression analysis of the correlation between costs and overall gross revenues (including merchandise, TV rights, syndication, pay per view, etc.) for the 20 top box office performers of 2014 shows a very strong correlation (about 84%) between the two (see plot below).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*-rpHiQFv0zNCvpAy.jpeg" /></figure><p>Major $ in, major $ out.</p><p>The same seems to be true with net production costs: spending more on producing tentpole movies seems to strongly correlate (70%) to higher global box office returns. Clearly, inflating production budgets for big action movies aimed at large audiences are generating higher global income in Hollywood, as the plot below shows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*GJG2wigWQi44B2zd.jpeg" /></figure><p>Each dot in the graphs above represents one of the top 20 best performing movies at the box office in 2014. It seems like the dots are more or less aligned on a single line: statisticians call this a positive linear relationship; here between production costs and global box office income. Translation: in 2014, every extra dollar the studios spent on production costs for a blockbuster movie generated extra global box office income on average 70% of the time.</p><p>In finance, these are definitely great odds. But look a little closer and unpack the numbers a little further, say by splitting domestic and foreign box office income, and things get, well … very interesting.</p><p>Clearly, bigger production budgets generate a lot of money for Hollywood abroad, since the 2014 data shows a 75% correlation between production budgets and foreign box office income (see plot below), which represented a solid 63% of total box office revenue for that year’s top 20 performers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*3Fg28eHQaBGqcNFA.jpeg" /></figure><p>See how the dots once again align on a single invisible line? Again, this indicates a positive linear relationship, and it means that big production budgets make a lot of sense if Hollywood wants to make money abroad.</p><p>But what about domestic box office? Let’s look at the data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*qBpCM55dKHhVMZN7.jpeg" /></figure><p>The picture here is radically different: the data shows that, on average, net production costs have a very small, only 4.8%, correlation with domestic box office income, which represented 37% of the total in 2014. This means there’s virtually no relationship with how much money Hollywood spends on producing big blockbuster movies and how much money these films gross domestically. At this stage, this suggests that bigger budgets, bigger stars and bigger action didn’t generate bigger domestic audiences in 2014, but they definitely correlated to higher foreign income, where audiences seem to be more sensitive to the traditional blockbuster mix.</p><p><strong>2. There is no relationship between net production costs and net studio revenue:</strong></p><p>The data is very clear: plotting the correlation between net production costs and net revenues shows almost no relationship (4%) between how much Hollywood studios spent producing big blockbuster movies in 2014 and how much net profit they extracted from it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*mcQPwPep3bd7D9tu.jpeg" /></figure><p>See how the dots are all over the place? This means that, in 2014, blockbuster movies made a lot of money … but that that income was pocketed elsewhere in the Hollywood ecosystem, including with those who actually made the movies, especially the lead cast, on which shoulders these blockbusters usually rest (it’s particularly important for foreign markets). Movie stars with a lot of international pull, such as Tom Cruise or Channing Tatum, usually negotiate a substantial profit participation in their movies. Various profit participations (the director gets a cut, the writer gets a cut, etc.) are the reason why profit margins for big tentpole movies are so low in Hollywood (“Transformers” was the highest grossing movie of 2014, with more than a billion dollars, yet only made $250 million for its studio). They are also why it’s so difficult to secure outside financing for even big action movies these days. It seems the studios, which bear most of the financial risks in movie production and distribution, are getting a terrible deal. And that in Hollywood, unlike in other industries, the fruits of labor are juicier than those of capital. Marx and Ari Gold would be proud.</p><p><strong>3. Domestic marketing budgets for blockbusters are basically wasted:</strong></p><p>The second tenet of blockbuster economics is that inflating marketing budgets will generate more awareness in the public and thus higher box office grosses.</p><p>Let’s look at what the data says about this. The sample data doesn’t explicitly mention marketing costs, only “releasing costs”, but it’s safe to assume that the terms are interchangeable.</p><p>Here again, blockbuster economics seem strong for foreign markets: the data shows that, in 2014, higher foreign marketing budgets (which represented roughly 50% of the total marketing spend) strongly correlated (83%) to higher foreign box office returns (see below):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*lx65oQjQmcWefDju.jpeg" /></figure><p>Foreign marketing budgets were well spent, and performed well.</p><p>But here again the domestic picture seems extremely different, an this may be the bigger shocker in this data: the correlation between domestic marketing budgets (which represented 50% of the overall budgets for these movies in 2014 … around $1.3 billion) and domestic box office income (37% of total) for the top 20 performing films in 2014 was … very near zero. <br>0.3% to be precise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*ayqqwvOn8wYNeo-_.jpeg" /></figure><p>There could be a simple explanation for this. The function of marketing, especially in the film industry, is to build awareness of an upcoming release as well as intent to go see it in theaters. Marketing may be efficient for obscure or indie movies, which we become aware of a few weeks before their release (and therefore we build intent to see through increased awareness), but big franchise movies like the next “Transformers” or “Avengers” get hyped up months, if not years before their release.</p><p>The US moviegoing public is constantly up to date with the latest cast information, plot leaks, teasers, trailers, etc. This 365-day buzz machine, which usually happens through earned, not paid media, does a powerful job of building awareness and intent even without the studios spending a dime (nooo, thank YOU Nerdist !).</p><p>By the time the paid marketing campaign for a blockbuster movie rolls around, people are already aware of the movie, and they’ve already made up their mind. The box office is already locked up, and marketing budgets won’t tip the scale one way or another.</p><p>This is the irony of domestic film marketing: it’s spent mostly where it doesn’t matter, not where it would have the most impact: on independent movies or properties for which it would efficiently build awareness and intent.</p><p>One of the most commonly mentioned drivers behind expensive domestic marketing campaigns for films is that studios have to contractually take on the cost of raising their stars’ public profiles by spending large amounts on inefficient ad campaigns. A-list talent, in turn, guarantees higher returns on foreign markets, where moviestar appeal still gets people into the theaters, and where studios make most of their income.</p><p>But as we have seen, this income gets siphoned off by huge costs, including financial redistribution to the talent itself, which, as the data shows, seems to be the only one in Hollywood laughing all the way to the bank.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e02e57d85498" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Elon Musk’s Intellectual Laziness on AI]]></title>
            <link>https://medium.com/@ybergquist/elon-musks-intellectual-laziness-on-ai-928aebc5d5ef?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/928aebc5d5ef</guid>
            <category><![CDATA[god]]></category>
            <category><![CDATA[dogs]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[elon-musk]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Wed, 16 Aug 2017 04:59:05 GMT</pubDate>
            <atom:updated>2017-08-16T04:59:05.106Z</atom:updated>
            <content:encoded><![CDATA[<p>Elon is at it again. In a tweet on August 11, he reiterated his thoughts on artificial intelligence by claiming that “If you’re not concerned about AI safety, you should be. Vastly more risk than North Korea”. He also called for AI to be regulated, just like “everything (cars, planes, food, drugs, etc.) that’s a danger to the public”.</p><p>This was the latest exposition of his sensationalist thoughts on the topic, which if you are reading this you are familiar with by now.</p><p>He’s not the only one, of course: Bill Gates and Stephen Hawking have also raised concerns about unbridled AI. But none of them have been more vocal and unsubstantiated than Musk, who recently went as far as saying “we are summoning the demon” by pursuing super-intelligent AI.</p><p>Even his most detailed arguments are astonishingly crude.</p><p>For example, this one:</p><p>“the biggest threat about AI is not that it will develop a will of its own, but rather that it will follow the will of people who will establish its utility function, its optimization function. And that optimization function, if it is not well thought out, even if its intent is benign it could have quite a bad outcome. For example, if you were a hedge fund manager or a portfolio manager and you said: “all I want the AI to do is maximize the value of my portfolio, the AI could decide that the best way to do that would be to short consumer stocks, long defense stocks, and start a war. And that would obviously be quite bad”.<br> <br>… and this one:</p><p>“ If there’s a digital superintelligence, and its optimization or utility function is something that is detrimental to humanity, then it will have a very bad effect. It could be about getting rid of spam of something: the quickest way to get rid of spam is to get rid of humans”.</p><p>As Elon knows, it would be far more complex to “get rid of humans” than to maximize a stock portfolio or get rid of spam, and no real AI application would opt to pursue a high-complexity, low-probability scenario over a low-complexity, high-probability scenario. We have in fact gotten rid of spam (when was the last time a Viagra ad appeared in your inbox?), and there are many portfolio-maximizing AI applications out there.</p><p>Let me be clear: Elon Musk is by far the greatest entrepreneur, and one of the greatest minds, of our time. But I have yet to see evidence that Elon isn’t out of his depth on this topic, which makes his indictment of Mark Zuckerberg’s expertise a bit ironic.</p><p>First off, “AI” as a blanket term is a fuzzy concept. There are many levels of machine intelligence, depending on how broad the domain of their application is. We right now classify AI applications in two categories:</p><p>NARROW AI: 99.9% of today’s AI applications (including self-driving cars, DeepMind’s Alphago, and pretty much everything else you have seen or read about) are in the category of what we call “narrow AI” (or weak AI). These are AI applications which operate at or above human-level of intelligence but in a narrow domain. A self-driving car application can’t be ported over to drive a motorcycle, for example, let alone trade stocks. Even DeepMind’s Alphago was trained to play Go on a 19X19 Go broad, meaning that if the board had been bigger, smaller or even triangular, the applications would have failed.</p><p>ARTIFICIAL GENERAL INTELLIGENCE: this is what Elon (and all others) are referring to: AI applications that, contrary to narrow AI, are operating at or above human-level intelligence, and possess the ability to generalize across domains. So the same application or architecture can be used to drive a car, trade stocks, learn a language. Just like our human brains are one (extremely complex) paradigm of intelligence, capable of doing all of these things, often even concurrently.</p><p>So what Elon is warning about is AGI. And frankly, he’s not entirely wrong about the potential of AGI and its theoretical dangers. But he is definitely out of his depth, and perhaps even intellectually lazy about the reality, architecture, and inherent boundaries of Artificial General Intelligence (AGI). And his arguments betray interesting cognitive biases present throughout the tech industry: the tendency to oversimplify extraordinarily complex systems, which so often leads to startup failure.</p><p>He brushes aside the incredible difficulty of AGI. Don’t take my word for it, go straight to the only mathematical and engineering blueprint towards AGI: Ben Goertzel, Cassio Pennachin and Nil Geisweiller’s “Engineering General Intelligence” (<a href="https://www.amazon.com/Engineering-General-Intelligence-Part-Cognitive/dp/9462390266">https://www.amazon.com/Engineering-General-Intelligence-Part-Cognitive/dp/9462390266</a>).</p><p>This is perhaps due to the Silicon Valley view of AGI as an interconnected pyramid of neural nets, which is the track that its team at OpenAI seems to have doubled down on. Many experts, including the people I work with at Novamente and Corto (the very people who created the OpenCog project, and in fact invented the term AGI) are confident is not the way to achieve AGI. Neural nets are too limited to deal with uncertainty, and cannot in and of themselves create machines that can think, or even reason from sparse and ambiguous data.</p><p>He assumes AGI applications will have unbounded rationality, which is rather strange considering that any and all AI applications rely on input data, and even in the far-off future the amount of data available about reality, and the universe in general, will always be considerably limited. Even if AGI applications don’t have an intelligence or computation problem, they’ll definitely have a vision problem and a data problem, due to the fact that our observable reality and universe is only a fraction of the available universe. Without a unifying view of our reality, AI applications will represent it in a very diverse manner, which will most likely prevent them from collaborating “flexibly and scalably enough” (to borrow from the excellent Yuval Noah Harari) to fully dominate us.</p><p>He seems to conveniently step over the main scenario of human-AI relationship: the fact that the boundaries between biological and synthetic intelligence are getting blurrier and blurrier. How can synthetic intelligence turn against biological intelligence when there’s no difference between the two? After all, biological intelligence is most likely just a transition towards synthetic intelligence. Is that what he means? Does he mean we’ll lose certain foundational aspects of our human nature, such as emotions? This most likely will happen, but we will end up investigating deeper emotions more connected to the nature of the reality that we live in. The fact that Musk disregards this is puzzling since he just started a brain-computer interface company called Neurolink.</p><p>Most importantly, he seems to ignore that all intelligence — especially machine intelligence — operates within a normative framework (motivated by goals inside of larger goals inside of larger goals) that is not, and never will be, finite. Simply because nobody will ever have enough data to measure and validate the ultimate questions of “why?” and “what for?”. These questions have infinite answers. And machines can’t and won’t be able to see enough of the infinite to bind it or even invent it. Humans will have many opportunities to do so. As we develop truly thinking machines, we will have many opportunities to create this ultimate framework, of “why?” and “what for?”. Perhaps even by instituting ourselves as the recipients of an ultimate normative framework that can neither be measured nor observed, and thereby cannot be questioned by machines (many of us have been calling that framework “God”).</p><p>The main, most immediate, and most punishing consequence of developing human-level machine intelligence is that it will destroy millions of jobs. Let’s talk more about that, please.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=928aebc5d5ef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Elon Musk’s Intellectual Laziness on AI]]></title>
            <link>https://medium.com/@ybergquist/elon-musks-intellectual-laziness-on-ai-b4fda4f309a6?source=rss-1600f8fa2199------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4fda4f309a6</guid>
            <category><![CDATA[dogs]]></category>
            <category><![CDATA[elon-musk]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Yves Bergquist]]></dc:creator>
            <pubDate>Tue, 15 Aug 2017 23:20:55 GMT</pubDate>
            <atom:updated>2017-08-18T17:00:36.655Z</atom:updated>
            <content:encoded><![CDATA[<p>Elon is at it again. In a tweet on August 11, he reiterated his thoughts on artificial intelligence by claiming that “If you’re not concerned about AI safety, you should be. Vastly more risk than North Korea”. He also called for AI to be regulated, just like “everything (cars, planes, food, drugs, etc.) that’s a danger to the public”.</p><p>This was the latest exposition of his sensationalist thoughts on the topic, which if you are reading this you are familiar with by now.</p><p>He’s not the only one, of course: Bill Gates and Stephen Hawking have also raised concerns about unbridled AI. But none of them have been more vocal and unsubstantiated than Musk, who recently went as far as saying “we are summoning the demon” by pursuing super-intelligent AI.</p><p>Even his most detailed arguments are astonishingly crude.</p><p>For example, this one:</p><p>“the biggest threat about AI is not that it will develop a will of its own, but rather that it will follow the will of people who will establish its utility function, its optimization function. And that optimization function, if it is not well thought out, even if its intent is benign it could have quite a bad outcome. For example, if you were a hedge fund manager or a portfolio manager and you said: “all I want the AI to do is maximize the value of my portfolio, the AI could decide that the best way to do that would be to short consumer stocks, long defense stocks, and start a war. And that would obviously be quite bad”.<br> <br>… and this one:</p><p>“If there’s a digital superintelligence, and its optimization or utility function is something that is detrimental to humanity, then it will have a very bad effect. It could be about getting rid of spam of something: the quickest way to get rid of spam is to get rid of humans”.</p><p>As Elon knows, it would be far more complex to “get rid of humans” than to maximize a stock portfolio or get rid of spam, and no real AI application would opt to pursue a high-complexity, low-probability scenario over a low-complexity, high-probability scenario. We have in fact gotten rid of spam (when was the last time a Viagra ad appeared in your inbox?), and there are many portfolio-maximizing AI applications out there, including our very own <a href="https://www.wired.com/2016/01/the-rise-of-the-artificially-intelligent-hedge-fund/">Aidyia</a>.</p><p>Let me be clear: Elon Musk is by far the greatest entrepreneur, and one of the greatest minds, of our time. But I have yet to see evidence that he isn’t out of his depth on this topic, which makes his indictment of Mark Zuckerberg’s expertise a bit ironic.</p><p>First off, “AI” as a blanket term is a fuzzy concept. There are many levels of machine intelligence, depending on how broad the domain of their application is. We right now classify AI applications in two categories:</p><p>NARROW AI: 99.9% of today’s AI applications (including self-driving cars, DeepMind’s Alphago, and pretty much everything else you have seen or read about) are in the category of what we call “narrow AI” (or “weak AI”). These are AI applications which operate at or above human-level of intelligence but in a narrow domain. A self-driving car application can’t be ported over to drive a motorcycle, for example, let alone trade stocks. Even DeepMind’s Alphago was trained to play Go on a 19X19 Go board, meaning that if the board had been bigger, smaller or even triangular, Alphago would have needed to be re-architected and re-trained.</p><p>ARTIFICIAL GENERAL INTELLIGENCE: this is what Elon (and all others) are referring to: AI applications that, contrary to narrow AI, are operating at or above human-level intelligence, and possess the ability to generalize across domains. So the same application or architecture can be used to drive a car, trade stocks, learn a language. Just like our human brains are one (extremely complex) paradigm of intelligence, capable of doing all of these things, often even concurrently.</p><p>So what Elon is warning about is AGI. And frankly, he’s not entirely wrong about the potential of AGI and its theoretical dangers. But he is definitely out of his depth, and perhaps even intellectually lazy about the reality, architecture, and inherent boundaries of Artificial General Intelligence (AGI). His arguments also betray interesting cognitive biases present throughout the tech industry: the tendency to oversimplify extraordinarily complex systems, which so often leads to startup failure, but which tendency to overlook technical complexity, also leads to massively disruptive companies.</p><p>Musk brushes aside the incredible difficulty of AGI. Don’t take my word for it, go straight to the extremely detailed mathematical and engineering blueprint towards AGI: Ben Goertzel, Cassio Pennachin and Nil Geisweiller’s “Engineering General Intelligence” (<a href="https://www.amazon.com/Engineering-General-Intelligence-Part-Cognitive/dp/9462390266">https://www.amazon.com/Engineering-General-Intelligence-Part-Cognitive/dp/9462390266</a>). This is for now the only “technical Bible” towards building AGI, which this book actually introduced.</p><p>This is perhaps due to the Silicon Valley view of AGI as an interconnected pyramid of neural nets powered by reinforcement learning, which is the track that Elon’s team at OpenAI seems to have doubled down on. Many experts, including Ben and Cassio, who work with at Novamente and Corto (the very people who created the OpenCog project, and in fact invented the term AGI) are confident this neural nets-only approach is not the way to achieve AGI. Neural nets are very powerful to solve certain machine learning problems, but far too limited to deal with uncertainty. They cannot in and of themselves create machines that can think, or even reason, from sparse and ambiguous data, which is a core feature of AGI.</p><p>He assumes AGI applications will have unbounded rationality, which is rather strange considering that any and all AI applications rely on input data, and even in the far-off future the amount of data available about reality, and the universe in general, will always be considerably limited. Even if AGI applications don’t have an intelligence or computation problem, they’ll definitely have a vision problem and a data problem, due to the fact that our observable reality and universe is only a fraction of the available universe. Without a unifying view of our reality, AI applications will represent it in a very diverse manner, which will most likely prevent them from collaborating “flexibly and scalably enough” (to borrow from the excellent Yuval Hoah Harari) to fully dominate us.</p><p>He seems to conveniently step over the main scenario of human-AI relationship: the fact that the boundaries between biological and synthetic intelligence are getting blurrier and blurrier. How can synthetic intelligence turn against biological intelligence when there’s no difference between the two? After all, biological intelligence is most likely just a transition towards synthetic intelligence. Is that what he means? Does he mean we’ll lose certain foundational aspects of our human nature, such as emotions? This most likely will happen, but we will end up investigating deeper emotions more connected to the nature of the reality that we live in. The fact that Musk disregards this is puzzling since he just started a brain-computer interface company called Neurolink.</p><p>Most importantly, he seems to ignore that all intelligence — especially machine intelligence — operates within a normative framework (motivated by goals inside of larger goals inside of larger goals) that is not, and never will be, finite. Simply because nobody will ever have enough data to measure and validate the ultimate questions of “why?” and “what for?”. These questions have infinite answers. And machines can’t and won’t be able to see enough of the infinite to bind it or even invent it. Humans will have many opportunities to do so. As we develop truly thinking machines, we will have many opportunities to create this ultimate framework, of “why?” and “what for?”. Perhaps even by instituting ourselves as the recipients of an ultimate normative framework that can neither be measured nor observed, and thereby cannot be questioned by machines (many of us have been calling that framework “God”).</p><p>The main, most immediate, and most punishing consequence of developing human-level machine intelligence is that it will destroy millions of jobs. Let’s talk more about that, please.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4fda4f309a6" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>