<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ge Wang on Medium]]></title>
        <description><![CDATA[Stories by Ge Wang on Medium]]></description>
        <link>https://medium.com/@artfuldesign?source=rss-56fb5386cb66------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*12vl2Wr8VhP0uzl0xA_GlQ.png</url>
            <title>Stories by Ge Wang on Medium</title>
            <link>https://medium.com/@artfuldesign?source=rss-56fb5386cb66------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 05:47:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@artfuldesign/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[GenAI Art Is the Least Imaginative Use of AI Imaginable]]></title>
            <link>https://medium.com/artful-design/genai-art-is-the-least-imaginative-use-of-ai-imaginable-eb071671a9c9?source=rss-56fb5386cb66------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb071671a9c9</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[art]]></category>
            <category><![CDATA[music]]></category>
            <category><![CDATA[artful-design]]></category>
            <category><![CDATA[culture]]></category>
            <dc:creator><![CDATA[Ge Wang]]></dc:creator>
            <pubDate>Fri, 24 Jan 2025 20:11:42 GMT</pubDate>
            <atom:updated>2025-03-18T20:49:15.752Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="A man wearing a “Rage Against the Machine” t-shirt sits and looks to the side; a thought-bubble next to him reads, “What is the point?”" src="https://cdn-images-1.medium.com/max/1024/1*Hn8FUuigOArhZZjBcmmv7g.jpeg" /></figure><p>Recently, AI music company Suno’s CEO was <a href="https://www.404media.co/ceo-of-ai-music-company-says-people-dont-like-making-music/">quoted as saying “I think the majority of people don’t enjoy the majority of the time they spend making music”</a> in what sounds like yet another justification of Suno’s prompt-based (and very popular) generative AI music service. Like all good misleading statements, this one is partially based on truth: when we work at our favorite activities — cooking, gaming, hiking, music-making, writing (to name a few) — the process is often not easy or pleasant or smooth. Many musicians spend vastly more time in practice rooms than onstage; gamers willingly play punishingly difficult video games; hikers go up and down big mountains while dealing with blisters, mosquitos, and a host of inconveniences. All of these are arguably not “enjoyable” the majority of the time, and yet people still choose to undertake these activities. Why? Well, Suno quietly leaves out the part that the <em>process</em> (with its inherent challenges) is often what makes an activity ultimately meaningful and fulfilling. It’s like Suno telling hikers and mountaineers that clambering up a mountain is unpleasant business! Why not take a helicopter to the summit? Meanwhile, Suno is selling helicopter tickets.</p><p>Statements like Suno’s perfectly captures the prevailing public mindset about AI: that Artificial Intelligence is little more than a labor-saving optimization tool. This mindset tends to be good for #Capitalism, but betrays not only a lack of understanding of why people make music, but also a profound lack of imagination regarding how we could, or would want to live with our technologies in our lives. I, for one, would go as far as to say using generative AI for creative expression in this manner (“describe what you have in mind and AI will create it for you”) amounts to<strong><em> the least imaginative of use of AI that I can imagine.</em></strong> Such systems offer the promise of skipping the labor by bypassing the creative process and, yes, the difficulties, confusion, and frustration inherent in such endeavors, but…</p><p><em>What if the point of art is that we actually </em><strong>make</strong><em> it?</em></p><h4>“What’s Your Story?”</h4><p>I do not despise AI, per se, but I am forever wary of people who make and deploy technology in uncritical, culturally apathetic ways. I am pretty sure this makes me something of an Old Fart; or maybe I have always been that. But I do have some background in this topic. I am an <a href="https://ccrma.stanford.edu/~ge/">Associate Professor of Music and (by Courtesy) Computer Science</a> at Stanford University’s <a href="https://ccrma.stanford.edu/">Center for Computer Research in Music and Acoustics (CCRMA)</a>, where I teach “Music and AI” as a critical-making course, cross-listed in Music and Computer Science. I wrote the book <a href="https://artful.design/"><em>Artful Design: Technology in Search of the Sublime</em></a> as a photo-comic manifesto of why we ought to build tools playfully, artfully, in accordance not only with perceived needs, but also with the invisible values that underly the needs; it probes the question, “how do we want to live with our technologies?” I invent <a href="https://artful.design/ocarina/">things that nobody asked for and that solve no problems that quite exist</a>; I direct <a href="https://slork.stanford.edu/">laptop</a> and <a href="https://svork.stanford.edu/">VR orchestras</a>. Once upon a time, I co-founded a mobile music company called Smule. I am a Senior Fellow and a Faculty Associate Director of Stanford’s <a href="https://hai.stanford.edu/">Human-centered Artificial Intelligence</a> Institute (Stanford HAI, for short). I sometimes <a href="https://artful.design/hai/">write about AI and art</a>, and <a href="https://www.nytimes.com/2023/12/30/technology/openai-artist-alexander-reben.html">offer my perspectives</a>, such as they are. My wife and I <a href="https://www.youtube.com/watch?v=WVqDJ34HnYE">backpack in the mountains</a>. We are parents to a one-year old baby daughter; I think a lot about her growing up in a future I can barely imagine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j_P3qj4_bHQ6sS-Ot4rGTg.jpeg" /><figcaption>Making things that nobody asked for and that solve no problems that exists, Exhibit A: <a href="https://artful.design/ocarina/">Ocarina</a> for the iPhone.</figcaption></figure><p>Many years ago, I arrived at Princeton University to pursue my Ph.D. in Computer Science, with vague plans to build “the world’s most advanced algorithmic composition engine for music” (essentially GenAI for music, in today’s parlance). But before I even set foot on Princeton’s impeccably manicured campus, brimming with classic Gothic architecture, cherry blossom trees, and well-kept lawns — something, or rather someone, gave me pause on my aspirations.</p><p>On my drive up to New Jersey in that late summer of 2001, I stopped by a house party in Washington D.C. that featured a local cover band. The band was <em>tight</em>. After the performance, I went up to the guitarist and told him so.</p><p>“Thank you,” he said, “so, what’s <em>your</em> story?”</p><p>“I am going up to Princeton to start grad school in computer science.” I replied, “I want to build the world’s most advanced automated music composition machine.”</p><p>The guitarist studied me for a moment and asked, succinct and earnest, “What’s the point?”</p><p>It was a good question. No, it was a great question; maybe <em>the</em> question. It stopped me in my tracks and I did not give an answer. I did not have an answer. “I’ll let you know if I figure that out.” I grinned, as I pocketed the question for future rumination.</p><p>That brief exchange would domino into a “personal crisis of faith”, one that I would relish, for it would challenge my beliefs and continually beckon me to reflect on the role of technology for things that, like Music, <em>really</em> matter to us.</p><p>Over the next six years of graduate school, I worked on computer music with seriously playful characters like professors Perry Cook and Dan Truemen and fellow grad student Rebecca Fiebrink. I never did attempt to build “the world’s most advanced algorithmic composition engine” — because I never could answer the question “what is the point?” (without resorting to hand-wavy and frankly bullshit answers like “democratize music-making”). Instead, I built a <em>tool</em>, a <a href="https://chuck.stanford.edu/">domain-specific programming language for music</a>; thinking maybe I could create one more expressive tool with which people can make music and, if they wished, explore “what’s the point?” for themselves.</p><p>More than twenties years later, I am a tenured professor at a so-called “elite university” in the heart of Silicon Valley, ground zero of Big Tech. Meanwhile, my personal crisis of faith persists. I still ask, “what is the point?”. I still do not have an answer that satisfies, but I have come to profoundly appreciate the question, which I now help my students to ask in all that they do, especially in shaping technology. Ever the contrarian, I task my “Music and AI” students (many of them engineers) to build “useless things that are interesting to you” as a way to playfully work with technologies like AI, and to probe for both capacities and limits. They respond by making things like<em> auto-rizz.ck</em>:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FFYc6uXBUL6k%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DFYc6uXBUL6k&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FFYc6uXBUL6k%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/73aa6776a151873ce93cfdd801addd1c/href">https://medium.com/media/73aa6776a151873ce93cfdd801addd1c/href</a></iframe><p><em>auto-rizz.ck</em> uses AI — not “generative” but interactive. It detects “seductive” glances and plays cheesy saxophone music (yes, that is all it does). It’s whimsical. It’s absurd. It fit the assignment brilliantly. I always ask students to ask themselves <em>why</em> they chose to build what they did. Matt, the inventor of auto-rizz.ck, had this to say (excerpted from <a href="https://mattreed-75933.medium.com/how-i-got-a-girlfriend-from-1-chuck-program-ee59d76c136a">his reflection</a>):</p><p><em>“I really enjoyed this assignment. I think its easy to adopt this toxic capitalistic mindset that everything you do or make has to be productive or “for” something. But when we follow that dogma, we forget to make beautiful things just because they’re beautiful, do funny things just because they’re funny, or make stupid projects just because they’re stupid. There’s so much beauty in doing things just because. It was nice to take a pause on life and make something just for the sake of making it (and for a grade, but that’s more of an afterthought).” — Matt</em></p><p>I ask my students to look for interesting questions in everything. I tell them that the power of a good question is not in the answer, but in the question’s capacity to regeneratively invite more questions, and to construct lenses for examining the world — and ourselves. It’s like the question, asked by the ancient Greeks more than two millennia ago, “What is the <em>good life</em>?” We are still asking ourselves this today.</p><h4>A Labor-creating Machine?</h4><p>Ruminating on AI and art and the point of it all, I am reminded of a John Cage sentiment:</p><p><em>“What we need is a computer that isn’t labor-saving, but which increases the work for us to do.” — John Cage (from “Diary: Audience 1966”)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OkdrctEnGK3Ra1_zjUqL6A.jpeg" /><figcaption>John Cage: composer, jokester, challenger of our most established assumptions.</figcaption></figure><p>I first learned of this quote through CCRMA Ph.D. candidate Nick Shaheed’s Qualifying Exam. Nick’s interpretation of its meaning in a world nearly 60 years later:</p><p><em>“So, why would I want to increase the work I need to do? Well, because it’s fun and it’s work I want to do! For me (and many many other artists), doing the work and the work itself are largely the same thing. Speaking personally, my creative practice is about as far from the idealized depiction of Mozart shown in (the film) Amadeus, a fully formed concept is not being put onto the page as the first draft. Beginning with just the most basic seed of an idea, and then the process of trying it out, realizing it, tweaking things, all these bits of effort are the bricks being laid that form the actual piece itself. Without the work being done to make the music, the end result would be very different (and probably, worse).” — Nick Shaheed (from “CCRMA Quals Exam 2024”)</em><br> <br>I resonate with Nick’s position — that the process not only shapes the outcome invariably, but constitutes an unalienable part of what the outcome is. It is the idea that art is to be <em>made,</em> for it is during the making that we realize what we are truly trying to say, far beyond the initial inkling of a concept. It is the observation that process brings its own intrinsic rewards, not despite of but <em>because of</em> the inherent challenges in making things for ourselves. This is all to say, process is not an obstacle to be removed on the path of creative expression. It <em>is</em> the path.</p><p>Any process of creation is simultaneously a process of learning, beset by setbacks, confusion, and frustration. Yet, such frictions can, if one sticks with the process, ultimately give rise to something fulfilling: a deeper understanding of how a thing works and how to work with it, accompanied by the feeling that we have learned a bit more about ourselves and what we are capable of achieving. Skipping the process makes about as much sense as “playing” a difficult video game by asking someone else (robot or human) to take the controller and play it in your place, while you verbally offer general prompts about what to do and where to go from the couch. That would still be an experience, maybe even an enjoyable one, but it would be a fundamentally different experience from undertaking the process for yourself.</p><p>Reflecting on these ideas only makes me sadder that voices such as Suno’s (who received another venture capital funding round of $125M late last year) continue to dominate and shape the public imagination on AI and creative expression, drowning out other voices that would also have something to say about what we truly want from this technology, and how we might want to live with it moving forward.</p><p>The technology is new, but what GenAI music companies like Suno are doing is not. Like the recording industry before them (and without whom, ironically, there would be no training data for GenAI), companies like Suno commodify creative expression as part of an aesthetic economy based on passive consumption. Thus it is in Suno’s core interest to usher people away from active creation, and toward a system of frictionless convenience that strives to lower the effort of production — and the effort of imagination beyond vague concepts to type into prompts — to <em>zero</em>. And while no doubt prompting-AI-systems will be a new kind of “muscle” for us all to build, one has to ask: what other muscles will atrophy? There is always a price to pay; the danger of living in a world of frictionless convenience might well be cultural and individual stagnation.</p><p>This brings us to the most misleading part of GenAI companies like Suno: they purport to be tools for creating art, but by making every effort to dismiss and eradicate the process of learning to express ourselves, what they are actually creating is a new generation of <em>consumers. </em>While these endeavors might make a few individuals a lot of money in the meantime, it is unlikely to be good in the long-run for culture — or our soul.</p><p>I am not philosophically opposed to generative AI, even for artistic creation. For one, I don’t know what “AI making art” evens means at this point; I barely, if at all, understand “humans making art”. I am inclined to say, therefore, “there should be room for that, too”. At the same time, I am committed to preserving and protecting the room for humans to labor profusely, unenjoyably, illogically to creatively express themselves. The former may happen more and more as AI entrenches itself further in our lives; simultaneously, there is something at stake in the latter. “There should be room for that, too” should apply both ways.</p><p>I once gave a talk at Stanford HAI’s conference on AI and Creativity, where I put forth my own “crisis” question for others: “What Do We (Really) Want from Artificial Intelligence?”. The talk offers no answers, but a set of lenses to think about AI and creative expression. Give it a watch, if you would like, and figure out for yourself the questions that matter to you.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F1tqgwlfi8P8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D1tqgwlfi8P8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F1tqgwlfi8P8%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/abc109603a9036d344ec295ad5c60888/href">https://medium.com/media/abc109603a9036d344ec295ad5c60888/href</a></iframe><h4>What I (Really) Want</h4><p>In these days of profound uncertainty about the future, I look at my daughter, who just turned one, and every fiber of my person tells me this: as she grows, I would want her to learn to value and even embrace the difficulty, confusion, frustration that come with learning, and that reside inherent in the craft of creatively expressing oneself, whatever form that may take — and for her to come to know the sublime joy hidden in the process…with or without AI.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb071671a9c9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/artful-design/genai-art-is-the-least-imaginative-use-of-ai-imaginable-eb071671a9c9">GenAI Art Is the Least Imaginative Use of AI Imaginable</a> was originally published in <a href="https://medium.com/artful-design">Artful Design</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Humans in the Loop]]></title>
            <link>https://medium.com/artful-design/humans-in-the-loop-b83e3bffa65e?source=rss-56fb5386cb66------2</link>
            <guid isPermaLink="false">https://medium.com/p/b83e3bffa65e</guid>
            <category><![CDATA[craft]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ethics]]></category>
            <category><![CDATA[design]]></category>
            <dc:creator><![CDATA[Ge Wang]]></dc:creator>
            <pubDate>Tue, 22 Oct 2019 08:15:09 GMT</pubDate>
            <atom:updated>2019-10-28T21:08:33.414Z</atom:updated>
            <content:encoded><![CDATA[<h4>The Design of Interactive AI Systems</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VyQikQZ5lLDYCk0Dz0Lx0w.jpeg" /><figcaption>An exchange between a philosopher and a computer musician.</figcaption></figure><p>I was once asked by a colleague in the Philosophy Department here at Stanford if robot musicians will ever exist, to which I replied that they may — someday — but <em>only</em> if we first figure out what it means to have <em>robot</em> <em>philosophers</em>. The exchange was admittedly a bit tongue-in-cheek, but it revealed a blind-spot in the way we talk about the future of AI: in our tendency to ask <em>whether</em> or <em>when</em> a given task will be taken over by automation, it is easy to ignore the deeper issue of what such a takeover would <em>mean</em>.</p><p>This may an understandable oversight when we’re thinking about manufacturing, clerical work or even driving a car. We’re less concerned with <em>how</em> these tasks are accomplished, and more concerned with the outcome — generally measured in cost, speed and safety. But when we imagine “automating” a pursuit like music making, we’re forced to balance the <em>product</em> of work with something deeper — the meaning we derive from the <em>process </em>of doing it.</p><p>Of course, automation is only accelerating in the age of AI, and it’s natural to ask how far it will go. But I believe this vision of robot musicians and philosophers suggests we have a deeper question to answer first: which values should guide it?</p><h4>The Artful Design of Technology</h4><p><a href="https://ccrma.stanford.edu/~ge/">As an Associate Professor</a> at Stanford University’s <a href="https://ccrma.stanford.edu/">CCRMA</a> (Center for Computer Research in Music and Acoustics — pronounced “karma”), I’d like to think I have a unique perspective on a question like this. I design <a href="http://chuck.stanford.edu/">programming languages for music</a>, instruments and musical toys like <a href="https://www.gewang.com/ocarina/">Ocarina</a> for the iPhone, direct the <a href="https://slork.stanford.edu/">Stanford Laptop Orchestra</a>, and explore <a href="https://ccrma.stanford.edu/groups/vr/">VR/AR design</a> for music. I am a part of the <a href="https://hai.stanford.edu/">Stanford Human-centered AI</a> initiative, and my students and I research the design of systems that fundamentally infuse technology and human interaction, in the pursuit of new tools for musical and other forms of human creativity. Supported by a 2016 Guggenheim Fellowship, I wrote <a href="https://artful.design/"><em>Artful Design: Technology in Search of the Sublime</em></a>, a comic book about the shaping of technology — and how technology shapes us.</p><h4>Full Automation: AI as a Big Red Button</h4><p>Let’s start by exploring why <em>total</em> automation may not be the endgame for AI we tend to think it is.</p><p>As Michael Polanyi once noted about the tacit nature of human knowledge, we <em>know</em> more than we can <em>tell</em>. This is part of what gives AI, and deep learning in particular, its incredible allure; its ability to spot patterns in complex phenomena that defy rule-based descriptions means it can essentially understand them “for us”. Unfortunately, it also makes it tempting to think of AI as a “Big Red Button” — a technology that reliably delivers the right answers while hiding the process that leads to them.</p><p>Take the notion of artistic <em>style</em>, for instance. Most of us would agree it’s “a thing”, but would find it tricky to define, and even more difficult — if not impossible — to explicitly program. We might recognize painting “in the style” of Van Gogh, or writing “in the style” of Hemingway, but we’d likely describe such qualities in generalities, and find them nontrivial to emulate ourselves. These are cases of knowing more than we are able to tell — and where deep artificial neural networks really shine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mZupaTQoIZ6Jmi9LGKBBRA.jpeg" /><figcaption>Cool Cat * Van Gogh’s The Starry Night yields “Deep Cat”</figcaption></figure><p>For example, through style-transfer deep artificial neural networks, made possible by Leon Gatys et al.’s <a href="https://arxiv.org/pdf/1508.06576.pdf">original work on neural style transfer</a> and leading to projects like Google’s <a href="https://deepart.io/">Deep Art</a>, one can infuse the “style” of <em>Starry Night</em> onto photos — including, for instance, Cool Cat — creating a kind of hybrid image we might call “Deep Cat”. Technologically, this technique is remarkable, but it also highlights a critical difference between the notions of <em>style </em>and <em>artistic meaning</em>: while the former has been successfully harnessed in this example, the latter has been utterly side-stepped. For an even more extreme example, take a look at the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pTOX5QCH4yaecaGVXrtoBQ.jpeg" /><figcaption>Honeymoon in Hell? Style-transfer from Munch’s <strong>The Scream of Nature</strong>.</figcaption></figure><p>It’s nearly comical to view these images side-by-side. On the left is a snapshot of a blissful moment, worlds apart from the angst captured on the right: an existential meltdown, a reaction to the absurdity of modern life, or however you interpret Edvard Munch’s <em>The Scream of Nature</em>. But once combined, all sense of meaning goes out the window. Is our happy couple still on vacation? And are they really not alarmed by the river of lava behind them, or the fiery firmament above them?!</p><p>The style-infused vacation photo is pleasant to look at, but its meaning is underwhelming, casual, kitschy. <em>The Scream</em>, in contrast, is a work of Art, inviting us to reflect, to feel. One image simply says “oh, I’ve been there,’’ while the other <em>screams</em>, silently, “I’ve <em>been</em> there.” It’s the difference between <em>style</em> — something AI is developing an impressive grasp of — and <em>meaning</em> — something even we humans still struggle with…</p><p>This brings us to the three primary shortcomings that define a “Big Red Button” (BRB) system.</p><p>First, to improve the BRB system’s output, you’d either have to start over with different input — or start <em>waaay</em> over and modify the system itself.</p><p>Second, BRB systems offer little user control. What if you wanted to specify <em>how much</em> of <em>Starry Night</em>’s style to impart unto your cat photo? Or mix and match styles in various balances? This kind of flexibility underlies our notions of experimentation, craft, and creativity — and are fundamental to the very concept of a <em>tool</em>.</p><p>Finally, and perhaps most importantly, there’s the fact that we don’t just value the <em>product</em> of our work; we often value the <em>process</em>. For example, while we enjoy eating ready-made dishes, we also enjoy the act of cooking for its intrinsic <em>experience</em> — taking raw ingredients and shaping them into food. Or take music; we may have access to more of it than ever before in the form of recordings — many of which represent the very pinnacle of the art — but we haven’t stopped singing, playing and composing for ourselves. From the earliest days of radio and recording to digital music, and now streaming, there’s remains—through various technological innovations—an intrinsic joy to the activity of making music.</p><p>It’s clear there is something worth preserving in many of the things we do in life, which is why automation can’t be reduced to a simple binary between “manual” and “automatic.” Instead, it’s about searching for the right balance between aspects that we would find <em>useful</em> to automate, versus tasks in which it might remain <em>meaningful</em> for us to participate. As easy as it can be to embrace the extremes — to rush into automating <em>everything</em> or to insist on automating <em>nothing</em> — ideal solutions often exist somewhere in between, as a <em>duality </em>between automation and human interaction, between autonomous technology and the tools we wield.</p><h4>A Different Approach: Designing with a Human in the Loop</h4><p>What if, instead of thinking of automation as the <em>removal</em> of human involvement from a task, we imagined it as the <em>selective inclusion</em> of human participation? The result would be a process that harnesses the efficiency of intelligent automation while remaining amenable to human feedback, all while retaining a greater sense of meaning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0sh42Fnwf6XLN-MBnqQTVw.jpeg" /><figcaption>General blueprint for a human-in-the-loop interactive AI system.</figcaption></figure><p>Essentially, the human-in-the-loop approach reframes an automation problem as a Human-Computer Interaction (HCI) design problem. In turn, we’ve broadened the question of “how do we build a smarter system?” to “how do we incorporate useful, meaningful human interaction <em>into</em> the system?”</p><p>This kind of design is at the center of research in fields like Interactive Machine Learning, in which intelligent systems are designed to augment or enhance the human, serving as a tool to be wielded through human interaction. We can see this type of research expressed in the <a href="http://portfolio.decontextualize.com/">works of Alison Parrish</a>, poetical engineer, as well as at the Stanford HAI launch event in March 2019, which highlighted <a href="https://www.youtube.com/watch?v=E7A1VwZ1O20">collaborative social systems</a> (Michael Bernstein), <a href="https://www.youtube.com/watch?v=nx5yNH63GVk">computers that learn to help</a> (Emma Brunskill), <a href="https://www.youtube.com/watch?v=B94X6LwHYxI">ambient intelligence in AI-assisted hospitals</a> (Serena Yeung), and <a href="http://interaction design for autonomy">interaction design for autonomy</a> (Dorsa Sadigh).</p><h4>Examples of Human-in-Loop Design</h4><p>A student in my Thinking Matters course at Stanford, <a href="https://artful.design/think66/">Design that Understands Us</a>, proposed a tool that uses AI to translate legal documents into more “human readable” forms. But there was a twist: the design included a single slider that allows the user to control the “level of jargon-ness”. Setting this slider to one extreme would lead the system to essentially regurgitate the original document, unchanged, and — at the other extreme — a vastly more colloquial version of it. Simple as it may seem, it’s this <em>gradient</em> of choices between the extremes that make the proposed system truly useful as a <em>tool</em> — with which one can experiment, learn (e.g., by seeing the same document with different extents of legal “jargon”), and tailor to one’s preference and need. A simple, well-articulated slider, in this case, makes the difference between a Big Red Button and a human-in-the-loop tool. It represents an entirely different ethos about what the system is meant to be. (More whimsically, one can imagine a similar tool that works in reverse: e.g., taking normal writing and making it look like a legal document — or, taking a legal document and further “jargon-izing” it. Not practical, but potentially fun and illuminating!)</p><p>In my own field — nestled somewhere between computer science, design, and music — there are researchers whose work embody this human-in-the-loop ethos as well. Among the most prominent of these is <a href="https://www.doc.gold.ac.uk/~mas01rf/homepage/">Dr. Rebecca Fiebrink</a>, a professor at Creative Computing Institute at University of the Arts London, Goldsmiths, University of London, and world expert at the intersections of AI, HCI, and Music. Her Ph.D. Thesis (Princeton, 2011), <a href="http://www.cs.princeton.edu/~fiebrink/drop/finalthesis/RebeccaFiebrinkThesisPQ.pdf"><em>Real-time Human Interaction with Supervised Learning Algorithms for Music Composition and Performance</em></a>, is a pioneering work that brought together the aforementioned disciplines, articulating a new way to think about designing systems with AI and human in the loop, with far-reaching implications well beyond music. She is the author of <a href="http://www.wekinator.org/">Wekinator</a>: software for real-time, interactive machine learning. One can adapt Wekinator to iteratively, efficiently, and incrementally train tools by example. Humans can continually refine the system by “showing” the system new examples of control mappings for musical instruments, video games, or any other task that has input, interaction, and output. It reframes machine learning tasks as HCI tasks — something we might even call “Human-AI-Interaction.”</p><p>This shift in thinking may seem trivial, but it can be incredibly powerful. For example, imagine the task of separating a musical recording into its individual tracks, like vocals, guitars, bass and drums. This has all sorts of applications, from music production to forensics to karaoke. It’s also <em>extremely</em> difficult, with professional tools still delivering imperfect results despite decades of attempts at designing the perfect algorithm.</p><p>But what if the goal of a “perfect algorithm” is the wrong way to think about it? That’s the question that inspired Nick Bryan, a former Ph.D. student of mine, to try a new approach with his <a href="http://isse.sourceforge.net/">thesis project</a>. He recognized that humans are still the reigning champions when it comes to picking out the thump of a bass note, the crack of a snare drum or the familiar timbre of a vocalist — even in the midst of a fully mixed recording. So he designed an interactive system that combined machine learning, audio signal processing, and human interaction. It all came together through an interface inviting a human user to draw annotations directly onto a visualization of a waveform, roughly pointing out <em>where</em>, in both time and a range of frequencies from bass to treble, a given instrument or vocal is most prominently heard. These annotations aren’t meant to be precise, of course; they’re simply a hint that tells the underlying algorithm where to focus its efforts. Moreover, this is an iterative process where a user can listen to the intermediate result at each step and further refine the separation by providing additional examples by drawing. It adds genuine structure and context to an otherwise meaningless stream of data — and helped Nick’s <em>selectively</em> automated tool outperformed its fully automated rivals. Once again, even a little human involvement, in the right place, can go a long, long way.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FRd3prIkO5bg%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DRd3prIkO5bg&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FRd3prIkO5bg%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4f5d570ad79ca5314f9427e8f624cde/href">https://medium.com/media/e4f5d570ad79ca5314f9427e8f624cde/href</a></iframe><h4>Benefits of Human-in-the-Loop</h4><p>With all this talk of “value” and “meaning”, however, it’s important to remember how practical the benefits of human-in-the-loop design can be.</p><p><strong>First, it means significant gains in transparency.</strong> Each step that incorporates human interaction demands the system be designed to be understood by humans to take the next action, and that there be some human agency in determining the critical steps. Ultimately human and AI undertake the task <em>alongside</em> one another, making it harder and harder for the process to remain hidden.</p><p><strong>Next, they incorporate human judgment in effective ways.</strong> At the end of the day, AI systems are built to help humans. The value of such systems lies not solely in efficiency or correctness, but also in human preference and agency. Humans-in-the-loop system puts humans in the decision loop.</p><p><strong>They also shift pressure away from building “perfect” algorithms.</strong> By incorporating human intelligence, judgement, and interaction into the loop, the automated aspects of the system is exempted from “getting everything right all at once” (as in a Big Red Button scenario). Because the system is built around human guidance, the system only needs to make meaningful progress to the next interaction point. In fact, it may be beneficial to do less rather than more at each step. As Rebecca Fiebrink demonstrates, Wekinator “only” uses a single-layer neural network — that’s all it really needs to be useful; as humans do the rest. Could it go deeper? Of course, but it doesn’t have to.</p><p><strong>Finally, they often enable <em>more</em> powerful systems, not less.</strong> Human-in-the-loop design strategies can often improve the performance of the system compared to fully automated <em>and </em>fully manual systems. This aligns with the notion that a hybrid system can do no worse than fully automated systems — i.e., as the design allows, the human can defer to the rest of the system whenever they may choose to do — but the right kind of human interaction can render the system fundamentally better at what it is built to do. In other words, this is <em>functional excellence</em>, achieved through finding an ideal balance for a given situation.</p><p>If there is truth to this, then a critical first ingredient in designing any human-centered AI systems is having the awareness to ask: in what ways can the system I am designing incorporate human curation into the loop? At what junctures can human judgement and preferences improve the system in its effectiveness — and the experience of using it? What might an interaction model or “user interface” look like? How does the AI model support such an interaction? What do I need to tweak to make it work?</p><p>There is no one-size-fits-all method to designing human-in-loop AI systems. But perhaps we can articulate a few general design principles as “things to think with” when designing a system.</p><ol><li><strong>Value human agency.</strong> Design AI systems that harness human preference, taste, and judgement.</li><li><strong>Granularity is a virtue.</strong> A Big Red Button is an all-or-nothing affair that offers little granularity of control. Break up the task to incorporate human interaction.</li><li><strong>Interfaces should extend us.</strong> Build <em>tools</em> — things we can learn to use — instead of <em>oracles</em>, which give us the right answers but withhold an explanation.</li></ol><h4>How Do We Want to Live with AI?</h4><p>The broader question, at the end of the day, is perhaps not “how do we design more intelligent machines”, but rather, “how do we want to <em>live</em> with those machines”? How do we align our AI systems to the kind of world we would want to live in? Beyond simple automation, how might we use AI to augment human capabilities, incorporating human interaction, preference, and judgement, in order to design more useful and <em>meaningful</em> AI systems?</p><p>It may still be a while before we have to decide where a robot songwriter or algorithmic playwright fits into our lives. But the question we’ll ask then is the same we should ask now — for creative and practical tasks alike: is the product really all that matters, or is there something special in the process too? I believe there is—and that there is a <em>balance</em> between automation and human interaction to be discovered in every situation—which is why I’d like to end by presenting a fourth design principle for the future of automation, a principle from <a href="https://artful.design/"><em>Artful Design: Technology in Search of the Sublime</em></a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xNwq4igF2k4B_HnonP1_zg.jpeg" /><figcaption>There is a unique balance to be discovered—and designed—in every new situation.</figcaption></figure><h4>Acknowledgements</h4><p>Special thanks to Alex Varanese, Patricia Alessandrini, Nick Bryan, and Rebecca Fiebrink for their ideas, suggestions, and edits throughout the writing of this article. Also thanks to Fei-Fei Li and John Etchemendy for inviting me to writing an article for the <a href="https://hai.stanford.edu/news/humans-loop-design-interactive-ai-systems">Stanford Human-Centered AI</a> initiative.</p><h4>Additional Readings</h4><p>Amershi, S., M. Cakmak, W. B. Knox, T. Kulesza. 2014. “<a href="https://www.aaai.org/ojs/index.php/aimagazine/article/view/2513">Power to the People: The Role of Humans in Interactive Machine Learning</a>.” A.I. Magazine(35):4. pp.105–120.</p><p>Bryan, N. J., G. J. Mysore, and G. Wang. 2014. “<a href="https://ccrma.stanford.edu/~gautham/Site/Publications_files/bryan-chi2014.pdf">ISSE: An Interactive Source<br>Separation Editor.</a>” <em>ACM CHI Conference on Human Factors in Computing Systems</em>. Toronto.</p><p>Bryan, N. J. 2014. <a href="https://ccrma.stanford.edu/~njb/research/phd-thesis.pdf"><em>Interactive Sound Source Separation</em></a>. Ph.D. Thesis. Stanford University.</p><p>Fiebrink, R. 2011. <a href="http://www.cs.princeton.edu/~fiebrink/drop/finalthesis/RebeccaFiebrinkThesisPQ.pdf"><em>Real-time Human Interactions with Supervised Learning Algorithms for Music Composition and Performance</em></a>. Ph.D. Thesis. Princeton University.</p><p>Gatys, L.A., A. S. Ecker, M. Bethge. 2015. “<a href="https://arxiv.org/pdf/1508.06576.pdf">A Neural Algorithm of Artistic Style.</a>”<em> Nature Communications</em>.</p><p>Parrish, A. 2019. <a href="http://portfolio.decontextualize.com/"><em>Portfolio</em></a>. Website. Accessed March 2019.</p><p>Wang, G. 2018. <a href="https://artful.design/"><em>Artful Design: Technology in Search of the Sublime (A MusiComic Manifesto)</em></a>. Stanford University Press.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b83e3bffa65e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/artful-design/humans-in-the-loop-b83e3bffa65e">Humans in the Loop</a> was originally published in <a href="https://medium.com/artful-design">Artful Design</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Music as Social Fabric]]></title>
            <link>https://medium.com/artful-design/music-as-social-fabric-14dd17275ce1?source=rss-56fb5386cb66------2</link>
            <guid isPermaLink="false">https://medium.com/p/14dd17275ce1</guid>
            <category><![CDATA[music]]></category>
            <category><![CDATA[craft]]></category>
            <category><![CDATA[social-network]]></category>
            <category><![CDATA[technology-and-design]]></category>
            <category><![CDATA[ethics]]></category>
            <dc:creator><![CDATA[Ge Wang]]></dc:creator>
            <pubDate>Thu, 03 Jan 2019 21:41:27 GMT</pubDate>
            <atom:updated>2019-01-04T03:37:00.392Z</atom:updated>
            <content:encoded><![CDATA[<h4>Designing Social Tools that Make Us More Expressive</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wrGXdsljipwyr8OR4emvPg.jpeg" /><figcaption>People gather at a park in Beijing to sing together. Music-making is something of an universal social medium— it has been a part of every society in human history. (image: Laurel F. | Creative Commons)</figcaption></figure><blockquote>“People who make music together cannot be enemies. At least while the music lasts.” — Paul Hindemith</blockquote><p>Let’s face it. The “social” part of tech-mediated social media still sucks.</p><p>Developers tend to reduce the functions of social tools to news feeds, photos, video sharing, likes, and comments. It’s much harder to consider the lasting impact of the activities, or to design for the sheer <em>experience</em> they foster. A 2017 study (from Mediakix) found the average person, at the current rate, will spend <a href="https://www.adweek.com/digital/mediakix-time-spent-social-media-infographic/">more than five years of their lives</a> on social media. That is literally more time than it takes to go to college—more time than you will spend <em>eating </em>in your E.N.T.I.R.E. L.I.F.E.!</p><p>This sheer amount of time—of life!—demands that we hold the design of social tools to a higher standard. I mean, shouldn’t such tools elevate us? Make us really <em>feel</em>—more expressive, truthful, and appreciative of our human connections? Right?!</p><p>But how do we design more <em>virtuous</em> social tools, tools that make us better?</p><p>Perhaps we can take a cue from music-making<em> </em>— the kind of activity that does not require you to know who you are playing music with, but will leave you feeling expressive, connected, and, well, <em>social.</em> If our social tools were made more like instruments and contexts for music-making, what might they look like? What would we <em>value</em> in such tools?</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F64483560%3Fapp_id%3D122963&amp;dntp=1&amp;url=https%3A%2F%2Fvimeo.com%2F64483560&amp;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F435147424_1280.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1280" height="720" frameborder="0" scrolling="no"><a href="https://medium.com/media/b2766b75f4fc367588b392bec852dd3a/href">https://medium.com/media/b2766b75f4fc367588b392bec852dd3a/href</a></iframe><ol><li><strong>Identity is not that important:</strong> Music goes beyond names, age, gender, culture, and location to create meaningful, authentic connection between people. For example, in <a href="https://www.gewang.com/ocarina/"><em>Ocarina</em></a> (designed 10 years ago, which is like a bazillion in app-years) had this globe feature that let’s you listen to strangers blowing into their phones from around the world (see video above). This is an <em>anonymous</em> social medium that is not about who is playing and how good they are, but that there are people out there making music, for the fun of it.</li><li><strong>Participation is a value (dammit!):</strong> With social media there’s a lot of looking at and scrolling through but not a lot of <em>doing</em> things together with other people (I have a separate diatribe about “likes” and comments). Social activities like music create an opportunity for meaningful shared experiences. It puts you in the zone, and gets you out of your funk (sometimes by employing funk). As composer Paul Hindemith once said, “People who make music together, cannot be enemies. At least while the music lasts.”</li><li><strong>Expression is a virtue: </strong>Social media has interactive components but wasn’t designed to go much farther than looking at and liking things. You can’t quite express yourself fully with five emojis — if we could, that would actually be super sad :(. To make something is a unique kind of joy that can only come from within. Yes, you can make a post, but when was the last time a post moved you the same way as listening to a favorite song? Or more broadly, why don’t our social tools make us <em>feel</em> more? Or feel more like ourselves?</li></ol><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F59391815%3Fapp_id%3D122963&amp;dntp=1&amp;url=https%3A%2F%2Fvimeo.com%2F59391815&amp;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F412729569_1280.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1920" height="1080" frameborder="0" scrolling="no"><a href="https://medium.com/media/3bb268fd3e02f27d0fb19e0854120bac/href">https://medium.com/media/3bb268fd3e02f27d0fb19e0854120bac/href</a></iframe><p>It is time the tools we use reflect our desire to express and to connect.</p><p>Here endeth our first rant. For more, follow me or the Artful Design series on Medium, and check out my book, <a href="https://artful.design/"><em>Artful Design: Technology in Search of the Sublime</em></a>. See you down the rabbit hole!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14dd17275ce1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/artful-design/music-as-social-fabric-14dd17275ce1">Music as Social Fabric</a> was originally published in <a href="https://medium.com/artful-design">Artful Design</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>