<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Taka Shimokobe on Medium]]></title>
        <description><![CDATA[Stories by Taka Shimokobe on Medium]]></description>
        <link>https://medium.com/@takashimokobe?source=rss-b71c2843f2b4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*M-IzsgEY7wjcw-GB.</url>
            <title>Stories by Taka Shimokobe on Medium</title>
            <link>https://medium.com/@takashimokobe?source=rss-b71c2843f2b4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 15:37:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@takashimokobe/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Evolution of Conversation Design: Chatbots, Voice User Interfaces, and Google Duplex]]></title>
            <link>https://medium.com/@takashimokobe/the-evolution-of-conversation-design-chatbots-voice-user-interfaces-and-google-duplex-815d7bee2233?source=rss-b71c2843f2b4------2</link>
            <guid isPermaLink="false">https://medium.com/p/815d7bee2233</guid>
            <dc:creator><![CDATA[Taka Shimokobe]]></dc:creator>
            <pubDate>Tue, 12 Mar 2019 03:30:01 GMT</pubDate>
            <atom:updated>2019-03-12T03:30:01.121Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IP4j9i9uPaRDFlqOpBza9g.jpeg" /><figcaption>Image Courtesy of Unsplash</figcaption></figure><h3><strong>Overview</strong></h3><p>Conversation design is present in our everyday devices. From our phones, laptops, and even our headphones, software such as virtual home assistants and chatbots are becoming ever present in the tech landscape of the 21st century. Recently, Google held a conference where they showcased Google Duplex: a tool that calls businesses for you to reserve tables at restaurants, book appointments, and order takeout — a bot that sounds remarkably like a human being. In this article I’ll highlight the evolution of conversation design in both chatbots and voice user interfaces, discuss Google Duplex and its implications on Human Computer Interaction, and talk about the future of conversation design in our lives.</p><h3>What is Conversation Design?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/0*E10WypaGi5-_q3WP" /><figcaption>Image Courtesy of ELearningIndustry.com</figcaption></figure><p>Conversation Design, as defined by Google’s design guidelines:</p><blockquote><strong>“Conversation design is a design language based on human conversation (similar to how material design is a design language based on pen and paper).” — Google Design Guidelines</strong></blockquote><p>The more an interface leverages human conversation in its design, the less users have to be taught to learn how to use the interface. Because human users likely have a mental model of how conversation and language works, conversation design employs this mental model as a framework to base the interaction off of. Conversation design bridges the <strong>gulf of evaluation </strong>(the distance from the user to the physical system) and the <strong>gulf of execution </strong>(the distance from the physical system to the user) by reducing the effort it takes from the user to communicate with a machine, whether that be through text or voice.</p><p>Conversation design is a synthesis of several design disciplines, including voice user interface design, interaction design, visual design, motion design, audio design, and UX writing (<a href="https://designguidelines.withgoogle.com/conversation/conversation-design/what-is-conversation-design.html">Google Design Guidelines)</a>. As conversation design is such a huge umbrella topic that shows up so many disciplines, I won’t be able to get too deep into the intricacies of conversation design, but I hope to be able to touch on how conversation design is considered in software through three examples: chatbots, voice user interfaces, and in Google Duplex.</p><p>Our everyday devices leverage conversation design in order to communicate how the interface allows us to interact with it through the medium of human language. From voice assistants that help us carry out hands free text messaging to chatbots that help streamline our online shopping experiences, conversation design is everywhere in the 21st century. Good conversation design is similar to what good user experience design looks like: <strong><em>it’s powerful, but almost unnoticeable</em></strong>.</p><p>As conversation design is based off of human language, it is inherently <strong>multimodal</strong>. Much like how humans have to consider a number of things while conversing with other human beings, there are a great number of things that conversation designers consider when designing experiences based off of human communication. Different use cases call for different considerations when designing conversation interactions. For example, it probably wouldn’t be the best design choice to design a voice user interface that pings the user when relevant data changes, like a chatbot would do. Conversation design is necessarily <strong>contextual</strong> because of this. Conversation design must consider <strong>how </strong>users currently hold conversations with humans, <strong>why </strong>the user is using conversation as an input source for the device, before getting to <strong>what </strong>the conversation flow will (or in this case, sound) like.</p><h3>A Brief History of Conversation Design</h3><p>Here is a brief history of both conversation design in chatbots and voice user interfaces.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/737/0*sEISPZ4aq0-gol-Q" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/288/0*uWJCmb0ieZzTc3Gz" /><figcaption>Image Courtesy of voicebot.ai and Futurism</figcaption></figure><p>While the list is large and rather comprehensive, I’ll highlight a couple of breakthroughs that I believe redefined conversation design.</p><p><strong>Audrey by Bell Labs</strong></p><p>Audrey was the first speech recognition software, developed by Bell Labs in 1952. Audrey could only recognize numbers between 0–9 (<a href="https://astaspeaks.wordpress.com/2014/10/13/audrey-the-first-speech-recognition-system/">Audrey: The First Speech Recognition System</a>, Asta Speaks, October 2014). While this was revolutionary in that it was really the first voice user interface, the capabilities of Audrey were extremely limited — when testing, Audrey had to have a history with the speaker in order to recognize their voice, and the speaker had to pronounce each digit with 350 millisecond intervals in between to work accurately (also known as Discrete Speech Recognition). This was not a very powerful voice user interface, but it set the framework for speech recognition in voice user interfaces for the future.</p><p><strong>ELIZA (Chatbot)</strong></p><p>ELIZA was a chatbot developed by Joseph Wizenbaum at MIT in the 1960s. ELIZA was designed to imitate a conversation with a psychotherapist by continuing to ask preset questions to the user when given text-based input (<a href="https://en.wikipedia.org/wiki/ELIZA">ELIZA Chatbot</a>, Wikipedia). Joseph Wizenbaum created ELIZA in an attempt to showcase how one-dimensional human-computer conversation was at the time. Ironically, ELIZA was one of the first chatbots that was able to attempt the Turing test, and many users became attached to the chatbot and assigned it human characteristics and emotions — probably not what Wizenbaum was trying to achieve. This chatbot set the groundwork for many others to come.</p><p><strong>HARPY</strong></p><p>Created from the “Speech Understanding Research Program” created at Carnegie Mellon (funded by the United States Department of Defense) (<a href="https://www.semanticscholar.org/paper/Harpy%2C-production-systems-and-human-cognition-Newell/d453e0a1022719f7a52155d0f71761ac756d0047">Harpy, production systems and human cognition</a>, Allen Newell, 1978), HARPY was able to understand more than 1000 words — a huge improvement from the 9 digits Audrey was able to identify. Unlike Audrey, HARPY was able to recognize speech continuously, meaning that it could extract information from concatenated strings of word (aka sentences). This form of speech is most natural for humans, as compared to having to perform discrete speech (breaking up words and syllables so that the voice user interface can process the information). Most voice user interfaces today, such as Alexa, Google Assistant, and Siri use continuous speech recognition today as part of their design.</p><p><strong>ALICE</strong></p><p>Inspired by the ELIZA chatbot, ALICE (Artificial Linguistic Internet Computer Entity) was the first chatbot to use natural language processing. This made ALICE capable of having more realistic conversations between humans and machine — instead of having a simple, predetermined answers, ALICE attempts to make real conversation with the user by applying heuristic patterns to text-based input from the user. It considered how sentences are constructed grammatically, how to draw inferences about sentences from input, and how to explain reasoning from contextual sentences. ALICE was not able to pass the Turing test however. You can still check out and use ALICE online today.</p><p><strong>Siri</strong></p><p>2011, Apple released Siri — a voice assistant that acted both as a voice user interface and a chatbot. Siri combined a voice user interface and a graphical user interface to allow a hands-free experience for users to navigate the iOS operating system. This voice user interface was particularly powerful because it allowed the user to execute much of the core functionality of the iPhone’s operating system simple through voice commands.</p><h3>Chatbots</h3><p><strong>What is a Chatbot?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*l9mbOvA50w87sKXu" /><figcaption>Image Courtesy of businessnewsdaily.com</figcaption></figure><p>Chatbots are software that users can interact with via a chat interface (<a href="https://chatbotsmagazine.com/the-complete-beginner-s-guide-to-chatbots-8280b7b906ca">The Complete Beginner’s Guide to Chatbots</a>, Matt Schlict, Chatbots Magazine Medium, April 2016). Chatbots are often powered by a set of rules defined by the developer but many chatbots today use artificial intelligence and machine learning to create a more personalized experience for the user. Siri, Google Assistant, Slack, and Facebook Messenger are all examples of products that use chatbots. For instance, Slack, a group messaging tool used in businesses has a chat channel called “#slackbot”. In slackbot, you can get to know more about Slack, get information about what you can do in Slack, or even just hold a conversation with this bot through text conversations — without any real human being on the other side.</p><p><strong>Use Cases for Chatbots</strong></p><p>Chatbots can be used in a variety of different contexts. They are especially useful for customer service and personalized shopping experiences by offering real-time help and service inside a digital interface, much like you would get if you were physically at the business location itself. Chatbots are also helpful for delivering personalized information to users, and engaging users by actively messaging them.</p><p>Here are some more use cases for chatbots (<a href="https://chatbotsmagazine.com/the-complete-beginner-s-guide-to-chatbots-8280b7b906ca">Use Cases for Chatbots</a>, Kevin Scott, Chatbots Magazine Medium, October 2016) :</p><ol><li><em>Weather notifications</em>. Relatively standard, a chatbot continuously updates you on weather notifications.</li><li><em>News bot</em>. You might imagine a text-based chatbot that updates the user on news important to the user (decided through a filtering system). This way, the user experience is improved by delivering only news that is important to the user — where the user can then “converse” with the chatbot to receive more details.</li><li><em>Friendship</em>. Since the ELIZA chatbot, people have been developing attachments to chatbots. Xiaoice is an artificial intelligence chatbot based on an emotional computing framework. It has over 630 million registered users, and 20 million in China state that they talk to this bot just for reasons of “friendship”. In 2016, it was reported that over 25% of its users has confessed their love to the chatbot seriously.</li></ol><p><strong>Interface Design of Chatbots</strong></p><p>As we’ve discussed before, chatbots have a variety of different use cases. Based on the context of how the chatbot will be used, the interface design and conversation design would vary. Here are some ways that Hillary Black, Head of Chatbot Strategy and Conversation Design at Black Ops suggest that companies employ conversation design into their chatbots (<a href="https://medium.com/swlh/what-is-conversation-design-and-how-to-design-your-chatbot-3754f04ab1e7">What is Conversation Design, and How to Design Your Chatbot</a>, Hillary Black, Medium, Jan 2018):</p><ul><li><strong>Personality Building</strong>. Many chat bots in existence today employ relatively little personality in their responses. As chatbots typically follow a linear conversation path, building a digital personality for your chatbot is a great way to delight users and increase engagement with users. Without a personality, the chatbot is less able to make a connection with the user that will allow the user to complete their goals while increasing chatbot engagement.</li><li><strong>Copywriting</strong>. Every word matters in a constrained digital space, especially in phones. By employing copywriting in the conversation design in your chatbot, you are able to minimize valuable real-estate usage creating the best conversation experience for your users.</li><li><strong>User Experience</strong>. When designing and planning a chatbot, the conversation should be designed with all of the possible paths that a user <em>could </em>take to get to their goal, whether that be checking the weather or news, receiving customer service, buying something online, etc.</li><li><strong>Conversation Strategy</strong>. Developing a plan before even writing a script (the concepts, future use of the chatbot, measurable goals, etc) becomes a critical part in assessing how successful the chatbot will be in the future.</li></ul><h3>Voice User Interfaces</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*prdwnQzKdq9luWaO" /><figcaption>Image courtesy of Toptal Publications — Medium</figcaption></figure><p><strong>Definition</strong></p><p>Voice user interfaces are <em>interfaces that allows for voice interaction between people and devices.</em> Today, voice user interfaces are generally used by the user to issue speech-based commands or requests for the device’s operating system to run. Voice user interfaces however, by definition, do not have to follow this, it’s simply enough for the VUI to recognize speech. Audrey was the first voice user interface, and all it could do was recognize 10 digits.</p><p>Voice user interfaces do not need to have a graphical user interface that comes with it — although those are popular supplementary interfaces for devices with screens. Siri in the iPhone and Google Assistant in Android phones are popular examples of combining graphical user interfaces with voice user interfaces. “Pure” voice user interfaces might include the Amazon Echo or the Google Home, since they don’t have a graphical user interface that accompanies them.</p><p>Voice user interfaces offer a hands-free experience to navigating the underlying operating system, but don’t offer any physical affordances to what the operating system is capable of. As such, the design principles that follow for graphical user interfaces don’t necessarily apply to voice user interfaces, as voice user interfaces must account for more discoverability for them to be useful. Since the interaction is purely speech based, the user must rely on a mental model of what the voice user interface can do as compared to a GUI. As previously said, voice user interfaces can be combined with graphical user interfaces in order to create a more cohesive user experience, but the graphical user interfaces have to be designed around the voice user interface in order to accurately represent what is going on in the system.</p><p>In addition, voice user interfaces rely heavily on building a strong “mental model” for the user. Since the VUI can’t visually represent the capabilities of the entire system (<a href="https://www.dropbox.com/sh/tge70h5dit1nfv0/AAA4CiNSqORaYAGApodh3hqPa?dl=0&amp;preview=486-W19-02-Cognitive-Foundations.pptx">CSC 486 Slides</a>, Franz Kurfess, Cal Poly San Luis Obispo Computer Science, February 2019), it tries to mimic the users’ mental model of how “normal speech” works to make the system more natural to use.</p><p><strong>User Flow of Voice User Interfaces</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ugygAYPNsFU892Bg" /><figcaption>System Flow of a Voice User Interface. Image courtesy of Liveblogspot.com</figcaption></figure><p>Here is a general system flow for voice user interfaces. The user gives the system some voice command, where the system transcribes its intent. The device then gives some feedback in the form of a voice response and some sort of peripheral output to indicate the VUI has processed the request.</p><p><strong>General Design Principles (Conversation Design)</strong></p><p>Here are some design considerations to keep in mind when designing for voice user interfaces (<a href="https://medium.muz.li/voice-user-interfaces-vui-the-ultimate-designers-guide-8756cb2578a1">Voice User Interfaces (VUI) — The Ultimate Designer’s Guide</a>, Justin Baker, Muzli, November 2018).</p><ol><li><strong>Triggers</strong> are user input that initiate listening from the voice user interface. For example, to activate Google Assistant, the user says “Okay Google”. This is an example of a voice trigger — the user utters a phrase that initiates the device to begin listening for speech. There are also tactile triggers (holding down the home button on the iPhone), and motion triggers (creating motion in front of some sort of sensor).</li><li><strong>Leading Cues</strong> are responses to triggers that signify to the user that the device has started listening. This can come in the form of a sound (auditory feedback), a vibration (tactile feedback), or a light (visual feedback).</li><li><strong>Feedback</strong>. After a user gives the voice user interface some task, some form of feedback that the device is actually listening to the user becomes helpful. One of the central rules of UX design is letting the user know what is going on at the system at all times, and this is what the system is doing. One example of this would be providing pulsating light “waves” corresponding to the frequency of the user’s voice that indicate the device is listening to the user.</li><li><strong>Ending cues </strong>lets the user know that the device has stopped listening to the user and is processing the request. Similar to leading cues, these come in the form of visual, auditory, tactile, and motion cues.</li></ol><p><strong>Conversational UX</strong></p><p>Because the voice user interface is less of a conversation partner than a tool used to carry out tasks for users, many of the conversational rules of human to human conversations don’t necessarily apply to VUI’s. Human to AI interaction requires layers of confirmation, redundancy, and rectification (<a href="https://medium.muz.li/voice-user-interfaces-vui-the-ultimate-designers-guide-8756cb2578a1">Voice User Interfaces (VUI) — The Ultimate Designer’s Guide</a>, Justin Baker, Muzli, November 2018) Here are some conversation UX principles to help facilitate VUI interaction:</p><ol><li><strong>Affirmative — </strong>When the VUI understands the request and its context, it should respond with an affirmative action (such as a reply) that confirms the speech and the action that follows. It should not be vague, but rather done with enough context so the user is sure it is doing what they want. For example, if a user asks to change the temperature of the room to 78 degrees, instead of saying “Sure”, the VUI should respond with “Sure, setting the room temperature to 78 degrees Fahrenheit”. This allows the user to ensure that the task specified has been processed by the VUI, and allows them to make corrective actions if the task was fumbled by the interface. In keeping with the room temperature example, if the VUI responded with “Sure, setting the room temperature to 78 degrees Celsius.”, the user could immediately respond with adjustments to ensure they don’t end up in an extremely hot room.</li><li><strong>Corrective </strong>— When the voice user interfaces fails to understand the request given by the user, it should provide corrective options that might satisfy their request. This allows the user to take steps in the right direction, or redo their request to get to their goal. For example, when a NLP error occurs, instead of simply saying “I don’t understand”, the voice user interface could say “I’m sorry, did you mean <em>so and so request</em>.” This mitigates the frustration the user with the VUI for not being able to complete the task, but also provides the user with future steps.</li><li><strong>Empathetic </strong>— If the VUI cannot fulfill the user’s request, instead of simply stating “I can’t do that.”, it should be empathetic and take ownership for its inability to complete the task. After doing so, it can then point the user to corrective actions. Empathy is important for voice user interfaces to establishing a more personable relationship between the user and the VUI while effectively mitigating user frustration or anger towards the device.</li></ol><p>Voice user interfaces are largely dependent on the speech recognition algorithms behind them. If the voice user interface cannot recognize speech for one reason or another, it becomes important to address their shortcomings in the conversation design of voice user interfaces.</p><p><strong>Common Obstacles in Interaction with Voice User Interfaces</strong></p><p>In a research study conducted by Myers et al (2018), researchers look at the most common causes of frustrations encountered by users of voice user interfaces. By looking at empirical data of 12 participants aged 18–44 years old, they found that the most common obstacle users encountered were <strong><em>Natural Language Processing Errors</em></strong>, or where the NLP “mishears” the participants or maps their utterance to the wrong intent. Myers et al. (2018) also analyzed the most common tactics used to try and overcome obstacles encountered while using voice user interfaces, with the most common being <strong><em>Hyper-articulation</em></strong> (reemphasizing the intended action), <strong><em>Simplification</em></strong> (dumbing down the original intended action) and <strong><em>New Utterance</em></strong> (repeating the intended action) (<a href="http://delivery.acm.org/10.1145/3180000/3173580/paper6.pdf?ip=129.65.145.17&amp;id=3173580&amp;acc=ACTIVE%20SERVICE&amp;key=F26C2ADAC1542D74%2E2870C5A035FC0FDB%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&amp;__acm__=1549152829_7cb3e12bf570c001da4366a5945e2900">Patterns for How Users Overcome Obstacles in Voice User Interfaces</a>, Myers et al., CHI, April 2018).</p><p>However, NLP errors are not the greatest threat to the user experience of voice user interfaces. Users often understand that when a NLP error occurs, the user should employ Hyper-articulation as a solution. Other obstacles, such as <strong><em>Unfamiliar Intent, Failed Feedback, and System Errors</em></strong> were encountered less often, but the fallback tactics resorted to users who encounter these error were much more varied, ranging from Selling, Restarting, Frustration Attempts, and Quitting. Because of the variation of fallback tactics that were used, users felt more frustration with these other errors due to having to try multiple times to complete the task. Unfamiliar Intent obstacles had the most diverse tactics used. These other obstacles are more likely to lend to a negative user experience in voice user interfaces.</p><p>Even though Natural Language Processor errors were encountered most frequently by users, many developers and designers hold the viewpoint that the advancement of NLPs will help solve these obstacles to the user experience of voice user interfaces.</p><p>The fall-back tactics found in this study are all examples of the <em>user having to adjust to the system. </em>These are some of the problems faced in terms of how user interact with voice user interfaces that hopefully will be addressed in the future design of voice user interfaces. But what would a system that <em>adjusted to the user look like instead? </em>Enter Google Duplex.</p><h3>Google Duplex</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/0*wYYZfTWl3jzOUdb6" /><figcaption>Image courtesy of ideatovalue.com</figcaption></figure><p>While VUI’s and chatbots provide a way for users to have conversations with machines, what happens when users can’t differentiate from talking to a machine or talking to a person? Google Duplex is a tool that seems to accomplish this task.</p><p>In a demo video shown at Google’s 2018 press conference, a women seems to simply be setting up an appointment with a haircut salon. However, the women is not a person — it is an Artificial Intelligence tool that mimics human conversation. It’s a system that continuously adjusts to the user’s input during the course of conversation, unlike voice user interfaces that the user has to adjust to (the user has to correct themselves). It’s a role reversal. Because of this, many of the tactics that people use when interacting with voice user interfaces become obsolete — when interacting with Google Duplex, they don’t have to simplify their sentences, they don’t have to be specific in their wording, and they definitely do not have to hyper-articulate.</p><p>However, it was made pretty clear that Google Duplex isn’t a conversation replacer. It’s secondary goal is to made tasks that are traditionally done on the phone, to be done through some taps on your phone — making your life easier. Here, the human acts as a trigger for the conversation to happen but the human is not involved in the interaction that occurs after.</p><p>Google Duplex works as an enhancement rather than a replacement for voice user interfaces. One remarkable thing about Google Duplex is the fast speech recognition processing time — it is able to process speech in a manner that seems as fast as human cognition.</p><p><strong>Design of Google Duplex</strong></p><p>Google Duplex accomplishes what sounds like natural human conversation by considering a number of different things that are relevant to human speech (<a href="https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html">Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone,</a> Leviathan and Matias, Google AI Blog, May 2018).</p><ol><li><em>The Contextual Meaning of Words</em>. In the demo, you can see that Duplex understands what the context of different words mean. For example, when booking an appointment, Duplex was able to understand “4” in the context of time, rather than perhaps a number of people, etc. Humans can consider re-interpretations of language relatively easily, but machines have a harder time.</li><li><em>Elaborations</em>. Google Duplex can ask for the user to elaborate on hidden ambiguities.</li><li><em>Syncs</em>. The incorporations of affirmation questions such as “Can you hear me?”</li><li><em>Interrupts</em>. When the NLP engine cannot comprehend the user for some reason, Duplex can ask the user “Sorry can you start over”</li><li><em>Pauses</em>. “Can you hold?</li><li><em>Latency Expectation</em>. For certain words, such as “hello”, humans have low latency expectations for response times. Google Duplex considers the situation of the user when talking to a human and adjusts latency expectation, with the possibility of responding to certain words in less than 100ms.</li><li><em>Understanding of vocal cues</em>. Similar to how voice user interfaces work — vocal cues connote that the user is about to say something. Duplex is always listening to these vocal cues, ready to respond when necessary.</li><li><em>Intonations</em>. Duplex incorporates intonations, such as “um” and “uh” into its speech.</li></ol><p><strong>How did people make Google Duplex?</strong></p><p>Google Duplex was made through a combination of using a recurrent neural network (RNN) &amp; a text to speech engine by training the AI with tons of real life conversation between people (phone conversation data). It uses built-in memory to allow it to make predictions based on past outcomes with users, and going for the combinations of words that seemed to work best in different contexts (<a href="https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html">Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone,</a> Leviathan and Matias, Google AI Blog, May 2018).</p><p>They also used user studies to find out how to make Duplex’s conversation sound more natural. The results from user studies found out that the addition of speech disfluencies (“ummm”, “uhhs”, and “hmm”) sound more like natural for human listeners — so they increased the frequency of intonations.</p><p><strong>Limitations</strong></p><p>Google Duplex is a closed domain AI. It can only hold conversations that it has been trained extensively in, and at this point Google says it’s only been trained in being able to reserve tables, call for orders, and make appointments. It fails when someone tries to hold a conversation outside of its domain — it can’t call your grandma for you. The moment you start to stray from what Duplex is trained to do, it quickly breaks down and it’s pretty easy to recognize that you are not talking to another human. If Duplex breaks down, it will resort to signaling to a human backup operator, who will then complete the conversation for you.</p><p>But what if it was open domain and could have those conversations for you?</p><p><strong>Human Computer Interaction Implications</strong></p><p>After reading the Google blog post, it seems like Google Duplex isn’t anything technically groundbreaking as it fails outside of any conversation it’s not trained in (it would likely fail the Turing test if not talking about setting up appointments, making reservations, or taking orders). Since Google Duplex has only recently been rolled out the the general public (available in 43 states for Google Pixel owners), its current implications on broader human computer interaction are not entirely clear yet.</p><p>However, there are huge potential implications on human computer interaction of Google Duplex that society may face when Google Duplex is released. Here are some speculations about these implications that Google Duplex might face, and a broader idea of what might happen if AI like Duplex are trained in other domains, or even if a similar product was launched as an open domain AI.</p><p><strong><em>UnCanny Valley</em></strong></p><p>A common response on the internet is that Google Duplex that while it sounds extremely similar to a human being, they are revolted by how closely it sounds to a real human. The uncanny valley is a hypothesis originally developed by Masahiro Mori that states once the appearance of a robot or machine reaches a certain similarity to a human beings, humans stop developing empathetic and positive emotions towards the machine, instead replacing it with feeling of uncanniness and disgust (<a href="https://spectrum.ieee.org/automaton/robotics/humanoids/the-uncanny-valley">The Uncanny Valley,</a> Masahiro Mori, IEEE Robotics and Automation, June 2012). The uncanny valley hypothesis can be seen to be repeated in the public’s response to Google Duplex. While some applaud Google for being able to seemingly achieve natural human interaction with machines, others are unnerved by it. If the general public’s reaction to human-like bots is disgust, what does this mean for Google Duplex?</p><p><strong><em>Explainable AI and Trust</em></strong></p><p>Explainable AI is a program funded by DARPA that aims to produce more explainable models for artificial intelligence while maintaining a high level of learning performance. (<a href="https://www.darpa.mil/program/explainable-artificial-intelligence">Explainable Artificial Intelligence (XAI)</a>, Gunning, DARPA. )</p><p>The idea behind it is that in order for humans to begin to trust artificial intelligence they must understand a general idea or have a concept of how artificial intelligence. I see the same problem being shown here for Google Duplex that other AI have, but the barrier of trust is much higher because of its unique position — it’s a bot that mimics human speech particularly well. Humans do not like being deceived, and the fact that Google Duplex is parading as a human may deepen the divide of trust between humans and artificial intelligence, requiring the general body of knowledge surrounding understanding artificial intelligence to be more comprehensible to the public.</p><p>Google has tried to mitigate this increase of distrust of AI by requiring that Google Duplex identify itself as a bot before engaging in speech with humans, but it should be realized that because Google is not doing anything particularly groundbreaking in terms of artificial intelligence, similar bots may be produced that do not have these safeguards. In addition, by exposing itself as a bot, Google Duplex mitigates the amount to which humans can carry out “natural” conversations with it. Users who understand that they are talking to a bot will likely speak differently with it then they would with a human.</p><p><strong><em>Trained in Other Domains</em></strong></p><p>As mentioned previously, Google duplex is a closed domain artificial intelligence tool that quickly fails when trying to hold conversations outside of its domain. However, what if Google Duplex, or some other similar tool was trained in areas outside domains? This could lead to potentially disastrous situations for individuals and society. Here are some areas for thought:</p><ul><li>What happens if Google Duplex was trained to be able to prank people? If it could call the police and report a fake crime?</li><li>Call someone for you that has a restraining order on you?</li><li>Use profanity in its speech?</li><li>Pretend to be a high-ranking official and make potentially world-threatening phone calls with real humans in position of power?</li></ul><p>These are just some areas in which a bot pretending to human could cause damage in, whether that be in individual lives and relationships, or broader society in general. This also brings in the question of the ethics of Google Duplex: is it ethical for machines to pretend to be humans? In the case of Google Duplex, once the user initiates an interaction between the bot and another human, the user loses control of what that interaction might look like. There is also something that could go wrong. As such, certain laws around the usage of such bots need to be developed in order to prevent the situations above (and more) to be avoided. (<a href="https://www.cognilytica.com/2018/05/17/the-implications-of-google-duplex/">The Implications of Google Duplex</a>, Newsletter — Cognilytica, May 2018)</p><p><strong><em>Open domain natural language with machines</em></strong></p><p>Perhaps the most important consideration of Google Duplex would be to try and understand what might happen if it became an open domain artificial intelligence that could replicate human speech to a near human speech? What if Google Duplex could hold intelligent, contextual conversations with your grandma? Could Google Duplex potentially replace you? Robots are already at the point to which they are incredibly lifelike (<a href="https://futurism.com/the-most-life-life-robots-ever-created/">Six Life-Like Robots that Prove the Future of Human Evolution is Synthetic</a>, Tangermann, Futurism, August 2017), and Google Duplex has shown that AI can simulate human speech almost perfectly. If we get to the point in the future where it robots are indistinguishable sounding and looking like humans, and are able to hold conversation with humans in any context possible, do robots and artificial intelligence become (or surpass) humans? That is a question far beyond this blog post, and me.</p><h3>To Summarize…</h3><p>Conversation as an interface for humans to interact with devices has come a far way from the humble beginnings of Audrey. In today’s modern society, devices leverage conversation to get close to a “natural user interface”, making it easier for users to first begin using these devices. People are talking to chatbots all around the world, whether that be for learning how to use a new interface, to streamline their online shopping experiences, or to simply just make casual conversation. Voice user interfaces are now not only available on our computers and cell phones, but have become household staples with products such as Alexa and Google Home constantly listening for their next command. Google Duplex has gotten strangely close to mimicking real human conversation, and with that open up a whole new world of possibilities and potential consequences that have yet to be seen. It’s clear that conversation design is not going anywhere, and it’ll be interesting to see how the way we talk and communicate with our devices change as conversation continues to evolve.</p><h3>References</h3><ol><li><a href="https://designguidelines.withgoogle.com/conversation/conversation-design/what-is-conversation-design.html">Google Design Guidelines</a></li><li><a href="https://www.dropbox.com/sh/tge70h5dit1nfv0/AAA4CiNSqORaYAGApodh3hqPa?dl=0&amp;preview=486-W19-02-Cognitive-Foundations.pptx">CSC 486 Slides</a>, Franz Kurfess, Cal Poly San Luis Obispo Computer Science, February 2019</li><li><a href="https://voicebot.ai/2017/07/14/timeline-voice-assistants-short-history-voice-revolution/">Voice Assistant Timeline: A Short History of the Voice Revolution</a>, Ava Mutchler, voicebot.ai, July 2017</li><li><a href="https://futurism.com/images/the-history-of-chatbots-infographic/">The History of Chatbots, Futurism</a>, November 2016</li><li><a href="https://astaspeaks.wordpress.com/2014/10/13/audrey-the-first-speech-recognition-system/">Audrey: The First Speech Recognition System</a>, Asta Speaks, October 2014</li><li><a href="https://en.wikipedia.org/wiki/ELIZA">ELIZA Chatbot</a>, Wikipedia</li><li><a href="https://www.semanticscholar.org/paper/Harpy%2C-production-systems-and-human-cognition-Newell/d453e0a1022719f7a52155d0f71761ac756d0047">Harpy, production systems and human cognition</a>, Allen Newell, 1978</li><li><a href="https://chatbotsmagazine.com/the-complete-beginner-s-guide-to-chatbots-8280b7b906ca">The Complete Beginner’s Guide to Chatbots</a>, Matt Schlict, Chatbots Magazine Medium, April 2016</li><li><a href="https://chatbotsmagazine.com/popular-use-cases-for-chatbots-925ef8f2b48b">Use Cases for Chatbots</a>, Kevin Scott, Chatbots Magazine Medium, October 2016</li><li><a href="https://medium.com/swlh/what-is-conversation-design-and-how-to-design-your-chatbot-3754f04ab1e7">What is Conversation Design, and How to Design Your Chatbot</a>, Hillary Black, Medium, Jan 2018</li><li><a href="https://medium.muz.li/voice-user-interfaces-vui-the-ultimate-designers-guide-8756cb2578a1">Voice User Interfaces (VUI) — The Ultimate Designer’s Guide</a>, Justin Baker, Muzli, November 2018</li><li><a href="http://delivery.acm.org/10.1145/3180000/3173580/paper6.pdf?ip=129.65.145.17&amp;id=3173580&amp;acc=ACTIVE%20SERVICE&amp;key=F26C2ADAC1542D74%2E2870C5A035FC0FDB%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&amp;__acm__=1549152829_7cb3e12bf570c001da4366a5945e2900">Patterns for How Users Overcome Obstacles in Voice User Interfaces</a>, Myers et al., CHI, April 2018</li><li><a href="https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html">Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone</a>, Leviathan and Matias, Google AI Blog, May 2018</li><li><a href="https://spectrum.ieee.org/automaton/robotics/humanoids/the-uncanny-valley">The Uncanny Valley</a>, Masahiro Mori, IEEE Robotics and Automation, June 2012</li><li><a href="https://www.darpa.mil/program/explainable-artificial-intelligence">Explainable Artificial Intelligence (XAI)</a>, David Gunning, DARPA.</li><li><a href="https://www.cognilytica.com/2018/05/17/the-implications-of-google-duplex/">The Implications of Google Duplex</a>, Newsletter — Cognilytica, May 2018</li><li><a href="https://futurism.com/the-most-life-life-robots-ever-created/">Six Life-Like Robots that Prove the Future of Human Evolution is Synthetic</a>, Tangermann, Futurism, August 2017</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=815d7bee2233" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>