The following post is an excerpt from my book ‘Designing Human-Centric AI Experiences’ on applied UX design for Artificial intelligence.
—

We tend to anthropomorphize AI systems, i.e., we impute them with human-like qualities. Consumer demand for personality in AI dates back many decades in Hollywood and the video game industry¹. Many popular depictions of AI, like Samantha in the movie Her or Ava in Ex-Machina, show a personality and sometimes even display emotions. Many AI systems like Alexa or Siri are designed with a personality in mind. However, choosing to give your AI system a personality has its advantages and disadvantages. While an AI that appears human-like might feel more trustworthy, your users might overtrust the system or expose sensitive information because they think they are talking to a human. An AI with a personality can also set unrealistic expectations about its capabilities. If a user forms an emotional bond with an AI system, turning it off can be difficult, even when it is no longer useful. It is generally not a good idea to imbue your AI with human-like characteristics, especially if it is meant to act as a tool like translating languages, recognizing objects from images, or calculating distances.

While an AI that appears human-like might feel more trustworthy, your users might overtrust the system.

We’re forming these tight relationships with our cars, our phones, and our smart-enabled devices². Many of these bonds are not intentional. Some argue that we’re building a lot of smartness into our technologies but not a lot of emotional intelligence³. Affect is a core aspect of intelligence. Our emotions give cues to our mental states. Emotions are one mechanism that humans evolved to accomplish what needs to be done in the time available with the information at hand — to satisfice. Emotions are not an impediment to rationality; arguably, they are integral to rationality in humans⁴. We are designing AI systems that simulate emotions in their interactions. According to Rana El Kaliouby, the founder of Affectiva, this kind of interface between humans and machines is going to become ubiquitous that it will just be ingrained in the future human-machine interfaces, whether it’s our car, our phone or smart devices in our home or in the office. We will just be coexisting and collaborating with these new devices and new kinds of interfaces⁵. The goal of disclosing the agent’s ‘personality’ is to allow a person without any knowledge of AI technology to have a meaningful understanding of the likely behavior of the agent⁶.

Here are some scenarios where it makes sense to personify AI systems:

1. Avatars in games, chatbots, and voice assistants.
2. Collaborative settings where humans and machines partner up, collaborate and help each other. E.g., Cobots in factories might use emotional cues to motivate and signal errors. An AI assistant that collaborates and works alongside people may need to display empathy.
3. If your AI is involved with caregiving activities like therapy, nursing, etc., it might make sense to display emotional cues.
4. If AI is pervasive in your product or a suite of products, and you want to communicate it under an umbrella term. Having a consistent brand, tone of voice, and personality would be important. E.g., Almost all Google assistant capabilities have a consistent voice across different touchpoints like Google lens, smart speakers, Google assistant within maps, etc.
5. If building a tight relationship between your AI and the user is a core feature of your product.

Designing a personality for AI is complicated and needs to be done carefully.

Guidelines for designing an AI personality

Designing your AI’s personality is an opportunity for building trust. Sometimes it makes sense to imbue your AI features with a personality and simulate emotions. The job of designing a persona for your AI is complicated and needs to be done carefully.

Here are some guidelines to help you design better AI personas:

Don’t pretend to be human

People tend to trust human-like responses with AI interfaces involving voice and conversations. However, if the algorithmic nature and limits of these products are not explicitly communicated, they can set expectations that are unrealistic and eventually lead to user disappointment or even unintended deception⁷. For example, I have a cat, and I sometimes talk to her. I never think she is an actual human but is capable of giving me a response. When users confuse an AI with a human being, they can sometimes disclose more information than they would otherwise or rely on the system more than they should⁸. While it can be tempting to simulate humans and try to pass the Turing test, when building a product that real people will use, you should avoid emulating humans completely. We don’t want to dupe our users and break their trust. For example, Microsoft’s ​​Cortana doesn’t think it’s human, and it knows it isn’t a girl, and it has a team of writers that’s writing for what it’s engineered to do⁹. Your users should always be aware that they are interacting with an AI. Good design does not sacrifice transparency in creating a seamless experience. Imperceptible AI is not ethical AI¹⁰.

Good design does not sacrifice transparency in creating a seamless experience. Imperceptible AI is not ethical AI¹¹.

Clearly communicate boundaries

You should clearly communicate your AI’s limits and capabilities. When interacting with an AI with a personality and emotions, people can struggle to build accurate mental models of what’s possible and what’s not. While the idea of a general AI that can answer any questions can be easy to grasp and more inviting, it can set the wrong expectations and lead to mistrust. For example, an ‘ask me anything’ call-out in a healthcare chatbot is misleading since you can’t actually ask it anything — it can’t get you groceries or call your mom. A better call-out would be ‘ask me about medicines, diseases, or doctors.’ When users can’t accurately map the system’s abilities, they may over-trust the system at the wrong times, or miss out on the greatest value-add of all: better ways to do a task they take for granted¹².

Consider your user

When crafting your AI’s personality, consider who you are building it for and why they would use your product. Knowing this can help you make decisions about your AI’s brand, tone of voice, and appropriateness within the target user’s context. Here are some recommendations:

1. Define your target audience and their preferences. Your user persona should consider their job profiles, backgrounds, characteristics, and goals.
2. Understand your user’s purpose and expectations when interacting with your AI. Consider the reason they use your AI product. For example, an empathetic tone might be necessary if your user uses the AI for customer service, while your AI can take a more authoritative tone for delivering information.

Consider cultural norms

When deploying AI solutions with a personality, you should consider the social and cultural values of the community within which it operates. This can affect the type of language your AI uses, whether to include small-talk responses, amount of personal space, tone of voice, gestures, non-verbal communications, amount of eye contact, speed of speech, and other culture-specific interactions. For instance, although a “thumbs-up” sign is commonly used to indicate approval, in some countries, this gesture can be considered an insult¹³.

—

Designing responses

Leveraging human-like characteristics within your AI product can be helpful, especially if product interactions rely on emulating human-to-human behaviors like having conversations, delegation, etc. Here are some considerations when designing responses for your AI persona:

Grammatical person

The grammatical person is the distinction between first-person (I, me, we, us), second-person (you), and third-person (he, she, they) perspectives. Using the first person is useful in chat and voice interactions. Users can intuitively understand a conversational system since it mimics human interactions. However, using first-person can sometimes set wrong expectations of near-perfect natural language understanding, which your AI might not be able to pull off. In many cases, like providing movie recommendations, it is better to use second-person responses like ‘you may like’ or third-person responses like ‘people also watched.’

Tone of voice

What we say is the message, and how we say is our voice¹⁴. When you go to the dentist, you expect a different tone than when you see your chartered accountant or your driving instructor. Like a person, your AI’s voice should express personality in a particular way; its tone should adjust based on the context. For example, you would want to express happiness in a different tone than an error. Having the right tone is critical to setting the right expectations and ease of use. It shows users that you understand their expectations and goals when interacting with your AI assistant. An AI assistant focused on the healthcare industry may require some compassion, whereas an AI assistant for an accountant may require a more authoritative/professional tone, and an AI assistant for a real estate agency should have some excitement and enthusiasm¹⁵.

Strive for inclusivity

In most cases, try to make your AI’s personality as inclusive as possible. Be mindful of how the AI responds to users. While you may not be in the business of teaching users how to behave, it is good to establish certain morals for your AI’s personality. Here are some considerations:

1. Consider your AI’s gender or whether you should have one. By giving it a name, you are already creating an image of the persona. For example, Google Assistant is a digital helper that seems human without pretending to be one. That’s part of the reason that Google’s version doesn’t have a human-ish name like Siri or Alexa¹⁶. Ascribing your AI a gender can sometimes perpetuate negative stereotypes and introduce bias. For example, an AI with a doctor’s persona with a male name and a nurse with a female name can contribute to harmful stereotypes.
2. Consider how you would respond to abusive language. Don’t make a game of abusive language, and don’t ignore bad behavior. For example, if you say ‘fuck you’ to Apple’s Siri, it denies responding to you by saying ‘I won’t respond to that’ in a firm, assertive tone.
3. When users display inappropriate behavior, like asking for a sexual relationship with your AI, respond with a firm no. Don’t shame people, but don’t encourage, allow, or perpetuate bad behavior. You can acknowledge the request and say that you don’t want to go there.
4. While it can be tempting to make your AI’s personality fun and humorous, humor should only be applied selectively and in very small doses¹⁷. Humor is hard. Don’t throw anyone under the bus, and consider if you are marginalizing anyone.
5. You will run into tricky situations when your users will say that they are sad, depressed, need help, or are suicidal. In such cases, your users expect a response. Your AI’s ethics will guide the type of response you design.

Don’t leave the user hanging.

Ensure that your users have a path forward when interacting with your AI. You should be able to take any conversation to its logical conclusion, even if it means not having the proper response. Never leave users feeling confused about the next steps when they’re given a response.

—

Risks of personification

While a human-like AI can feel more trustworthy, imbuing your AI with a personality comes with its own risks. Following are some risks you need to be mindful of:

1. We should think twice before allowing AI to take over interpersonal services. You need to ensure that your AI’s behavior doesn’t cross legal or ethical bounds. A human-like AI can appear to act as a trusted friend ready with sage or calming advice but might also be used to manipulate users. Should an AI system be used to nudge users for the user’s benefit or for the organization building it?
2. When affective systems are deployed across cultures, they could adversely affect the cultural, social, or religious values of the community in which they interact¹⁸. Consider the cultural and societal implications of deploying your AI.
3. AI personas can perpetuate or contribute to negative stereotypes and gender or racial inequality. For example, suggesting that an engineer is a male and a school teacher is female.
4. AI systems that appear human-like might engage in the psychological manipulation of users without their consent. Ensure that users are aware of this and consent to such behavior. Provide them an option to opt-out.
5. Privacy is a major concern. For example, ambient recordings from an Amazon Echo were submitted as evidence in an Arkansas murder trial, the first time data recorded by an artificial-intelligence-powered gadget was used in a U.S. courtroom¹⁹. Some AI systems are constantly listening and monitoring user input and behavior. Users should be informed of their data being captured explicitly and provided with an easy way to opt out of using the system.
6. Anthropomorphized AI systems can have side effects such as interfering with the relationship dynamics between human partners, causing attachments between the user and the AI that are distinct from the human partnership²⁰.

—

The above post is an excerpt from my book ‘Designing Human-Centric AI Experiences’ on applied UX design for Artificial intelligence.

Designing great AI products — Personality and emotion was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

The following post is an excerpt from my book ‘Designing Human-Centric AI Experiences’ on applied UX design for Artificial intelligence.

—

I’m a stickler about having Chai and biscuits in the morning. Every week I go to the neighborhood grocery store to buy a regular packet of Britannia Marie Gold biscuits. When paying at the counter, I don’t check prices; I simply hand out my card. The store owner keeps these biscuits stocked for me and sometimes even gives me a discount. Over the past few years, I have come to trust this little store.

Almost all successful human interactions are built on trust. You often spend more money on brands that you trust over cheaper alternatives, which is why companies spend a lot of money on building their brand. When you go to a doctor for a health checkup, the underlying assumption is that you can trust their diagnosis. Most of us prefer to deposit our savings with a trusted bank. We don’t make large purchases on websites we don’t trust. We tend to leave our kids to play in safe places with trustworthy people.

Without trust, it would be debilitating to perform even the simplest of tasks. Without trust, almost all systems break, and society would fall apart. Trust is the willingness to take a risk based on the expectation of a benefit¹. Trust in business is the expectation that the other party will behave with integrity. In a high trust environment, people are honest and truthful in their communication. There is a fair exchange of benefits, and people operate in good faith. There is adequate transparency into the workings of the system. Insufficient transparency leads to distrust. Clear accountability is established; people don’t play blame games. A successful team of people is built on trust, so is a team of people and AI.

A successful team of people is built on trust, so is a team of people and AI.

Trust in AI

We can use AI systems in organizations to generate predictions, provide insights, suggest actions, and sometimes even to make decisions. The output of AI systems can affect different types of stakeholders directly or indirectly. AI systems are not perfect. They are probabilistic and learn from past data. Sometimes they make mistakes which is why we require humans to oversee them. For AI products to be successful, people who use them or are affected by them need to trust these systems.

Components of user trust

When building trust with the users of your AI system, you are essentially trying to develop a good relationship. Your users need to have the right level of confidence when using or working alongside your AI. Following components contribute to user trust in your product:

1. Competence
2. Reliability
3. Predictability
4. Benevolence

Competence

Competence is the ability of the product to get the job done. Does it improve the experience or address the user’s needs satisfactorily? A good-looking product or one with many features that do not fulfill user needs is not competent. Strive for a product that provides meaningful value that is easy to recognize². Google search is an example of a competent product. It generally offers satisfactory results for your questions.

Reliability

Reliability indicates how consistently your product delivers on its abilities³. A reliable product provides a consistent, predictable experience that is communicated clearly. A product that performs exceptionally well one time and breaks down the next time is not reliable. Apple’s iPhone is an example of a reliable product. It might not carry all the features its competitors have, but you can reasonably trust the ones it does have.

Predictability

A predictable interface is necessary, especially when the stakes are high. If the user comes to your product to perform critical, time-sensitive tasks, like quickly updating a spreadsheet before a client presentation, don’t include anything in your UI that puts habituation at risk⁴. A probabilistic AI-based solution that can break the user’s habit is not ideal in such cases. However, suppose you think that users have open-ended goals like exploration. In that case, you can consider a dynamic AI-based solution that sometimes breaks user habits, e.g., selecting a movie from dynamic AI-based suggestions.

Benevolence

Benevolence is the belief that the trusted party wants to do good for the user⁵. Be honest and upfront about the value your users and your product will get out of the relationship. Patagonia is a clothing brand that makes jackets that does a great job with benevolence. While their products can be expensive, they encourage people to reuse and repair their Patagonia clothes, giving a percentage of their sales for environmental causes. The company is upfront about its value to the customer.

Trust calibration

AI can help people augment or automate their tasks. People and AI can work alongside each other as partners in an organization. To collaborate efficiently, your stakeholders need to have the right level of trust in your AI system.

Users can sometimes over-trust or distrust your AI system, which results in a mismatch of expectations. Users may distrust your AI when trust is less than the system’s capabilities. They may not be confident about your AI’s recommendations and decide not to use them. Users rejecting its capabilities is a failure of the AI system. Over-trust happens when user trust exceeds the system’s capabilities, leading to users trusting an AI’s recommendation when they should be using their own judgment. For example, users over-trusting the suggestions of a stock prediction service can lead to a financial loss. Overtrust can quickly turn into distrust the next time this person uses the service.

Product teams need to calibrate user trust in the AI regularly. The process to earn user trust is slow, and it’ll require proper calibration of the user’s expectations and understanding of what the product can and can’t do⁶.

How to build trust?

AI systems are probabilistic and can make mistakes. It is the job of product creators to build trustworthy relationships between the AI and its users. Building trust is not about being right all the time; it is about integrity, accepting mistakes, and actively correcting them. Users should be able to judge how much they can trust your AI’s outputs, when it is appropriate to defer to AI and when they need to make their own judgments. There are two essential parts to building user trust for AI systems viz. Explainability and Control.

Explainability

If we don’t understand how AI systems work, we can’t really trust them or predict the circumstances under which they will make errors⁷. Explainability means ensuring that users of your AI system understand how it works and how well it works. Your explanations allow product creators to set the right expectations and users to calibrate their trust in the AI’s recommendations. While providing detailed explanations can be very complicated, we need to optimize our explanations for user understanding and clarity.

Control

Users should be able to second-guess the AI’s predictions. Users will trust your AI more if they feel in control of their relationship with it. Giving users some control over the algorithm makes them more likely to feel the algorithm is superior and more likely to continue to use the AI system in the future⁸. You can do this by allowing users to edit data, choose the types of results, ignore recommendations, and correct mistakes through feedback.

Building trust is not about being right all the time; it is about integrity, accepting mistakes, and actively correcting them.

Your AI system will work alongside people and will make decisions that impact them. People and AI can work alongside each other as partners in an organization. To collaborate efficiently with your AI system, your stakeholders need to have the right level of trust. Building trust is a critical consideration when designing AI products.

—

The above post is an excerpt from my book ‘Designing Human-Centric AI Experiences’ on applied UX design for Artificial intelligence.

Designing great AI products — Building trust was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

The following post is an excerpt from my book ‘Designing Human-Centric AI Experiences’ on applied UX design for Artificial intelligence.

—

Building AI products is technical but also requires creativity. Popular culture depicts people who work in technology as science and math nerds with low EQ. Characters like Steve Urkel on Family Matters, Sheldon Cooper on The Big Bang Theory, and Dennis Nedry from Jurassic Park all create a picture of social misfits, man-children, more comfortable with a slide rule than a conversation¹. The media loves to perpetuate these stereotypes, which can be incredibly misleading. While most people in tech are interested in science and technology, the best people I know that build AI products have eventful lives with interesting hobbies. Building AI products is more similar to making music or producing a film than doing math or science.

Building AI products is more similar to making music or producing a film than doing math or science.

An orchestra needs different musicians to communicate, collaborate and work together to create a great performance. Similarly, different members of your AI product team need to work together to build a great user experience. If you wonder why ‘simple’ changes to your product take so long to make, then start by understanding the interaction between you and your engineering team.

Think of engineers less as people who churn out code and more like creative problem solvers. Building a great user experience is everyone’s responsibility. Extend your UX family. As a product designer or product manager, getting too prescriptive too quickly may result in unintentional anchoring and diminish the creativity of your engineering counterparts. There are many ways to approach any AI problem, and your engineers love overcoming technical challenges. Trust them to use their intuition and encourage them to experiment².

Your job as a designer is to help your product team make great user-centered choices.

Effective collaboration

At its heart, building AI products is a creative endeavor. You are trying to solve a user problem by building not just the software but also imbuing it with intelligence. Training an AI model is a slow and iterative process. Engineers often need to use their judgment and imagination when tuning the algorithm. Your job as a designer is to help your product team make great user-centered choices along the way³.

Collaboration is a two-way activity. Inspire your team members with stories, prototypes, customer feedback, findings from user research, and introduce them to UX methods and design principles. Sharing the design process can help your team be comfortable with iteration and work wonders for your ability to influence the product. Mostly, you just have to treat your team members as people, full of ambitions to learn and grow, motivations to do their best work, and a range of skills they want to exercise.

Our attempts to create artificial intelligence have, at the very least, helped highlight the complexity and subtlety of our own minds.

Easy things are hard

Many of the things that children can easily do, like picking up objects of different shapes, recognizing the tone of voice, navigating playgrounds, etc., have turned out to be surprisingly difficult for AI systems to achieve than seemingly complex tasks like diagnosing diseases, beating human champions at Chess and Starcraft, or solving complex algebraic problems. Easy things are hard. There’s a famous rule of thumb in any complex engineering project: the first 90 percent of the project takes 10 percent of the time, and the last 10 percent takes 90 percent of the time⁴. And while child’s play is hard, our most critical jobs such as running organizations, governments, caregiving, or teaching chemistry that have mostly complex and unobservable environments are even harder for AI. Our attempts to create artificial intelligence have, at the very least, helped highlight the complexity and subtlety of our own minds.

Collaborate, don’t dictate

Most designers I know cringe at the idea of someone sitting next to them and asking them to change fonts, colors, and other elements on a design artifact. To me, nothing is more annoying than someone breathing down your throat and micromanaging your work by telling you exactly what to do. It might be ok to do it in some cases, but most times, I’d like to be left alone to do focused work. Apart from demotivating people, micromanagement signals that you don’t trust them to make decisions.

Many of us do the same when it comes to working with tech teams. After building a proposal, getting buy-in from leadership, we go to our engineers and say something like ‘Hey, do this quick, by this deadline’ and then run away. Engineering teams then come back with a timeline that you did not account for. Seemingly ‘simple’ changes take much longer than anticipated. Your tech team does not seem as excited, engaged, or motivated. The relationship between developers and businesspeople is not well understood, but is critical to solving business problems with technology⁵.

Apart from demotivating people, micromanagement signals that you don’t trust them to make decisions.

Saying things like ‘This should just be a quick thing,’ or ‘You can do this in a day’ is not only annoying but also meaningless. Unless you understand how things are put together, unless you know how the infrastructure is built, you have no idea how long it will take⁶. How can product teams commit to a deadline if they don’t understand how long something will take to build? And not having enough time to build will lead to one or more of these undesirable outcomes:

• Features will get cut to meet the deadline.
• Product quality will suffer.
• Your team will be burnt out.

Share problems, not solutions

The key to getting product teams and developers to work well together is for the product teams to share problems, not solutions. Instead of presenting engineers with a solution that’s already defined, product designers and managers can share the problem and ask engineers to help figure out the fastest way to solve it based on how the existing systems are constructed. Including your tech teams early in the discussion is not just about being nice or not hurting their feelings; it also has serious advantages. Many developers have some deeper insight into the integration or feasibility of some products or some features. Finding the shortest technical path in context is what engineers do for a living. It’s what they’re trained to do in computer science classes⁷. But instead of harnessing their knowledge and insight, most companies just tell engineers precisely what to do. They ask them to take off their creative and problem-solving hats. Don’t tell, ask. Share problems, not solutions.

Don’t tell, ask. Share problems, not solutions.

The key to building great AI products is to engage your product and tech teams in leveraging their whole brain towards the problem. Design is creative, and so is code. “I’ve always thought that engineering is one of the most creative jobs in the world,” Amazon’s CTO, Werner Vogels, says⁸. Yet many teams don’t realize that writing code is creative; they don’t create an environment where developers can exercise this creative muscle, and everybody loses. Trust that your tech teams are adept at applying the craft of software to serving customers and solving problems, and building great products.

Imagine you are trying to decide between two equally important features to present to the leadership team for buy-in. Before you do that, you share these two features with your tech teams to get some early estimates. After discussing with your developers, you realize that one of the features will take one day to build vs. another that would take three months. Wouldn’t your pitch to leadership change based on this newly acquired information? Engineers can often make these quick estimations, and you should leverage their instincts in picking ideas that give the most return on investment.

You could define the biggest, scariest problems your organization faces and do a ‘call for solutions.’ While not every solution might be worth pursuing. But by framing the largest problems, you give your team members the opportunity to think about and empathize about the same thing. This can also save you from the trap of falling in love with your own idea. Great ideas can come from anywhere. As designers and managers, our goal is to let the best idea win, not the idea of the person with the best compensation.

Don’t dictate what tech teams should build. Don’t tell them what code to write or how to write it. Start with simple questions like, ‘Hey, wouldn’t it be cool if we could do X?’, ‘Hey, is there some way to build this?’ or ‘what’s the fastest/best way to make X possible?’ Collaborate with them and trust them to make great engineering decisions. Don’t turn to your engineers just for code, but also for creative problem-solving. When you tell someone exactly how they should do something, it’s a one-way street, and you don’t engage them. But when you ask someone to solve a problem, the conversation becomes more collaborative, and they are likely to become more engaged.

Motivation

Motivated and engaged teams are key to effective collaboration. When you’re a designer working in an AI product team, motivating and engaging your team members is less about standing in front of a podium and giving a sermon and more about small, regular, and consistent interactions about the user experience. While you might sometimes get a chance to present in front of a large audience, you would often communicate with a small set of people in person, over video calls, through documents, and over email or messaging. Don’t wait for the perfect solution. Share progress with your team members and involve them early. Inspire them with examples, slideshows, personal stories, vision videos, prototypes, or even analysis from user research. Help them build a deeper understanding of your product principles and experience goals. Aim to create a feeling of urgency, energy, and that what they are building is important.

When you’re a designer working in an AI product team, motivating and engaging your team members is less about standing in front of a podium and giving a sermon and more about small, regular, and consistent interactions about the user experience.

—

References

• 1, 5, 6, 7, 8: Lawson, Jeff, and Eric Ries. Ask Your Developer. Harper Business, 2021.
• 2, 3: Holbrook, Jess. “Human-Centered Machine Learning”. Medium, 2017, https://medium.com/google-design/human-centered-machine-learning-a770d10562cd.
• 4: Mitchell, Melanie. Artificial Intelligence. 1st ed., Farrar, Straus And Giroux, 2019.

—

The above post is an excerpt from my book ‘Designing Human-Centric AI Experiences’ on applied UX design for Artificial intelligence.

Designing great AI products — Effective collaboration for designers was originally published in UX Planet on Medium, where people are continuing the conversation by highlighting and responding to this story.

The following post is an excerpt from my book ‘Designing Human-Centric AI Experiences’ on applied UX design for Artificial intelligence.

—

An organization is essentially lots of people and machines working together to achieve a particular outcome — different teams and departments in an organization work towards specific objectives that align with the company’s goals. For example, an HR department has the goal of increasing employee headcount and improving retention, leading to higher output for the company.

AI cannot do everything. Problems such as running a company or teaching psychology are much more challenging than playing chess or recognizing faces. They are complex problems for which it is difficult to find perfect solutions. At work, a common theme is not to try to do everything yourself. Collaboration in an organization generally means dividing up a function into jobs and tasks, which are then delegated to specific individuals based on their strengths. Some of these tasks can be done better by people, while some can be performed better by an AI.

Whenever you employ people, add machines, or include AI, think of it as hiring the person, machine, or the AI to accomplish your goal.

Roles for AI

When incorporating AI into your workflows, you can also think of AI as a part of the organization and assign it specific roles. You can give AI different levels of control, and these roles can range from tools, assistants to AI managing a group of people.

Tools

When people use AI as a tool, they have the most amount of control. Search functionality that helps you find text in images is an example of AI as a tool.

Assistants

Although the line between a tool and an assistant is thin, you can think of assistants as systems that respond to your requests and are proactive at the same time. Grammarly, a software that corrects your grammar while you write, or Google assistant proactively suggesting when you should leave for work, are examples of assistants.

Peers

You can think of AI as a peer when it completes an independent function without the involvement of other humans. A robot in a factory assembly line acts like a peer working alongside workers. It performs the assigned job, which is not controlled by the human next to it.

Managers

AI as a manager is generally a system that organizes people and sometimes assigns them tasks to achieve greater output. Although the idea of an AI manager may sound scary, we are already surrounded by many machine managers. Think of a traffic light, air traffic control systems, or the automated management of an Amazon warehouse¹.

Figure 3–2. Roles for AI. Level of control that AI has is lower on the left and higher on the right.
(a) AI as a tool: Google lens translating text on images. Source: https://play.google.com/store/apps/details?id=com.google.ar.lens
(b) AI as an assistant: Grammarly assisting when writing an email. Source: https://www.grammarly.com.
(c) AI as a peer: Robots working alongside people in a factory. Source: Photo by Lenny Kuhne on Unsplash.
(d) AI as a manager: Traffic lights managing cars and people. Source: Photo by Florian Wehde on Unsplash.

Jobs

A job is a way of accomplishing a goal. Organizations have many jobs to be done. Whenever a company employs people, adds machines, or includes AI, they essentially hire the person or AI to accomplish a goal. Any job is a collection of tasks performed by an individual or a team. For example, the job of an architect is to design a building. Within this job, the architect does multiple tasks like talking to clients, creating drawings, planning space, choosing materials, etc. The job of the recruitment team is to find the right candidate. Systems can have jobs too. The job of the digestive system is to digest food. Similarly, machines also have jobs like the refrigerator has a job of keeping food cold.

Using AI can lead to four implications for jobs²:

1. AI can augment jobs, as in the case of AI assistants that correct grammar when writing an email.
2. AI can contract jobs by making parts of the job shorter, as in the case of robots in fulfillment centers.
3. AI can lead to a reorganization of jobs. For example, radiologists might spend more time on analysis than sifting through scans.
4. AI can shift the emphasis of specific skills required for a job. For example, in the case of self-driving school buses, the driver may act more like a coordinator, emphasizing communication skills over purely driving ability.

A job is a collection of tasks.

Tasks

A task is a unit of activity that an individual performs to do their job. A job is a collection of multiple tasks. For example, a product designer aims to achieve the best possible user experience given the constraints. They accomplish this by performing various jobs like talking to users, building personas, wireframing, prototyping, communicating with stakeholders, etc. You can further break these jobs into individual tasks like sending emails, planning timelines, choosing suitable interaction patterns, ensuring accessibility, etc.

A study by the Mckinsey Global Institute identified that it is technically feasible to automate 50% of activities — not jobs, but tasks in the US economy with AI. If we map these tasks back to jobs, they found that about 60% of occupations have a third of their tasks automable³. This tells us that many more jobs will be complemented rather than replaced by AI in the near future.

Many more jobs will be complemented rather than replaced by AI in the near future.

Most real-world problems are complex. Organizations operate in mostly unobservable environments with a great deal of uncertainty over long timescales. We are yet to build AI that can run an organization effectively. However, AI can help people with some of their tasks, enhancing overall effectiveness. Incorporating AI can improve the speed and quality of outcomes by streamlining processes, automating tasks, speeding up people’s jobs, etc. You can think of incorporating AI as a way of increasing the collective intelligence of the organization and thereby improving its impact.

—

References

1. Dzieza, Josh. “Robots Aren’T Taking Our Jobs — They’Re Becoming Our Bosses”. The Verge, 2020, https://www.theverge.com/2020/2/27/21155254/automation-robots-unemployment-jobs-vs-human-google-amazon.
2. Agrawal, Ajay et al. Prediction Machines. Harvard Business Review Press, 2018.
3. Ford, Martin R. Architects Of Intelligence. Packt, 2018.

—

The above post is an excerpt from my book ‘Designing Human-Centric AI Experiences’ on applied UX design for Artificial intelligence.

Designing great AI products — AI roles, jobs, and tasks was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

We usually think of intelligence as a quality that individuals possess. However, many of our largest and most complex problems are often solved by groups of individuals. If we think of intelligence as an ability to solve problems, groups can also act intelligently. We call this intelligence ‘Collective intelligence’ — the result of a group of individuals acting together in ways that seem intelligent. Collective intelligence is something that makes systems effective, productive, adaptable, or even resilient.

‘Collective intelligence’ — the result of a group of individuals acting together in ways that seem intelligent.

Although the individuals in the system might not have the same goals, most systems do work towards achieving a particular outcome. A company’s goal could be increasing its profits or market cap while a government’s goal could be improving the well-being of its citizens. This desired outcome can simply be described as the impact of the system. Increasing collective intelligence can help increase the efficiency, productivity, or output of the system, and thereby its impact.

The collective intelligence of a system can be increased through three different ways:

1. Addition

• Adding more humans into the system. Eg. Hiring more
• Adding more machines into the system. Eg. Purchasing more equipment

2. Improvement

• Improving the machines. Eg. Hardware or software upgrades
• Improving humans in the system. Eg. Learning a new skill, reading a book

3. Connection

HUMAN-MACHINE COLLABORATION

• Making machines help humans. E.g. Spellcheck
• Making humans help machines. E.g. Writing software programs

OTHER

• Making humans help humans. E.g restructuring organisations
• Making machines help machines. E.g Software that automates tasks for the program

Using AI to increase the intelligence of groups

When we decide to introduce technologies like Artificial Intelligence into the system, we are inherently trying to increase the system’s collective intelligence. AI increases the collective intelligence of the system by:

1. Improvements in the machines themselves
2. Making machines help other machines
3. Improving human-machine collaboration

AI improving machines

Imagine a machine that boils eggs. You put in some eggs, set the timer and you get boiled eggs. Pretty soon you run into some obvious problems with this machine. It only boils chicken eggs well. What about duck eggs? What about Quail? Let’s make a smarter egg boiler. It uses an AI algorithm to analyze different types of eggs, temperature, humidity and whole host of other parameters to boil any kind of egg perfectly. Instead of creating a new product altogether, they simply build an attachment for the original egg boiler that converts it into a smart boiler. In a similar fashion, a software update to an electric boiler can make it more efficient, humane and appear smarter.
This is an example of AI improving existing machines. This can happen either through improvements in the hardware or the software.

AI making machines help other machines

There is a factory that produces pizza. The factory consists of two main machines: one that makes the base and one that makes the topping. However these machines do not talk to each other. There is a human operator that moves the pizza bases to the the topping machine where the pizza gets built. The company makes some simple software updates and some hardware changes to make the transition between these machines seamless and without the need of a human operator. This is very effective but not very smart. Some engineers tweak the software of these machines and plug it with an AI based sentiment detector. The system predicts the demand for pizza in the locality where the company has customers as well as identifies more areas for supply. It discovers new products like plain pizza base or topping sauce and makes the machines collaborate together to meet demand. Such a system would greatly increase the efficiency of the system. This is an example of making machines help other machines. The AI in this case acts like a manager of the machines.

Cobots: Improving human-machine collaboration

Often humans and machines are seen as adversaries fighting for each other’s jobs. This view however neglects the powerful opportunities in collaboration between the two sides. Now as our tools become more interconnected and intelligent, machines and humans start looking like symbiotic partners in the system. This symbiotic relationship can enable new business models and create jobs that never existed. And yes it will impact jobs too. Imagine a flexible, nimbler robot that works alongside humans to help perform generic, repetitive tasks. Or an algorithm that produces a variety of design options based on the designers preferences and goals. The designer can simply select the most optimum option and start using it as a reference or straight up modify it. This idea is sometimes also referred as ‘Organic AI,’ where machines are flexible and easily partner with humans to perform tasks. Such teams can easily adapt to new data and novel market conditions. The machine can become an extension of the human mind and body.

Although improving machines and helping them talk to each other sound like great opportunities, so far we do not have human systems that are fully automated and operate without any person overseeing them. The opportunities that will truly increase the impact of systems lies somewhere in the middle, where humans and AI collaborate and push each other to achieve higher levels of performance.

References

Superminds: The Surprising Power of People and Computers Thinking Together

Human + Machine: Reimagining Work in the Age of AI

Don’t forget to give us your 👏 !

Artificial Collective Intelligence was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

The entire world is organised into systems. Be it an ecosystem in the forest, nations, markets, communities, governments, or companies. We all form a part of different systems at different times in our life. If you have ever worked in an organization or been to school, you have been a part of a system.

Ever used email at work? Then the computer might have collaborated with you on composing the email, checking for spelling errors, and making sure the email reaches its target receiver. Most modern human systems like organizations, communities, or nations are a combination of humans and machines collaborating, helping, and working together. It is very rare to find a system with only humans or only machines in it. Your neighbourhood tech company is nothing but a group of humans and machines working together.

Note: Machines here are defined as any technology that helps humans. Machines could be hardware-based in the case of a conveyor belt, washing machine, or a robot, or they could be software like email, word processor, or chatbots.

If Capitalism is a failure, then what is the Alternative? | Data Driven Investor

When you observe a system from the outside (a company, a government), it sometimes feels like the system has a mind of its own. Let’s call this idea a ‘Supermind’ — a group of individuals acting together in ways that seem intelligent. The individual in this case could be a human or a machine. The verb ‘acting’ signifies the processes and interconnections between components of the system. One thing to note here is that the individual components do not always need to cooperate or have the same goals, however they need to be acting together.

‘Supermind’ — a group of individuals acting together in ways that seem intelligent.

We usually think of intelligence as a quality that individuals possess. However, many of our largest and most complex problems are often solved by groups of individuals (or Superminds). If we think of intelligence as an ability to solve problems, then Superminds can also act intelligently. We call this intelligence ‘Collective intelligence’ — the result of a group of individuals acting together in ways that seem intelligent. Collective intelligence is something that makes systems effective, productive, adaptable, or even resilient.

‘Collective intelligence’ — the result of a group of individuals acting together in ways that seem intelligent. Collective intelligence is something that makes systems effective, productive, adaptable or even resilient.

Although the individuals in the system might not have the same goals, most systems do work towards achieving a particular outcome. A company’s goal could be increasing its profits or market cap while a government’s goal could be improving the well-being of its citizens. This desired outcome can simply be described as the impact of the system.

Increasing collective intelligence can help increase the efficiency, productivity or output of the system, and thereby its impact.

References

Superminds: The Surprising Power of People and Computers Thinking Together

Systems, Superminds and Collective Intelligence was originally published in DataDrivenInvestor on Medium, where people are continuing the conversation by highlighting and responding to this story.

Adam Smith’s idea of the division of labor talks about allocating roles based on relative strengths. It is primarily targeted towards the division of physical labor. Computers and AI making predictions enable a cognitive division of labor between humans and machines based on their respective cognitive strengths. While machines are good at sifting through large volumes of data quickly (prediction), humans are better at interpreting complex information (judgement). Prediction and judgement are complements, as the use of AI prediction increases, the value of human judgement rises.

It is important to note that current AI techniques don’t actually bring intelligence but an important component of it — Prediction. Prediction is the process of filling missing information from previous data. For eg. ‘Alexa’ or ‘Google’ doesn’t know the capital of India, but is able to predict that when someone asks a question like this, they might be looking for New Delhi.

📝 Moravec’s paradox
Easy things for a five-year-old are the hard things for a machine, and vice versa. What’s easy for us (folding a towel, for instance) can be extremely difficult for machines. And what’s easy for machines (spotting hidden patterns in huge datasets, for instance) can be extremely difficult for us to do.

What machines do better

• Machines are much better at statistical reasoning and thereby predictions
• Machines can churn through huge volumes of data
• Machines are fast at churning this data
• Performing routine, repetitive and redundant tasks
• They can learn new skills quickly and in a distributed way
• They can operate non-stop

What humans do better

• Resolving ambiguous information
• Interpreting unusual and complex information
• Judgment in determining the payoff or utility of a particular outcome
• Making moral and ethical choices

📝 Distributed learning
Multiple machines can share information with each other instantaneously. This means that when one machine learns something new, all of them learn it. Machines can also spread out the learnings into chunks of information. 100 robots learning different parts of a problem in parallel for 1 hour is equivalent to each learning 100 hours worth of information in the same amount of time.

Machines helping humans

Machines are great at sifting through large amounts of data. This makes them great at detecting anomalies and generating predictions.

1. Detecting anomalies

Averaging lots of data points is one of the most powerful ideas in data science. To decide whether something is an anomaly you need to know what is expected on average and how far is the measured data point from this average a.k.a variability. In AI, this means scanning a stream of data points often in realtime and identifying those that don’t match the pattern. Successful detection can save money in case of credit card fraud detection and even lives in the case of detecting diseases.

2. Generating predictions

Most applications of AI do some sort of predictions based on previous data. Better predictions can enable new actions. Recommending what movies to watch, taking the next turn when driving, or suggesting the appropriate phrase when composing an email are all examples of predictions that AI systems make.

Humans helping machines

People are generally better than machines at making moral and ethical choices as well as analysing and judging complex problems. While an AI can predict better, a human can make better judgements.

1. Training

Much like a child learning a new skill, AI systems need to be trained and nurtured. This training is often done by humans. In the case of self-driving cars, the AI is shown examples of humans driving decently to figure out how to drive. The person’s job is to curate these examples and correct the AI in case it makes mistakes.

2. Making judgements

We use judgement to determine a payoff, utility, or reward from an outcome. AI predictions reduce uncertainty while judgement assigns value. Judgement is needed to make a decision from a prediction. However in case of predictions that lead to an obvious course of action, AI can make the final decision.

3. Making ethical choices

Humans are extremely valuable when making ethical and moral choices. Sometimes the most profitable course of action might not be the most ethical. Predictions that AI makes could lead to discrimination. Humans can help in evaluating the non-quantifiable impact of AI systems and making policies accordingly.

Cobots: Human-machine symbiosis

Often humans and machines are seen as adversaries fighting for each other’s jobs. This view however neglects the powerful opportunities in collaboration between the two sides. Imagine a flexible, nimbler robot that works alongside humans to help perform generic, repetitive tasks. Or an algorithm that produces a variety of design options based on the designer’s preferences and goals. The designer can simply select the most optimum option and start using it as a reference or straight up modify it. This idea is sometimes also referred to as ‘Organic AI,’ where machines are flexible and easily partner with humans to perform tasks. Such teams can easily adapt to new data and novel market conditions. The machine can become an extension of the human mind and body.

For a long time, dedicated pieces of machinery have performed specific tasks, often away from humans. With the rise in computing, improvements in software and hardware, people started using machines as tools for work and pleasure. However, these tools weren’t very smart and humans had to operate them largely through command and control type instructions. Press the button to start, press again to stop, pull the lever to pick up…

Now as our tools become more interconnected and intelligent, machines and humans start looking like symbiotic partners in the system. This symbiotic relationship can enable new business models, create jobs that never existed, and impact existing jobs.

References

1. Superminds: The Surprising Power of People and Computers Thinking
2. Together Human + Machine: Reimagining Work in the Age of AI
3. Prediction Machines: The Simple Economics of Artificial Intelligence

Originally published at https://woktossrobot.com on May 16, 2020.

When working with tech teams, your job is to help your engineers make great user-centred decisions. Designing ML systems is a creative process for its programmers too. Getting too prescriptive too quickly can inhibit the creativity of your engineering counterparts. Trust them to use their intuition, encourage experimentation and user testing. Training a model is a slow process and tools for visualisation aren’t great. So engineers frequently need to use their imaginations when tuning an algorithm. Try to inspire your tech teams with decks, stories, prototypes, videos to get them excited about the end goal. Your ability to influence the product effectively depends greatly on your engineering team being comfortable with iterations and is great for the robustness of your ML pipeline.

Further reading and references

https://woktossrobot.com/aiguide/references.html

Use this flowchart to decide whether you need to invest in AI/ML capabilities for your product.

A compilation of best practices for designers, managers and HCI practitioners to build human-centred AI products.

Designing AI products

How can designers work well with AI/ML Tech teams? was originally published in UX Planet on Medium, where people are continuing the conversation by highlighting and responding to this story.

In traditional software systems, outcomes are discrete and the creators are aware of the behaviour of these systems (rigid set of instructions). AI/ML-based systems bring a fundamental shift to this way of thinking where instead of programming a system to do a certain action, it’s creators provide data and nurture them to curate outcomes based on the input. These systems learn over time.

Systems with AI/ML components are probabilistic in nature and the product might work differently for different users. There are three key considerations that product creators need to keep in mind when building AI products:

1. Building trust with users of the AI system
2. Designing feedback — Helping users improve the AI model
3. Handling errors and failures — What to do when things go wrong

Building trust with users of the AI system

Because AI-driven systems are based on probability and uncertainty, it is important for users to understand how and when to trust the system. The right level of explanation is key to helping users understand how the system works. When users have the right level of control over the system, they can understand how and when to trust it to help accomplish their goals.

Explaining how the AI system works is key to building trust. When onboarding users it is important to explain what the system can do and how well can it do it.

Users shouldn’t implicitly trust your system, rather the system should help the user in assessing whether a result is trustworthy. There are cases where some users are completely averse to AI decisions while in others they are too trusting of the system. Ideally, both these extremes need to be avoided.

Build trust with the user’s of your AI system by:

• Explaining how the system uses data
• Explaining confidence of system results
• Partially explaining in cases where giving a full explanation would lead to confusion
• Reducing bias in your datasets

Explaining how the system uses data

Whenever possible, the AI system should explain the following aspects of data use:

Scope
Show an overview of the data being collected about the user and which aspects of the data are being used for what purpose. At the same time, the system should inform the user that a lack of data might mean that they need to use their own judgement. Counterfactuals may be used to explain why the AI did not make a certain decision or prediction.

Reach
Explain whether the system is personalized to the user (Spotify daily playlist) or aggregated (Amazon product suggestions) across all users.

Editability
Tell users whether they can remove, reset or edit the data being used by the AI system.

Explaining confidence of system results

Confidence levels are a readily available AI system output and are a great tool that can be used to explain the workings of an AI system. Confidence levels help users gauge how much trust to put in AI systems.

Confidence level is a statistical measurement that indicates how confident the AI system is about a particular outcome.

Let’s assume our biryani detector categorizes results in low, medium and high confidence levels. For each result type, we design it to answer differently. When you give it an image of biryani, it has a high level of confidence in its result and we design it to say “That’s biryani! Looks delicious.” When you give it an image of a pulao which somewhat resembles a biryani, we design it to say “Hey this might be biryani! You can tell me if it’s not.” When you give it an image of a pasta, we design it to say “That’s not biryani!.”

A confidence level is often a value between 0 (no confidence) and 1 (full confidence) which is converted into a percentage. In most real-world scenarios, you will never get a definite 0 or 1.

There are multiple ways of indicating confidence levels:
Because different user groups may be more or less familiar with what confidence and probability mean, it’s best to test different types of displays early in the product development process.

Categorical
This involves categorising confidence values into buckets (high, medium, low). The category information can be further used to render UI, alter messaging and indicating further action that the user needs to perform. Cutoff points for the categories are decided by the team creating the system.

N-best alternatives
Rather than showing one result with an explicit confidence level, in this method, multiple results are shown with their confidence levels indicated. E.g. This photo may contain a goat, llama or a ram. This approach is especially useful in low-confidence situations.

Numeric
This method utilises simple percentage values to indicate confidence levels. Showing numeric values is risky since it assumes that users have a baseline understanding of probability (is 80% confidence high or low?).

Partial explanation

There are cases when there is no benefit in providing an explanation or giving an explanation would lead to confusion. For e.g. When adjusting the screen brightness based on the environment, it might not make sense to prompt the user every time the system performs the action. Partial explanations intentionally leave out parts of the system’s workings that might be too complex, unknown or unnecessary. Partial explanations can be general or specific.

General
Explaining how the AI system works in general terms. E.g. This app recognizes food with an 80% accuracy in general.

Specific
Explaining why the AI provided a particular output at a particular time E.g. When recognizing hotdog, the system shows similar images to indicate what data the system used to recognize the food item.

Bias

Machine learning is not a pipeline, it’s a feedback loop. Bias in the AI result is a reflection of the bias present in the data used to train the AI. Training data, even if it is collected from the real world will amplify existing biases. Committing to fairness at each step of ML design is key to tackling bias, thereby building trust with users.
When using real world data, human bias will be invariably introduced due to personal experiences and inherent biases. These patterns might be amplified even further by the AI model. Often minimising bias comes down to reducing bias in the data used to train the AI.

How ML systems can fail users

• Representational harm
  When a system amplifies or reflects negative stereotypes about particular groups.
• Opportunity denial
  When systems make predictions and decisions that have real-life consequences and lasting impacts on individuals’ access to opportunities, resources, and overall quality of life.
• Disproportionate product failure
  When a product doesn’t work or gives skewed outputs more frequently for certain groups of users,
• Harm by disadvantage
  When a system infers disadvantageous associations between certain demographic characteristics and user behaviours or interests.

Recommended practices to reduce bias

There is no standard definition of fairness and no single correct model for all ML tasks. Here are a few recommended practices.

• Update training and test datasets frequently
• Engage with researchers, social scientists, designers and other experts to generate diverse perspectives.
• Use representative datasets for model training
• Public datasets can be especially biased and often need to be augmented to generate unbiased results. Understanding various perspectives is key to reducing bias.
• Collect feedback regularly

Designing feedback

Helping users improve the AI model

User Feedback

User feedback is the communication channel between your users, your product, and your team. Leveraging feedback is a powerful and scalable way to improve your technology, provide personalized content, and enhance the user experience.

Align feedback with model improvement
When users have the opportunity to offer feedback, they can play a direct role in personalizing their experiences and maximizing the benefit your product brings to them. It is important for creators to design feedback systems for tuning the model. Ideally, what people want to give feedback on aligns with what data is useful to tune your model. However, that’s not guaranteed. Find out what people expect to be able to influence by conducting user research. For example, when using a video recommender system, people may want to give feedback at a different conceptual level than the AI model understands. They may think “show me more videos about parenthood” while the model interprets “show me more videos by this creator”.

Reward function
An AI model is guided by a reward function, also known as ‘loss function’ which helps the model determine whether it’s results were correct or incorrect (or a percentage value of how correct was it). Think of it as a reward that the system receives every time it guesses correctly. The AI system will try to optimize for this reward function. The goal of the feedback mechanism is to feed into the reward function by collecting user input.

Types of feedback

Implicit
Data gathered about user interactions from product logs. This type of feedback is not asked but users need to be aware that the system is collecting this feedback and get their permission. User’s also need a way to view their logs and have an option to opt out. All this information can be provided in the terms of service.

Explicit
Users deliberately provide this feedback and is often qualitative in nature like whether a recommendation was useful or an answer was incorrect. E.g. forms, like, dislike, ratings, open text fields.

Dual
This type of feedback can contain both implicit and explicit signals. Liking a piece of content can signal an explicit positive response as well as tune the recommendation algorithm implicitly

Why users give feedback?

• Material rewards E.g. Cash rewards, coupons, gifts
• Symbolic rewards E.g. Virtual badges, social proof
• Personal utility E.g. Bookmarking, tracking runs
• Altruism. In many cases, people are just nice. E.g. Community building on Quora
• Intrinsic motivation E.g. Venting about a product on Twitter

Designing feedback

Most valuable ML systems evolve with their user’s maturity while using the system. Feedback needs to be mutually beneficial to your users and the model. You ideally want to provide clear mental models that encourage users to give feedback

Fail gracefully
Look at failures as opportunities to improve the system. Users will be disappointed the first time the system fails, however, if the mental model establishes the idea that the system improves over time, then the failure can establish a feedback relationship. Once this is established, failure can be perceived as something that is not only forgivable, but also as something they can help fix.

Align the perceived and actual value of giving feedback
The feedback mechanism should tell users why feedback needs to be given and what the benefit of it is. This also helps in avoiding meaningless responses.

Connect feedback with changes in the user experience
Acknowledging user feedback can build trust, however, it is even better if the product shows how the input influences the AI and their experience. E.g. Providing genre feedback can change your Spotify playlists

Provide editability
User preferences change over time. Feedback mechanisms should enable them to control and adjust the preferences they communicate to the ML model.

Handling errors and failures

What to do when things go wrong

For AI-based products, often the experience may differ from user to user. Thereby what defines an error would also differ. Users might test your product in ways you might never have imagined which might lead to false starts, misunderstandings and unexpected behaviours. Designing for these cases is critical when building systems with AI/ML components. At the same time, errors are also opportunities for feedback that can help the AI learn faster. Before you start designing how your system will respond to errors, try to identify errors that your users can perceive already, or that you can predict will occur.

Errors and failures can be of the following types:

• User perceived errors
• System errors that user’s don’t perceive
• Data errors
• Input errors
• Relevance errors

User perceived errors

In traditional software systems, there are mainly two types of errors; User errors (Users make an error. Creators blame the user) and System errors (Errors that arise due to a fault in the system. Users blame the creators.) AI systems introduce a third kind of error where the result doesn’t make sense in the user’s current context. The context can be the user’s preferences, lifestyle patterns or cultural values. E.g. A recipe app giving meat recipe recommendation despite consistently rejecting meat-based recipes. These user-perceived errors are often referred to as context errors. User perceived errors can be categorized as follows:

Context errors
The system is “working as intended,” but the user perceives an error because the actions of the system aren’t well-explained. For eg. a friend’s flight confirmation adds an event on your calendar.
Resolution: These errors can be tackled based on their frequency and severity by changing the system’s function to better align with user expectations. Another way could be to better explain the system’s workings during onboarding.

Fail states
The system fails to provide the right answer or any answer at all due to inherent limitations. For eg. An app that recognizes songs fails to find a song in a tribal language.
Resolution: In such cases, the system can ask the user to provide the correct answer which can be used to train the model.

System errors that user’s don’t perceive

These errors are invisible to users and need not be explained in the interfaces. However, being mindful of these would help train the AI.

Happy accidents
The system flags something as a poor prediction or an error, however users perceive these as an interesting feature. Capitalising on these might often lead to delight. E.g. Asking a smart speaker to water the plants gives a funny response.

Background errors
In these cases, neither the user or the system registers an error despite the system not working correctly. E.g. A search engine returning an incorrect result without being identified. Sadly there’s no easy way out of these error cases and can be tackled through internal testing.

Data errors

Data errors are system-level errors and are of the following types:

Mislabeled or misclassified results
The AI system outputs incorrect results due to mislabeled or misclassified due to poor training data.
Resolution: Allow users to give guidance or correct the data via feedback mechanisms that feed into the model to improve the dataset or alert the team.

Poor inference or incorrect model
The ML model is not precise despite giving sufficient data. E.g. A food classification app returns a large number of false positives for grapes despite giving it a sufficient data.
Resolution: Allow users to give guidance or correct the data via feedback mechanisms that feed into model and helps tune it.

Missing or incomplete data
Cases of missing or incomplete data are often observed when the model reaches the limits of its capabilities.
Resolution: Explain how the system works, it’s limitations and what is missing. Allow users to give feedback on the missing data.

Input errors

Input errors are often user-level errors and are of the following types:

Unexpected or incorrect input
When the user enters an incorrect input often assuming that the system would correct it. E.g. expecting an auto-correct while the system doesn’t correct.
Resolution: Check the user’s input with a range of correct answers and provide a suggestion. E.g. Did you mean this?

Breaking a habit
Change in UI causes a break in the habit of individuals using the feature leading to a different undesired result.
Resolution: Implement AI systems that don’t break habits. This can be done by assigning a location in the interface for low confidence AI output, allowing users to revert to defaults or retain a specific interaction pattern.

Miscalibrated input
When a system improperly weighs the importance of an action or choice. E.g. Liking a specific recipe calibrates the AI to show recipes by the same author instead of similar recipes.
Resolution: Explain how the system works, it’s limitations and what is missing. Allow users to give feedback to correct it.

Relevance errors

Relevance errors are often system-level errors and are of the following types:

Low confidence
When the AI system gives low confidence outputs leading to reduced accuracy. E.g. Hotel price prediction algorithm cannot give accurate predictions due to a changing political climate.
Resolution: Explain why this happened and what can be done E.g. We do not have sufficient information, maybe try again in a few days.

Irrelevance
When the system outputs a high confidence answer that is irrelevant.
Resolution: Allow users to provide feedback to improve the model.

Recap

When designing AI/ML systems, there are three key considerations that product creators need to keep in mind:

• Building trust with users of the AI system
• Designing feedback — Helping users improve the AI model
• Handling errors and failures — What to do when things go wrong

Further reading and references

https://woktossrobot.com/aiguide/references.html

Use this flowchart to decide whether you need to invest in AI/ML capabilities for your product.

A compilation of best practices for designers, managers and HCI practitioners to build human-centred AI products.

Designing AI products

Design for Artificial Intelligence was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

Prototyping for ML systems is especially hard. If the value of your product lies in using user data to build a personalised experience, a prototype with dummy data wouldn’t feel authentic. Also if you have a fully built ML system, it might be too late to make any meaningful changes. However, there are two prototyping approaches that work fairly well for ML systems:

Using personal examples

When prototyping with early mockups, ask your users to get personal data like contact lists, photos, favourite songs etc. to the sessions. This can be like a fun homework for the participants. This data can be used to simulate the responses from the system. However, do make sure that you inform the participants how the data would be used and if or when it would be deleted.

Wizard of Oz studies

Wizard of Oz studies have participants interact with what they believe is an autonomous system. For example, when testing a chat-based interface, a teammate can simulate answers from what would be the AI system. Having a teammate imitate actions like answering questions, give movie recommendations, etc. can simulate ‘intelligent’ systems. When participants engage with what they perceive to be AI, they tend to form mental models and adjust their behaviour accordingly. Observing these changes can be extremely valuable when designing AI systems.

Use this flowchart to decide whether you need to invest in AI/ML capabilities for your product.

A compilation of best practices for designers, managers and HCI practitioners to build human-centred AI products.

Designing AI products

Prototyping experiences for Artificial intelligence was originally published in UX Planet on Medium, where people are continuing the conversation by highlighting and responding to this story.