<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Marta Ratajski on Medium]]></title>
        <description><![CDATA[Stories by Marta Ratajski on Medium]]></description>
        <link>https://medium.com/@martawlusek?source=rss-fb62f4219807------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FMGMb3mlXCeKfr3_Kj4_NQ.png</url>
            <title>Stories by Marta Ratajski on Medium</title>
            <link>https://medium.com/@martawlusek?source=rss-fb62f4219807------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 02:49:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@martawlusek/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Designing AI-Powered Language Learning SaaS]]></title>
            <link>https://medium.com/@martawlusek/designing-ai-powered-language-learning-saas-75fbe54242c9?source=rss-fb62f4219807------2</link>
            <guid isPermaLink="false">https://medium.com/p/75fbe54242c9</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[product-design]]></category>
            <category><![CDATA[ai-design]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ux]]></category>
            <dc:creator><![CDATA[Marta Ratajski]]></dc:creator>
            <pubDate>Wed, 13 Mar 2024 09:05:47 GMT</pubDate>
            <atom:updated>2025-05-24T05:10:31.619Z</atom:updated>
            <content:encoded><![CDATA[<h3>Designing AI-powered Language Learning SaaS</h3><h4>How to design a voice-based conversational AI agent for language learners</h4><figure><img alt="AI Tandem — Foreign Language Conversational Voice Assistant" src="https://cdn-images-1.medium.com/max/1024/1*DoMD7EJs8SAYMQTgUZuytg.png" /><figcaption>AI Tandem — Foreign Language Conversational Voice Assistant. Image source: <a href="https://martawlusek.netlify.app/ai-tandem">Marta Wlusek</a></figcaption></figure><p>Technology evolves rapidly, and so do design principles and frameworks, born out of the need to help designers keep up with the pace of new technologies. Recently user experience and customer experience designers could have noticed a new trend in merging service design with product design. This is because many digital products, such as software, websites, and apps have embarked on new territories, including services as part of their product features, for instance, chatbots or AI assistants. On the other hand, services increasingly take advantage of digital tools to offer customers new experiences and analyse their level of satisfaction.</p><p>This trend mirrors the shift from goods-dominant (G-D) logic to service-dominant (S-D) logic. In a nutshell, Service-Dominant Logic is a theoretical framework whose main principle argues that “firms can only make value propositions and customers are active participants in coproduction” (Vink et al., 2021), or in other words, value is co-created through real-time interactions, meaning customer and provider work together to create value.</p><p>A B2C SaaS (Business-to-Consumer Software as a Service) is a great example of the aforementioned merger between products and services. B2C SaaS is, essentially, a business model that provides <em>to</em> customers over-the-internet (cloud-based), registration-based (predominantly subscription-based) services, which allow customers to access the businesses’ products and use them for their specific needs, to solve specific problems, or to achieve specified goals.</p><p>In this article, I will take an imagined B2C SaaS as a case study to explore how both service providers and customers can benefit the most from the implementation of S-D logic principles. Furthermore, I will dive deeper into the innovation opportunities in designing for SaaS through leveraging the power of data analytics and artificial intelligence.</p><h3>Designing a Service</h3><p>Let’s get started by making the terminology straight: <a href="https://www.youtube.com/watch?v=ojqN3tZqcew">service design is not the same as designing a service</a>. According to Nilsen Norman Group (2018), service design concentrates first and foremost on planning the company resources and optimising employees’ responsibilities in order to create the best possible customer experience. Designing a service, on the other hand, means creating a customer journey, i.e. the end-to-end of what the customer experiences. In this case study, I give priority to the latter.</p><p>When designing a service we have to keep in mind the timeline-based nature of services. Experiencing a service is an over-time process. One of the most suitable UX frameworks to visualise how customers are involved in this process, and what they experience at each step or touchpoint, is through mapping their customer journey (or user journey).</p><h3>Customer Journey Mapping</h3><p>First things first, we have to define who is going on the “journey” with a SaaS and what goal a person (in UX known as <em>Persona</em>) wants to accomplish.</p><p><em>Let me introduce Joanna, our Persona.</em></p><p>Joanna moved to Sweden about a year ago and started working as a breakfast prep cook in a hotel in Stockholm, while still looking for a job in marketing — a field she graduated from and gained professional experience. Her goal is to learn Swedish faster and without stress to speed up her professional development. She has already acquired a basic knowledge of Swedish but is not confident in speaking the language. Her pain point is the lack of Swedish friends and acquaintances with whom she could practice speaking. Additionally, at the hotel, she works in a team consisting entirely of immigrants, mostly from her country of origin, which makes learning the language at work nearly impossible. That is why she looks for alternative ways to learn and practice the Swedish language.</p><figure><img alt="AI Tandem — Customer Journey Map" src="https://cdn-images-1.medium.com/max/1024/1*_N_zis5sqYzu_Vk02eXtNA.jpeg" /><figcaption>Figure 1. AI Tandem — Customer Journey Map. Image source: <a href="https://martawlusek.netlify.app/ai-tandem">Marta Wlusek</a></figcaption></figure><h4>Step 1: Onboarding</h4><p>Joanna discovers an AI-powered SaaS named AI Tandem. As we can see on the journey map (Figure 1), Joanna ought to go first through an onboarding process in which she is prompted to answer a couple of questions about herself and her learning goal before she can fully dive in. This journey step ensures that her profile matches recommendations for AI Tandems she will be presented with and will choose from.</p><h4>Step 2: Initial engagement, feedback and customisation</h4><p>After selecting her AI Tandem and engaging in the first video call conversation, Joanna is asked to give feedback on whether her Tandem’s speech tempo and vocabulary matched or mismatched her language proficiency. At the same time, AI compares the user’s inputted data in personal settings with the actual conversation to measure and predict the most appropriate language learning experience for the user.</p><h4>Step 3: Personalised interaction</h4><p>Joanna interacts further with her AI Tandem, which adapts to her goals and behaviour with personalised content. Data is continuously collected from the user’s feedback and behaviour regarding content relevance, difficulty, and engagement. AI Tandem learns from the data about the user’s interests, which enables a personalised experience tailored to the user’s needs.</p><h4>Step 4: Progress tracking and feedback loop</h4><p>After some time spent using the app, Joanna can review her progress. Data analytics track the user’s interaction with the app over time and conduct progress evaluations, measuring the learner’s pronunciation accuracy, vocabulary expansion, and grammatical error rate. AI Tandem uses this information to adapt accordingly in real-time in the background, introducing continuously new vocabulary and raising the difficulty level.</p><h4>Step 5: Continuous development of language and friendship</h4><p>Joanna’s regular engagement in conversations with her AT Tandem has led to improving her Swedish language skills and gaining confidence in speaking it. She is now ready to start working as a marketing manager, nevertheless, she continues talking with her AI Tandem, who has meanwhile become her virtual friend. AI Tandem remembers her learning journey and offers continuous personalised conversations, inclusive of emotional support for further enhancement of the user’s language skills and personal growth.</p><h3>AI-powered SaaS</h3><p>AI Tandem is a learning AI Agent and voice-based conversation assistant designed for language learners. It features human-like<em> </em>speech, personalised difficulty levels, and adaptivity. The main advantage of AI Tandem is that it can recognise speech with different foreign accents and mistakes made by language learners. This is especially important to address the following pains:</p><ul><li>For language teachers or native speakers, it can be difficult to recognize mispronounced words or understand non-fluent language learners’ speech.</li><li>For non-fluent language learners, it can be difficult to understand native speakers or communicate effectively with others.</li></ul><p>This can cause frustration, embarrassment and anxiety, and limit opportunities for personal and professional growth. Existing AI-based apps like Replika provide conversation features and emotional support, but may not be trained to understand languages other than fluent English.</p><p>AI Tandem starts by recognising common pronunciation and grammar mistakes using predictive Large Language Models. It is trained on different languages’ grammatical rules and large amounts of speech data of language learners. This enables the real-time comparison of what the learner has said with the grammatical rules, allowing the understanding of a sentence or a word even though the learner mispronounced it. The transformer neural network-based language prediction models can identify and analyse patterns, for instance, that learners whose native language is A, make more frequently specific mistakes when speaking language B. For example, learners whose native language is Polish or Czech are more likely to use articles incorrectly or omit them altogether when speaking foreign languages that have indefinite and definite articles. Over time, AI Tandem learns from these mistakes, speech patterns, and user behaviour and adapts accordingly. This provides an adaptive and personalised experience for each user.</p><p>There is one challenge we cannot simply ignore when talking about the current technical state of LLMs. One of the main trade-offs for user experience, at the time I am writing this article, is the scalability and long latency of LLMs voice assistants on devices with limited memory, such as mobile phones (Alizadeh et al., 2023). Although, it is the speed and the scale at which the transformer large language models can function that defines their innovation and value for businesses and users.</p><p>In 2012 Pedro Domingos wrote in his journal paper “A Few Useful Things to Know about Machine Learning”:</p><blockquote>“In most of computer science, the two main limited resources are time and memory. In machine learning, there is a third one: training data. Which one is the bottleneck has changed from decade to decade. In the 1980’s it tended to be data. Today it is often time. Enormous mountains of data are available, but there is not enough time to process it, so it goes unused. This leads to a paradox: even though in principle more data means that more complex classifiers can be learned, in practice simpler classifiers wind up being used, because complex ones take too long to learn. Part of the answer is to come up with fast ways to learn complex classifiers, and indeed there has been remarkable progress in this direction”.</blockquote><p>Fast forward to 2024, we can ask whether Domingos meant Moore’s Law and the impending development of LLMs and generative AI, such as Generative Pre-trained Transformers (GPTs).</p><h3>Value co-creation</h3><p>Since every SaaS has the vast potential to analyse the enormous mountains of data that users create as a result of their interactions, the value of SaaS can include not only the provision of products but also the creation of personalised services for its users. This mutual creation by both the user and SaaS leads to what is, according to S-D logic, called: “value cocreation” (Helkkula et al., 2018).</p><p>In the case of our AI Tandem SaaS, the “value cocreation” unfolds through the co-creation of the learning process, where Joanna benefits from a personalised, adaptive language learning experience, while AI Tandem continuously learns and improves its teaching methods and content generation based on Joanna’s interactions. Ultimately, this feedback loop enhances the learning experience for both parties involved. By leveraging the data analytics and insights gained from it, the AI updates its models or algorithms, resulting in more accurate natural language understanding, better error correction, and the ability to offer tailored and accurate learning experiences for Joanna as well as other users, thus continuously improving their language acquisition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i4_s71WSpW7xYXY9YK8w3w.png" /><figcaption>Figure 2. Designing Feedback Loop. Image source: <a href="https://martawlusek.netlify.app/ai-tandem">Marta Wlusek</a></figcaption></figure><h3>Artificial Intelligence vs Human Intelligence</h3><p>Artificial Intelligence (AI) that mimics Human Intelligence (HI) is a key driver of innovation and is increasingly applied in service nowadays (Huang, 2018).</p><p>But what exactly defines HI? A quick glance at the Encyclopedia Britannica reveals researchers’ recent stance that the defining characteristics of human intelligence are “the abilities to learn from experience, adapt to new situations, understand and handle abstract concepts, and use knowledge to control an environment.”</p><p><em>Adaptability</em> and <em>learnability</em> are significant aspects of AI. They are especially crucial to Adaptive AI, which continuously analyses input data to identify patterns for continuous learning and adaptivity. It is also important to Generative AI, which, in addition to the above, attempts to mimic human intelligence by producing human-like content based on learned patterns.</p><p><em>AI Tandem utilizes both the adaptive and generative abilities of artificial intelligence.</em></p><p>Moreover, the AI Tandem idea is to create a personal tutor or virtual foreign language friend designed to act like a human. Even more, to outperform a human! Looking back at the pain points mentioned earlier in the article, it becomes clear that the AI Tandem’s job should be to understand even the most mispronounced words with strong accents and incorrect grammar, while humans would have difficulty understanding the same, or in the worst-case scenario, they would have no clue what the learner was trying to communicate. Being a language learner myself, I have experienced countless times how teachers have struggled to decode incorrectly pronounced words by me or by other students. And I would say, it is a rather embarrassing experience for both sides.</p><p>But would both sides benefit from the release of AI Tandem to the world? While the potential benefits of such service for the learners become to me apparent, I do not see such a win for the teachers. Neither AI Tandem offers language tutors assistance, nor augments some of their job’s tasks so they could concentrate on other more creative or interesting parts of their role.</p><p>According to recent studies regarding employee labour replacement, the progression of AI in mastering HI follows a sequential path, with AI initially replacing mechanical tasks, then analytical and intuitive, and eventually progressing to the overtaking of empathetic tasks (Huang &amp; Rust, 2018). Empathetic intelligence, considered the most advanced form of artificial intelligence, enables AI-powered services to encompass both front-end interactions, such as emotional, communicative, social, interactive, and relational, as well as back-end support, such as analytics for emotional user experience and engagement (Huang &amp;Rust, 2018). The emergence of applications like AI Tandem, aiming at mimicking human-like conversations and emotional understanding, spotlights the potential problem of AI in replacing empathetic jobs.</p><p>Add to this the ability of AI to constantly improve with experience and constant connection to the cloud, as well as our unique human abilities, and we shall soon become co-creators of innovative services that will result in the emergence of the phenomenon known as collective intelligence.</p><p><strong>References:</strong></p><p>Alizadeh, K., Mirzadeh, I., Belenko, D., Khatamifard, K., Cho, M., Del Mundo, C. C., … &amp; Farajtabar, M. (2023). LLM in a flash: Efficient Large Language Model Inference with Limited Memory. <em>arXiv preprint arXiv:2312.11514</em>.</p><p>Domingos, P. (2012). A few useful things to know about machine learning. <em>Communications of the ACM</em>, <em>55</em>(10), 78–87.</p><p>Helkkula, A., Kowalkowski, C., &amp; Tronvoll, B. (2018). Archetypes of service innovation: implications for value cocreation. <em>Journal of Service Research</em>, <em>21</em>(3), 284–301.</p><p>Huang, M. H., &amp; Rust, R. T. (2018). Artificial intelligence in service. <em>Journal of Service Research</em>, <em>21</em>(2), 155–172.</p><p>Nielsen Norman Group. (2018, March 30). <em>Service Design 101</em> [Video]. YouTube. <a href="https://www.youtube.com/watch?v=ojqN3tZqcew">https://www.youtube.com/watch?v=ojqN3tZqcew</a></p><p>Vink, J., Koskela-Huotari, K., Tronvoll, B., Edvardsson, B., &amp; Wetter-Edman, K. (2021). Service ecosystem design: Propositions, process model, and future research agenda. <em>Journal of Service Research</em>, <em>24</em>(2), 168–186.</p><p><strong>More information about this project, including a prototype, is available on my portfolio website: </strong><a href="https://martawlusek.netlify.app/ai-tandem">https://martawlusek.netlify.app/ai-tandem</a></p><p>© Marta Wlusek. All rights reserved.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75fbe54242c9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The risk of algorithmic biases to inclusion and accessibility in AI-powered workplace tools]]></title>
            <link>https://medium.com/@martawlusek/the-risk-of-algorithmic-biases-to-inclusion-and-accessibility-in-ai-powered-workplace-tools-4e3e789cc62f?source=rss-fb62f4219807------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e3e789cc62f</guid>
            <category><![CDATA[accessibility]]></category>
            <category><![CDATA[inclusion]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-bias]]></category>
            <category><![CDATA[ui-ux-design]]></category>
            <dc:creator><![CDATA[Marta Ratajski]]></dc:creator>
            <pubDate>Fri, 14 Jul 2023 17:09:21 GMT</pubDate>
            <atom:updated>2024-03-14T22:17:11.416Z</atom:updated>
            <content:encoded><![CDATA[<p>AI is here to stay, and it will soon come to our workplaces, introducing new “super tools” that will enable us to perform our jobs faster and more efficiently. It may become part of our project or product management software, affording us to interact with the tool differently. The new interactions will rely on natural language, such as conversational interfaces, chatbots, or voice commands. Tasks that we are used to doing manually — by clicking on buttons, selecting options from dropdown lists, or navigating menus — will be done by communicating with an AI assistant as if we were interacting with a human, either in writing or orally. We may need to ask the AI assistant to update statuses, assign tasks, or request information. Clear and effective communication will be essential to populate inputs correctly and obtain accurate responses.</p><p>It won’t be the boss who decides whether the communication is clear and effective; it will be the algorithm.</p><blockquote>The quality of an AI algorithm depends on the dataset it has been trained on. If the data is limited or <strong>rigid</strong>, there is a risk of generating a biased algorithm.</blockquote><p>For instance, if a workplace introduces a tool that relies on written or spoken communication for operation, there is a risk of affecting non-native speakers due to algorithmic bias. If the AI-powered tool has been trained for interactions in a single language, considering only typical pronunciations and accents, it can make tasks for non-native-speaking employees more burdensome or challenging.</p><figure><img alt="Illustration of Cognitive Bias Codex by John Manoogian III and Buster Benson." src="https://cdn-images-1.medium.com/max/1024/1*iaudVu0U9kVzZGPMEt0ADw.jpeg" /><figcaption>How humans create biases. Image source: <a href="https://upload.wikimedia.org/wikipedia/commons/1/18/Cognitive_Bias_Codex_-_180%2B_biases%2C_designed_by_John_Manoogian_III_%28jm3%29.jpg"><strong>Wikimedia</strong></a><strong>.</strong></figcaption></figure><p>In such cases, although the data used to train the model may not be biased, it represents only a specific group, excluding others, thus leading to algorithmic biases when confronted with the underrepresented group. A similar bias was found in a study involving non-native English speakers who were wrongly diagnosed with early-stage Alzheimer’s disease by an AI algorithm because the training dataset consisted solely of native English speakers’ speech samples (1). In such cases, excluding non-native English-speaking participants from the study would not do much harm and could even help obtain more accurate results. However, in the workplace, excluding non-native speakers from the workforce is clearly not an option. So is flagging non-native English students’ essays as ChatGPT-written because they did not score as well as their native English-speaking colleagues in ‘language sophistication’ (typical for mother tongue users) — a criterium set by a biased algorithm in AI writing detectors (2).</p><figure><img alt="Illustration of simple sampling." src="https://cdn-images-1.medium.com/max/618/1*xnrqALQF5YI1YnZLyhGSOQ.png" /><figcaption>Obtaining a representative sample of a population. Image source: <a href="https://en.wikipedia.org/wiki/Bias#/media/File:Simple_random_sampling.PNG"><strong>Wikipedia</strong></a>.</figcaption></figure><p>Another important aspect to address is the accessibility of AI-powered tools. If the tools are not designed with accessibility in mind, they may occur to be inaccessible to employees with disabilities, adding another layer of obstacles to performing their jobs. For instance, if an AI tool requires writing as the main part of its operation or relies heavily on visual elements without providing alternative text or audio cues, visually impaired individuals may face difficulties in using it. Analogically, if a voice conversation feature is introduced in place of manual interactions, completing tasks for employees with a hearing impairment or speech impairment, who until now have been doing well interacting with a tool conventionally, will become nearly impossible.</p><p>Individuals from both underrepresented groups are also more likely to fall victim to hiring bias in AI-powered hiring tools, which may contain discriminatory data, disadvantaging candidates with non-native sounding names and people with disabilities if they are not adequately represented in the company’s historical data of successful employees (3).</p><p>To ensure that some employees do not feel underrepresented or excluded from the workforce, we have to design AI-powered tools for inclusion and accessibility.</p><p>To mitigate the risk of biases in AI tools we should take at least these 3 steps in the AI design process:</p><h4><strong>1. Making sure the training dataset is more diverse and includes underrepresented groups in the data samples</strong></h4><p>Incorporating a more inclusive dataset into the training process can help the AI algorithm become better equipped to evade bias and promote diversity.</p><h4>2. Testing and evaluating algorithms for biases before publishing models</h4><p>To identify any instances where a model might favor certain groups or exhibit inaccuracies that disproportionately impact specific groups.</p><h4>3. Providing alternative and accessible user interfaces or/and multilingual support for interaction with AI models</h4><p>Both non-native and disabled employees would greatly benefit from having the option to use manual interfaces. In addition to that, multilingual support would enable the non-native-speaking workforce to use their native language when communicating with AI. For example, in a voice conversation, if the user does not speak English on a native level (e.g. Swedish pronunciation of the English word “general”, i.e. “yeneral”, would perhaps not be recognized by an AI unless trained on data that includes such samples. However, over time, the AI, like humans, could learn this pattern).</p><p>By taking these actions into account, we can mitigate the risk of biases and create AI-powered workplace tools that promote inclusion, accommodate diverse needs, and empower all employees. The integration of AI-powered tools in our workplaces holds immense potential for improving efficiency and productivity, and if designed well, AI can even aid in addressing different users’ specific needs.</p><p>Below is an example of well-designed accessibility and inclusion features:</p><p><a href="https://uxdesign.cc/what-designers-can-learn-from-apples-latest-accessibility-features-e16b49870e74">What designers can learn from Apple’s latest accessibility features</a></p><p>With a human-centric approach to designing AI-powered tools and with inclusion and accessibility in mind, we can harness the potential of AI to foster an environment where everyone can thrive and contribute their unique perspectives and abilities.</p><p><strong>References:</strong></p><ol><li><a href="https://www.quantib.com/blog/understanding-the-role-of-ai-bias-in-healthcare">https://www.quantib.com/blog/understanding-the-role-of-ai-bias-in-healthcare</a></li><li><a href="https://hai.stanford.edu/news/ai-detectors-biased-against-non-native-english-writers">https://hai.stanford.edu/news/ai-detectors-biased-against-non-native-english-writers</a></li><li><a href="https://abilitymagazine.com/does-artificial-intelligence-impact-hiring-people-with-disabilities/">https://abilitymagazine.com/does-artificial-intelligence-impact-hiring-people-with-disabilities/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e3e789cc62f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How human-centered design can help us create trustworthy AI]]></title>
            <link>https://medium.com/@martawlusek/can-we-trust-ai-c379b9ef81c?source=rss-fb62f4219807------2</link>
            <guid isPermaLink="false">https://medium.com/p/c379b9ef81c</guid>
            <category><![CDATA[trustworthy-ai]]></category>
            <category><![CDATA[human-centered-design]]></category>
            <category><![CDATA[human-centered-ai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ux]]></category>
            <dc:creator><![CDATA[Marta Ratajski]]></dc:creator>
            <pubDate>Sat, 15 Apr 2023 08:52:23 GMT</pubDate>
            <atom:updated>2025-10-30T06:22:53.497Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="A human’s hand reaching out to a robot’s hand." src="https://cdn-images-1.medium.com/max/1024/1*WS8w8kzYmRfkawll4pes8w.jpeg" /><figcaption>Photo by Tara Winstead. Image source: <a href="https://www.pexels.com/photo/person-reaching-out-to-a-robot-8386434/">Pexels</a></figcaption></figure><p>AI has enormous potential to enhance human knowledge and experiences and make our work and daily lives easier. However, there is also growing distrust in this technology and its presumed ability to replace human labour and take over decision-making.</p><blockquote>“Some even worry that in the future AI systems may completely reject human beings, who will eventually lose control.” — <em>Wei Xu</em></blockquote><p>Unknowingly to say whether this concern will ever be valid, one issue is already present: the problem of trust. Thus, the trust issue is nowadays one of the main challenges to address when designing AI-powered digital services and products. The question then arises of how to design AI-based solutions that are ethical, inclusive, fair and transparent, responsible and understandable, and are not intended to replace humans, thus fostering a sense of trust in end-users and stakeholders.</p><p>Instead of rushing to answer the posed question, let us first break down the problem of trust in AI in more detail. In the research paper by Väänänen et al. (1), trustworthiness is defined “as the confidence of whether AI will act as intended when facing a given problem.” The core technology of AI, machine learning, and its learning process (especially deep learning) are often considered opaque to non-technical users. The results coming from AI-based decision-making are not easily understood, leading to what is referred to as the black-box effect (2). The lack of transparency in the black-box phenomenon, in which the inner operations of AI are not explainable or comprehensible to users, raises concerns about accountability and fairness and leads users to question the decisions made by the system, impairing their confidence and trust.</p><p>An interesting example of the application of AI in the industry, where trust has a crucial role, can be found in the case study designed for the healthcare system and written by Wärnestål (3). The author proposed the design idea for the development of AI-powered technology to improve the triage process for busy and stressed Emergency Room nurses. The implementation would involve placing “triage cameras” in self-service booths in the ER waiting rooms, allowing for faster assessments of patients.</p><p><a href="https://uxdesign.cc/how-to-design-ai-powered-services-lessons-from-the-emergency-room-e5a7468674a">How to design AI-powered services — lessons from the emergency room</a></p><p>The main takeaway from the design sprint was the finding that the implementation of the AI-powered cameras in ER clinics is challenging because the quality of nurse-patient interaction mediated by a machine seemed to negatively impact the quality of care provided. Patients’ main concern was to rely on the machine’s decision-making efficiency: “What if the camera says I’m not ill?”. Nurses, on the other hand, expressed the concern that the machine would take over their “clinical gaze”, the skill deeply rooted in the years of their professional practice and experience, which allows them to independently assess the gravity of the patient’s situation.</p><p>Trust scepticism is also likely to be raised regarding the storage of personal data and its security. Keeping up with regulations, such as the European Union’s General Data Protection Regulation (GDPR), and ensuring that AI-powered products and services comply with them can be a significant obstacle to the release or utility of AI-powered solutions.</p><h3>Is a human-centered approach to designing the solution?</h3><p>Imagine the scenario that an AI-powered “triage camera” was developed by AI engineers without taking into account any human-centered design approach or process. The engineers fell in love with their idea of the value they could create by enabling state-of-the-art machine learning technology (3). They proudly presented the result of their hard work, which would significantly speed up the triage process, and then mounted the cameras in self-service booths in the ER, waiting for the nurses to arrive at work and start taking advantage of them. What could possibly go wrong? Wärnestål argues that this is not how value is created, “we need to understand how technology is <em>used</em>. (…) Value occurs when the design takes skills, workflow, and user experience into account.” (3)</p><p>It seems like the triage camera engineers would fit well with the concept coined by Maeda (4): “Temple of Tech” (vs. “Temple of Design”), where the emphasis is on technological innovation and development, at the expense of the user experience. In the Temple of Tech, technology is seen as an end in itself, rather than a means to an end. The focus is on building more advanced and complex technologies, often with little regard for how they will actually be used or by whom. Similar to the emergence of computers in the 1980s, when users were mostly programmers who prioritized technical factors over usability, likewise today, AI research focuses primarily on technological aspects, leading repeatedly to usability problems (2).</p><p><a href="https://howtospeakmachine.com/2019/11/17/the-temple-of-design-doesnt-rule-this-century/">Excerpt: The Temple of Design doesn&#39;t rule this century.</a></p><p>Therefore, user control and human-machine collaboration are important factors from a human-centered perspective to be considered when designing AI-powered services and experiences. By user control, I mean here both taking into account user needs as well as user autonomy. On one hand, users should be able to have influence over what is for them developed (1). On the other hand, they should have control over the user interface, whether it is customizing their settings and preferences (and through it gaining more understanding of how the AI system works), or whether it is deleting their data. Also, having the perception by end-users and stakeholders of being in control would help mitigate the trust problem.</p><p>The next important factor, which establishes common ground for human-machine collaboration, is people-machine interdependencies known as “teaming” (5). It presupposes that both parties have much better capabilities to improve performance when collaborating together than what each party could achieve on its own. This requires adopting a design approach for mutual complementation, instead of replacing a human. As long as we will not delegate all tasks to machines, we will stay in control. This is how the implementation of a human-centered design approach can contribute to the development of more trustworthy AI.</p><h4>References:</h4><ol><li>VÄÄNÄNEN K., ALA-LUOPA S., LEHTIÖ A., OLSSON T. (n.d.). Towards Human-Centered AI that Respects Human Autonomy and Sustainability. Available at: <a href="https://cris.tuni.fi/ws/portalfiles/portal/42143780/NordiCHI_WS_HCAI_KVetal_refs.pdf">https://cris.tuni.fi/ws/portalfiles/portal/42143780/NordiCHI_WS_HCAI_KVetal_refs.pdf</a></li><li>Xu, W. (2019). Toward human-centered AI: a perspective from human-computer interaction. interactions, 26(4), 42–46.</li><li>Wärnestål, P. (2022). How to design AI-powered services: Lessons from the emergency room. UX Collective, [online]. Available at: <a href="https://uxdesign.cc/how-to-design-ai-powered-services-lessons-from-the-emergency-room-e5a7468674a/">https://uxdesign.cc/how-to-design-ai-powered-services-lessons-from-the-emergency-room-e5a7468674a/</a></li><li>Maeda, J. (2019). How to Speak Machine: Laws of Design for a Digital Age. Penguin Random House.</li><li>Johnson, M., &amp; Vera, A. (2019). No AI is an island: the case for teaming intelligence. AI Magazine, 40 (1), pp. 16–28.</li></ol><p>If you found this article informative, you may also be interested in checking out another article where I explore more challenges in designing for AI:</p><p><a href="https://bootcamp.uxdesign.cc/designing-for-ai-a6abf086b164">Designing for AI</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c379b9ef81c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Designing for AI]]></title>
            <link>https://medium.com/@martawlusek/designing-for-ai-a6abf086b164?source=rss-fb62f4219807------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6abf086b164</guid>
            <category><![CDATA[human-centered-design]]></category>
            <category><![CDATA[interaction-design]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ux-design]]></category>
            <dc:creator><![CDATA[Marta Ratajski]]></dc:creator>
            <pubDate>Sun, 19 Mar 2023 17:05:27 GMT</pubDate>
            <atom:updated>2025-02-27T17:04:42.032Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>4 challenges for designers when designing AI-powered products and services</em></h4><p>The advent of Artificial Intelligence (AI) has revolutionized the way people interact with computing systems. From voice assistants to self-driving cars, AI-powered technologies have been having an increasing influence on various aspects of our lives. As a result, interaction designers should now go beyond their usual scope of designing for non-AI computing systems and start incorporating design processes and methods for AI-powered systems. In doing so, not only will they stay competitive in their field, but they will also help maximize the benefits of AI technology while minimizing the risks AI can pose to people.</p><p>At this point, it is becoming increasingly important for the design community to understand the unique challenges posed by AI systems in order to establish a viable approach to human-centered artificial intelligence (HCAI or HAI) design. In this article, I aim to explore some of the challenges of AI that pose the biggest hurdles for interaction design professionals when designing AI-powered products and digital services.</p><h3><strong>1. Non-Deterministic Behaviors</strong></h3><p>The first significant challenge that designers face is designing for non-deterministic and unexpected behaviors of AI-powered systems. Unlike non-AI computing systems, which are deterministic in nature and are based on fixed programmed rules or algorithms (such as conventional software), AI systems are based on machine learning, which can lead to unpredictable behaviors. The unpredictable nature of machine behavior of AI systems poses a challenge for designers since it can cause biases, dangerous mistakes, and undesirable outcomes that could have negative societal impacts (Xu et al., 2023).</p><p>The confusion matrix is used to illustrate how AI can produce false positive and false negative predictions that cause unexpected outcomes or errors.</p><figure><img alt="A table called the confusion matrix that displays the number of true positives, true negatives, false positives, and false negatives, where the predicted values are compared to the actual values." src="https://cdn-images-1.medium.com/max/1024/1*K6XTe_9kUtWQxTLiG9QImg.png" /><figcaption>A <a href="https://en.wikipedia.org/wiki/Confusion_matrix"><strong>confusion matrix</strong></a>. Image: Marta Wlusek</figcaption></figure><p>Therefore, designers should participate in the creation of explainable AI (<a href="https://medium.com/ibm-data-ai/building-explainable-ai-applications-with-question-driven-user-centered-design-36b71f15e506">XAI</a>) to ensure that users understand what the machine “is thinking” and why. The goal is to design an artificial intelligence system that is aware of its level of competence/incompetence to inform the user or to signal the operator of the system to take the initiative.</p><h3><strong>2. Absence of Human-Centered Design in AI Development</strong></h3><p>Another challenge is the absence of human-centered design processes among AI engineers, who often claim to be capable of designing interactions by themselves (Xu et al., 2023). To address this issue, AI engineers should collaborate with designers to leverage their skills and knowledge to design AI systems that align with human-centered design principles. One possible solution would be to use modeling diagrams in the requirements stage of the AI design process to represent the flow of information and data between different components of the system, including user interfaces, AI algorithms, and data sources. This could help designers better understand the system’s workings, enabling them to incorporate human-centered design processes specific to AI systems. Furthermore, it could facilitate communication among designers, developers, and data scientists throughout the development process. In the prototyping stage, designers should work with real data (e.g., through APIs) rather than relying solely on the “<a href="https://www.interaction-design.org/literature/article/prototyping-learn-eight-common-methods-and-best-practices">Wizard of Oz</a>” approach.</p><h3><strong>3. Unintentional Interactions</strong></h3><p>Unlike conventional non-AI interfaces, which are rather inflexible in the sequence of instructions they accept and the level of detail they take as input, AI-based technologies, such as gesture recognition, motion recognition, speech recognition, and even emotion, intent, or context detection (Xu et al., 2023), are more flexible than non-AI interfaces. However, this also means that users might trigger AI systems unintentionally. Therefore, designers should not only design for intentional and conscious interactions but also for unintended ones. This is especially important in avoiding accidental triggering of AI systems, which can have undesired consequences.</p><h3><strong>4. Human-Machine Collaboration</strong></h3><p>Designing for collaboration between users and AI is also a major challenge for designers. Some researchers suggest that future HCI (Human Computer Interaction) work should explore whether humans and AI agents ( i.e., autonomous, decision-making AI systems) can actually work together as real team members, or whether an AI agent should merely serve as a “super tool” (Xu et al., 2023) or a “butler” (Noessel, 2017). Therefore, designers need to design for humans-machine hybrid intelligence (Xu et al., 2023), human-machine teaming (Johnson et al., 2019), human-machine process (De Cremer et al., 2021), or human-AI co-creation. This requires developing methods for designing autonomous intelligent systems that not only leverage knowledge from human-machine interaction but also from the psychology of human-human interactions, such as teamwork, collaboration, trust, autonomy, authority, etc.</p><h3><strong>Conclusion</strong></h3><p>Designing AI-powered systems requires a different approach than designing non-AI computing systems. Moreover, designing human-centered AI systems presents a number of unique challenges for interaction designers. These include designing for non-deterministic and unexpected AI system behaviors, designing for the inclusion of human-centered design processes in AI development, designing for intentional and unintentional interactions, and designing for human-machine collaboration. To overcome these hurdles, designers must be aware of these challenges and use them as an opportunity to incorporate relevant design processes and methods specific to AI systems to help shape the future of AI-powered products and digital services that are human-centered.</p><p><strong>Bibliography:</strong></p><p>De Cremer, D., &amp; Kasparov, G. (2021). AI Should Augment Human Intelligence, Not Replace It. Harvard Business Review. Retrieved from: <a href="https://hbr.org/2021/03/ai-should-augment-human-intelligence-not-replace-it">https://hbr.org/2021/03/ai-should-augment-human-intelligence-not-replace-it</a></p><p>Johnson, M., &amp; Vera, A. (2019). No AI Is an Island: The Case for Teaming Intelligence. <em>AI Magazine</em>, <em>40</em>(1), 16–28. <a href="https://doi.org/10.1609/aimag.v40i1.2842">https://doi.org/10.1609/aimag.v40i1.2842</a></p><p>Noessel, C. (2017). Designing Agentive Technology. Rosenfeld.</p><p>Wärnestål, P. (2022). <a href="https://www.studentlitteratur.se/kurslitteratur/teknik-datorer-it-och-bygg/webbutveckling-och-anvandbarhet/designing-ai-powered-services/">Designing AI-Powered Services</a>. Lund: Studentlitteratur. ISBN 978–91–44–16537–0.</p><p>Wei Xu, Marvin J. Dainoff, Liezhong Ge &amp; Zaifeng Gao (2023) <a href="https://www.tandfonline.com/doi/abs/10.1080/10447318.2022.2041900?journalCode=hihc20">Transitioning to Human Interaction with AI Systems: New Challenges and Opportunities for HCI Professionals to Enable Human-Centered AI</a>, International Journal of Human–Computer Interaction, 39:3, 494–518, DOI: 10.1080/10447318.2022.2041900</p><p><strong>Disclosure:</strong></p><p>As a non-native English speaker, I use the assistance of artificial intelligence to correct my writing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6abf086b164" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>