<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Erica Kochi on Medium]]></title>
        <description><![CDATA[Stories by Erica Kochi on Medium]]></description>
        <link>https://medium.com/@ericakochi?source=rss-77aea2876777------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*hBxi9SviKHaR1yGl.jpeg</url>
            <title>Stories by Erica Kochi on Medium</title>
            <link>https://medium.com/@ericakochi?source=rss-77aea2876777------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 00:20:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ericakochi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Prevent Discriminatory Outcomes in Machine Learning]]></title>
            <link>https://medium.com/@ericakochi/how-to-prevent-discriminatory-outcomes-in-machine-learning-3380ffb4f8b3?source=rss-77aea2876777------2</link>
            <guid isPermaLink="false">https://medium.com/p/3380ffb4f8b3</guid>
            <category><![CDATA[human-rights]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[discrimination]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Erica Kochi]]></dc:creator>
            <pubDate>Thu, 22 Mar 2018 17:46:28 GMT</pubDate>
            <atom:updated>2018-03-22T19:29:31.314Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SYHnj54jEQtgVR1xSclwVw.jpeg" /><figcaption>Photo: ©UNICEF/UN0154769 Design: ©UNICEF/TusharGhei/2018</figcaption></figure><h3>Opportunity: Automated</h3><p>The opportunities that artificial intelligence (AI) can unlock for our world — from <a href="http://www.newsweek.com/2017/10/20/using-artificial-intelligence-find-cancer-cures-682477.html">discovering cures to diseases</a> that kill millions each year to significantly <a href="https://www.forbes.com/sites/bernardmarr/2018/02/21/the-amazing-ways-we-can-use-ai-to-tackle-climate-change/#765bcb104a35">reducing carbon emissions</a> — are expanding every day — and is already enabling pathways to <a href="https://www.thefsforum.co.uk/knowledge-centre/debate/application-ai-financial-inclusion/">financial inclusion</a>, <a href="https://ash.harvard.edu/files/ash/files/artificial_intelligence_for_citizen_services.pdf">citizen engagement</a>, <a href="https://medium.com/@teamrework/can-ai-make-personalised-healthcare-affordable-d0036537a05f">more affordable healthcare</a>, and many more vital systems and services. The same types of machine learning systems that might have highlighted a certain post in your Facebook newsfeed based on your online activity are being leveraged, for instance, to highlight certain applicants in a hiring process.</p><blockquote>Machine learning is one of the most powerful tools humanity has created- and it is more important than ever that we learn how to harness that power for good.</blockquote><p>While public attention often focuses either on the existential threats artificial super-intelligence poses to humanity (“the robots are coming to kill us”), or the opposite salvation narrative (“ AI will solve all our problems”) there is a more immediate-but less visible- risk that our reliance on ML-driven decision making poses in terms of the reinforcement of systemic bias and discrimination. Machine learning technologies are already making life-altering decisions for human lives on a daily basis. Examples come from the <a href="https://www.nytimes.com/2017/08/24/nyregion/showing-the-algorithms-behind-new-york-city-services.html"><em>New York Times</em></a>: “Algorithms can decide<a href="http://www.dnainfo.com/new-york/20161115/kensington/nyc-high-school-admissions-ranking"> where kids go to school</a>…<a href="http://www1.n/"> where building code inspections</a> should be targeted, and even what<a href="http://garyrubinstein.teachforus.org/2012/02/28/analyzing-released-nyc-value-added-data-part-2/"> metrics are used to rate a teacher</a>.”</p><p>As we empower machines to make critical decisions about who gets included and excluded from these types of vital opportunities, we need to be aware, cautious and deliberate to prevent discriminatory outcomes. After all, machine learning is only a tool, and <strong>the responsibility falls on </strong>people leading the way in the advancement of ML development, from corporate leaders down to system engineers to design and use machine learning applications in a way that not only improves business efficiency but also promotes and protects human rights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FPI4NCV5tYazBgJJ0Ns8bw.jpeg" /><figcaption>Photo: ©UNICEF/UNI99616/Siddique, Design: ©UNICEF/TusharGhei/2018</figcaption></figure><h3>What happens when machines learn to discriminate?</h3><p>Most of the stories we’ve heard about discrimination in machine learning come out of the US/ European contexts. Events like a<a href="http://www.huffingtonpost.com/2015/07/02/google-black-people-goril_n_7717008.html"> Google photo </a>mechanism that mistakenly characterized an image of two black friends as gorillas, as well as predictive policing tools that have been shown to <a href="https://hrdag.org/2016/10/10/predictive-policing-reinforces-police-bias/">amplify racial bias</a> have received extensive and important media coverage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wwyYddH6u6G31Cznlzq1rg.jpeg" /><figcaption>While the practice of using technology to automate decision making is not new, machine learning involves new dimensions of complexity, opaqueness, ubiquity, and exclusiveness that amplify longstanding problems related to unequal access. ©UNICEF/TusharGhei/2018</figcaption></figure><p>In many parts of the world, particularly in middle and low income countries, the implications of using Machine Learning to make decisions that fundamentally affect people’s lives without taking adequate precautions to prevent discrimination is likely to have far reaching, long-lasting, and potentially irreversible consequences.</p><p>Take for instance, any one of the following examples:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VnsSp7f2auBJBSa3nedATA.png" /><figcaption>Examples from around the world. ©UNICEF/TusharGhei/2018</figcaption></figure><blockquote>We need to look closely at the ways discrimination can creep into ML systems, and the ways that companies can act proactively to secure a bright future for ML.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zrmAgoIL7YcsQzhZQsYdvQ.png" /><figcaption>©UNICEF/TusharGhei/2018</figcaption></figure><h3>Where do machines learn to discriminate? Where does this risk come from, and why should we care?</h3><h3>It all starts with data</h3><p>Training data is what teaches machine learning applications how to classify, group, and predict. It’s these lessons that guide ML applications to make decisions. In order for machines to learn accurate and fair lessons, training data must be:</p><ol><li><strong>Adequately available and representative</strong></li><li><strong>Unbiased and error-free</strong></li></ol><p><strong><em>Consider the following scenario:</em></strong> In Kenya, micro loan company Tala’s smartphone app collects data on loan applicants including the number of people they contact daily, their movements and routine habits, like whether they call their mother every day or pay their bills on time. Tala suggests that using these inputs to gauge credit risk offers an alternative pathway to credit for those who lack a credit history. However, there are risks: as rural Kenyans have less digital infrastructure and fewer opportunities to develop a digital footprint, might they be unfairly excluded by algorithms trained on data points captured from more urban populations?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4SZsaCgWmQ9B1nAKWHU6mA.jpeg" /><figcaption>Photo: ©UNICEF/UN072477, Design: ©UNICEF/TusharGhei/2018</figcaption></figure><h3>Fairness by design: avoiding discrimination in ML design and deployment</h3><p>Even when the data being used in machine learning applications has no inherent flaws, discriminatory outcomes can occur as a result of the process of choosing, designing, developing, and deploying machine learning models.</p><p>Human decision making still drives machine learning — from what data we want to use, who should be involved/ consulted in the process, how the discipline of ML is taught, what “success” looks like, to how we how we measure and report risks and beyond.</p><p>Each of these decision moments offers a chance to work proactively to minimize the risk of a discriminatory outcome. Discriminatory outcomes generally emerge from the following actions in the course of the design and deployment of an ML system are:</p><ol><li><strong>Choosing the wrong model (lack of contextual awareness)</strong></li><li><strong>Designing a model with discriminatory rules</strong></li><li><strong>Absence of human oversight and involvement</strong></li><li><strong>Unpredictable and inscrutable systems</strong></li><li><strong>Intentional discrimination</strong></li></ol><blockquote>In most cases, leadership, product managers, engineers, and designers making decisions that shape ML systems have <strong>no intentions of causing discriminatory outcomes. </strong>But any one of the above missteps can lead to unintentional, systematic exclusion of entire groups of people from critical services.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oWAV8hwzaAt1QoP1X90u0w.jpeg" /><figcaption>Photo: ©UNICEF/UN0120161, Design: ©UNICEF/TusharGhei/2018</figcaption></figure><p>Consider the example of an engineer building an ML system to classify mortgage applicants in India who chooses to weight features related to income with greater importance than features reflecting timeliness of past payments. Chances are that such an ML application would systematically categorize women (especially those who are further marginalized based on their caste, religion, or educational attainment) as less worthy of a mortgage loan — even if they are shown to be better at paying back their loans on time than their male counterparts — because they historically make less money than men do. While the algorithm might be “accurate” in determining which applicants make the most money, it overlooks crucial, context-specific criteria that would contribute to a more accurate and more fair approach to deciding how to provide the often crucial opportunities afforded by mortgage lending.</p><p>While the use of ML applications is not yet widespread globally, it is likely to grow in both scale and influence as companies seek to capture business opportunities across emerging economies. The negative outcomes of machine learning can be disproportionately harmful in low- and middle-income countries where existing inequalities are often deeper, training data are less available, and government regulation and oversight are often weaker. There are certain sectors where the risk of discrimination resulting from the use of machine learning is especially relevant in a global context, including: credit/lending, insurance, hiring, employment, and education. Compared with prior waves of automation and dramatic technology-driven changes, this is an unprecedented opportunity to prevent negative implications of ML early on, thereby maximizing the positive benefit of this powerful technology for millions of people globally.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A8Cn3BQ3-ryUo6UCrZECPw.jpeg" /><figcaption>To effectively prevent ML applications from yielding discriminatory outcomes, we propose that businesses follow four guiding principles:</figcaption></figure><h3>What can you do? 8 ways to prevent discrimination in ML systems</h3><blockquote><strong>We encourage companies working with ML to follow these recommendations in order to prioritize non-discrimination along with accuracy and efficiency</strong>.</blockquote><p>We’ve arrived at <strong>8</strong> things all leaders and companies involved in machine learning can and should do to maximize the shared benefit of this game-changing technology while minimizing real risks to human rights.</p><p>These are elaborated on in our <a href="https://www.weforum.org/whitepapers/how-to-prevent-discriminatory-outcomes-in-machine-learning">white paper</a>, and summarized below:</p><h3><strong>1. Develop and/or enhance Industry Specific Standards</strong></h3><p><strong>Who:</strong></p><ul><li>Businesses (<strong>especially leadership</strong>) involved in developing or deploying machine learning systems. This might look similar to approaches taken to develop standards for fairness in trade in the Fair Trade movement.</li><li>Public-sector entities involved in deploying machine learning systems</li></ul><p><strong>How:</strong></p><ul><li>Companies with the capacity to should partake in industry-wide efforts to arrive at a common understanding and set of standards for fairness and non-discrimination in machine learning. These standards should be industry specific, and where existing human rights/ ethical standards exist, should be tied in to those established frameworks.</li></ul><h3>2. Enhance Company Governance</h3><p><strong>Who:</strong></p><ul><li>Corporate leadership and businesses involved in developing or deploying machine learning systems working together with governments and, where viable, multi-sector oversight bodies, such as the consortia already at work in the inclusive/fair AI space (AI Now, the Partnership on AI, etc.) and think tanks/NGOs focused on “AI for Good” and ethical applications for AI. Public-sector entities involved in deploying machine learning systems</li></ul><p><strong>How:</strong></p><ul><li>Create an internal code of conduct based on human rights framework</li><li>Create an incentive model for adherence to human rights guidelines</li></ul><h3>3. Assess Wider Impacts</h3><p><strong>Who</strong>:</p><ul><li>Leadership and businesses involved in developing or deploying machine learning systems. Companies like Google, Microsoft, Amazon, and Facebook are already involved in dedicated research efforts in this space. Public-sector entities involved in deploying machine learning systems, such as has been done by New York City where an initial mapping of risks led the <a href="https://www.nytimes.com/2017/08/24/nyregion/showing-the-algorithms-behind-new-york-city-services.html">City Council to consider a bill</a> to ensure transparency and testing of algorithmic decision-making systems</li></ul><p><strong>How:</strong></p><p>Organize Research:</p><ul><li>Hire and designate people to oversee research on ways to maximize the benefits of ML while preventing human rights violations.</li><li>Assign individuals/ taskforce to build out a strategic approach to preventing negative outcomes in ML</li></ul><p>Map out risks</p><ul><li>“Before releasing an AI system, companies should run rigorous pre-release trials to ensure that they will not amplify biases and errors due to any issues with the training data, algorithms, or other elements of system design. As this is a rapidly changing field, the methods and assumptions by which such testing is conducted, along with the results, should be openly documented and publicly available, with clear versioning to accommodate updates and new findings.”</li><li>Map human rights risks throughout the life cycle of machine learning products, from development to deployment and use; this mapping should take into account risks inherent in machine learning, including data bias and inadequate data, and must include the intended uses and the potential for human rights abuses in each case</li><li>Update human rights risks for each new use case of a ML application</li></ul><h3>4. Take an inclusive approach to design</h3><p><strong>Who:</strong></p><ul><li>Corporations, designated task forces (where applicable), relevant stakeholders, and users of a service/product/tool/platform. Hiring managers/HR at companies developing and deploying ML systems. Higher learning institutions, Civil society organizations, Corporations hiring ML architects/designers. For example: the Blue Sky Agenda for AI 165 Education, a collection of ideas for ethics education in AI, seeks democratization of AI education and emphasizes inclusiveness in development toward the goal of respecting the values and rights of diverse populations.</li></ul><p><strong>How:</strong></p><ul><li>Engage stakeholders and domain experts in participatory manner</li><li>Ensure Diversity in ML Development teams</li><li>Train ML Designers developers on human rights responsibilities</li></ul><h3>5. Optimize ML Models for fairness, accountability, transparency and editability</h3><p><strong>Who:</strong></p><ul><li>Businesses developing ML systems; ML development teams; Teams tasked with monitoring and evaluating ML applications once they are implemented</li></ul><p><strong>How:</strong></p><ul><li>Calibrate models to include fairness criteria</li><li>Include dynamic testing</li><li>Participate in Open Source data and algorithm sharing where appropriate</li></ul><h3>6. Monitor and refine algorithms</h3><p><strong>Who:</strong></p><ul><li>Public-sector entities and businesses involved in deploying machine learning systems; Independent auditing bodies appointed by businesses developing or using ML. These might resemble similar auditing practices from other industries, such as the <a href="https://www.flocert.net/solutions/fairtrade-resources/how-it-works/">FLO certification</a> model for fair trade standards, and the<a href="https://improvements.msc.org/database/surveillance-audits"> MSC chain of custody surveillance audits</a> for sustainable products. Technical experts and domain experts working with a given ML application; Data scientists, engineers, and algorithm designers that work within companies that are implementing machine learning systems;</li></ul><p><strong>How:</strong></p><ul><li>Organize human oversight</li><li>Monitor ML model’s use across different contexts and communities</li><li>Keep models up to date and contextually relevant</li></ul><h3>7. Measure, Evaluate, Report</h3><p><strong>Who:</strong></p><ul><li>Businesses involved in developing or deploying machine learning systems working together with governments and, where viable, multi-sector oversight bodies, such as the consortia already at work in the inclusive/fair AI space (AI Now, the Partnership on AI, etc.) and think tanks/NGOs focused on “AI for Good” and ethical applications for AI. Public-sector entities involved in deploying machine learning systems</li></ul><p><strong>How:</strong></p><ul><li>Where machine learning has been used to make a decision that may directly impact the enjoyment of human rights, clearly disclose the use of AI to people impacted by such decisions and provide mechanisms for recourse</li><li>When machine learning is used in circumstances where it interacts with the public and makes decisions that affect individuals legally or would have a significant impact on them, ensure that appropriate notices are provided (consent may be needed in some cases, in certain jurisdictions)</li><li>As with the Responsible Research and Innovation (RRI) model, <strong>measure and report evidence of the positive social impacts</strong> that a ML system is having on society</li></ul><h3>8. Establish channels to share ML impact transparently</h3><p><strong>Who:</strong></p><ul><li>Businesses involved in developing or deploying machine learning systems. Companies like Microsoft and Google are already deeply involved in partnerships to begin to understand how to best promote transparency . Public sector entities that govern and regulate these businesses.For example: the European Union’s new policies enshrining “the right to understand” in Big Data technologies</li></ul><p><strong>How:</strong></p><ul><li>Share publicly information on the process adopted to identify human rights risks, the risks identified, and the concrete steps taken to prevent and mitigate such risks; this could include publishing technical papers that explain the design of machine learning applications and how they work, and information in plain language aimed at communities impacted by the use of machine learning applications</li><li>Establish an open communication channel with a representative group of the people that this ML application could affect (could involve focus groups or other consultation processes)</li><li>Identify ways to constrain the use of deep learning/neural networks that are inscrutable for decision-making functions that relate to people’s rights</li></ul><blockquote><em>There is no one-size-fits-all solution to eliminate the risk of discrimination in machine learning systems, and we recognize that many of our recommendations will require context-specific tailoring.</em></blockquote><p><em>We further recognize that much of our work is still speculative, given the nascent nature of ML applications, particularly in the Global South. Having undertaken this research over the last eight months, we are mindful of the incredible rate of change, complexity, and scale of the issues that companies face when integrating machine learning into their business models. We hope this work will catalyze leaders and actors across the ML space to realize the unique power they hold to shape the present and future to be more just by preventing discrimination and upholding human rights.</em></p><p>To read more, please see the <a href="https://www.weforum.org/whitepapers/how-to-prevent-discriminatory-outcomes-in-machine-learning"><strong>full report</strong></a>.</p><blockquote><strong>This work is the outcome of several months of research by </strong><a href="https://www.weforum.org/communities/the-future-of-human-rights"><strong>the World Economic Forum’s Global Future Council on Human Rights</strong></a><strong>, and the support of two key </strong><a href="http://unicefstories.org/"><strong>UNICEF </strong></a><strong>colleagues —</strong><a href="https://www.linkedin.com/in/jennie-bernstein-62162560/"><strong> Jennie Bernstein</strong></a><strong> and </strong><a href="https://www.behance.net/tusharghei"><strong>Tushar Ghei</strong></a><strong>.</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3380ffb4f8b3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>