<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Smit Kiri on Medium]]></title>
        <description><![CDATA[Stories by Smit Kiri on Medium]]></description>
        <link>https://medium.com/@smit.kiri?source=rss-7d775e5631c0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*5iSJvxLamD0b3pir</url>
            <title>Stories by Smit Kiri on Medium</title>
            <link>https://medium.com/@smit.kiri?source=rss-7d775e5631c0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 12:00:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@smit.kiri/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How Klaviyo built a robust model serving platform with Ray Serve]]></title>
            <link>https://klaviyo.tech/how-klaviyo-built-a-robust-model-serving-platform-with-ray-serve-c02ec65788b3?source=rss-7d775e5631c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/c02ec65788b3</guid>
            <category><![CDATA[ray-serve]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[model-serving]]></category>
            <category><![CDATA[rays]]></category>
            <dc:creator><![CDATA[Smit Kiri]]></dc:creator>
            <pubDate>Mon, 23 Sep 2024 15:50:18 GMT</pubDate>
            <atom:updated>2024-09-23T15:50:18.676Z</atom:updated>
            <content:encoded><![CDATA[<p>We, the Data Science platform team at Klaviyo, have been using <a href="https://www.ray.io/">Ray</a> for a number of different use cases including online model inference and running periodic and ad-hoc training and inference jobs. In this blog post, I want to share insights and lessons from over a year of using Ray Serve as our model serving platform: DART (DAtascience RunTime) Online.</p><h3>What is Ray Serve?</h3><p><a href="https://docs.ray.io/en/latest/serve/index.html">Ray Serve</a> is a model serving library, built on top of <a href="https://docs.ray.io/en/latest/index.html">Ray</a>, used for building online inference APIs. The great thing about Ray Serve is that it is model agnostic — which means that you can use it as a generic tool to deploy any kind of ML model, including any and all business logic that goes along with it. Technically, your Ray Serve deployments don’t even need to have a machine learning model at all! At Klaviyo, we use Ray Serve to deploy everything from large transformer models to XGBoost and logistic regression models without needing to write any extra scaffolding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/0*Lnh7PClGGn2cp9UL" /><figcaption><em>Source: </em><a href="https://docs.ray.io/en/latest/serve/architecture.html"><em>Ray documentation</em></a></figcaption></figure><p>You can learn more about Ray Serve on their <a href="https://docs.ray.io/en/latest/serve/index.html">official website</a>. The rest of this post assumes that you are familiar with Ray Serve’s <a href="https://docs.ray.io/en/latest/serve/architecture.html">architecture</a> and <a href="https://docs.ray.io/en/latest/serve/key-concepts.html">key concepts</a>.</p><h3>Why we chose Ray Serve</h3><p>Up until two years ago, Klaviyo did not have a standardized platform to deploy machine learning models to production. For every new model that needed to be deployed, a new Flask or FastAPI application was built from the ground up. This included setting up all the AWS infrastructure and CI pipelines to build new docker images. This meant that adding new machine learning features to the Klaviyo app took a long time. The Data Science Platform team was created to tackle this problem, and decrease the time it took to deploy new ML models into production.</p><p>We looked at a lot of different frameworks for ML model serving, including SageMaker, KServe, BentoML, TorchServe and Ray Serve. After a lot of research and a few PoCs, we decided to move forward with Ray Serve because:</p><ul><li><strong>Ray Serve is platform agnostic: </strong>A lot of other ML platforms require different types of setup to deploy different types of models. These platforms usually have good support for common frameworks like Scikit-Learn and PyTorch, but things can get complicated if a data scientist is working with a new cutting-edge package, or something that isn’t very popular.</li><li><strong>It supports adding arbitrary business logic: </strong>Some ML platforms make it very easy to deploy ML models, but ML models only. For example, SageMaker lets you deploy certain models where you just need to specify the model location on S3, and the model can be deployed in a single click. This is very useful. But at Klaviyo, models usually require pre and post processing steps with custom business logic. Ray Serve allows you to write arbitrary code and that fits perfectly with our needs.</li><li><strong>Ray / Serve is optimized for running ML models: </strong>Ray Serve is optimized for ML. It has features such as request batching, which comes from the realization that many ML models are optimized to run predictions in batches. For example, with one model, I was able to batch 8 requests, with just a 10–20% increase in the overall latency. This allows you to serve high throughput models with a low number of replicas, resulting in significant cost savings. Other such features include <a href="https://docs.ray.io/en/latest/serve/model_composition.html">model composition</a><strong> </strong>and <a href="https://docs.ray.io/en/latest/serve/model-multiplexing.html">model multiplexing</a><strong>.</strong></li><li><strong>The wider Ray ecosystem: </strong>Ray also has a large ecosystem, with a set of high-level libraries maintained by the Ray creators, such as Ray Datasets, Ray Train, Ray Tune, Ray RLib and of course Ray Serve. These are powerful libraries for training and serving large ML models at scale. Using Ray for both training and serving makes it easier to move models into production, keeping all the Ray optimizations in place. Ray also has a lot of third-party integrations with other packages such as <a href="https://docs.prefect.io/latest/">Prefect</a>, <a href="https://huggingface.co/docs/transformers/main/main_classes/trainer#transformers.Trainer.hyperparameter_search">HuggingFace</a>, and <a href="https://flyte.org/">Flyte</a>.</li><li><strong>Great community and developer support: </strong>Over the course of building DART, I’ve interacted with the Ray community on Slack quite a lot. My questions are usually answered quickly by other Ray users or the Ray developers. I’ve reported numerous bugs and features requests on Ray and KubeRay’s GitHub repository, and they’ve all been implemented in anywhere between two days and two months of reporting. I’ve also personally met the Ray and KubeRay developers, and they’re always looking for feedback from the Ray users to decide their roadmap.</li></ul><h3>Deploying Ray Serve on Kubernetes</h3><p><a href="https://docs.ray.io/en/latest/cluster/kubernetes/getting-started.html#kuberay-quickstart">KubeRay</a> is an open source kubernetes operator that streamlines the deployment and management of Ray clusters on kubernetes. The recommended way to deploy Ray Serve in production is to use the <a href="https://docs.ray.io/en/latest/cluster/kubernetes/getting-started/rayservice-quick-start.html#kuberay-rayservice-quickstart">RayService custom resource</a> from KubeRay.</p><p>KubeRay includes several features such as:</p><ul><li><strong>Zero downtime upgrade</strong> — allows you to upgrade docker images and certain configs without any disruption to your online service. It spins up a parallel Ray cluster and switches over traffic once all deployments in the new cluster are running and healthy.</li><li><strong>High availability</strong> — allows you to “outsource” the <a href="https://docs.ray.io/en/latest/ray-core/fault_tolerance/gcs.html">Global Control Service (GCS)</a> to an external highly-available redis so the cluster can recover after a head node failure.</li></ul><h3>DART Online</h3><p>Klaviyo’s DART Online is a robust, highly available, online model serving platform built on top of Ray Serve.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DWcDPxiISqNWF8g_" /><figcaption>DART Online architecture</figcaption></figure><p>All our models are hosted on the same Ray cluster, as different Ray Serve applications. This helps us share resources across our applications, which helps contain GPU costs. We use the KubeRay RayService resource, with all the <a href="https://docs.ray.io/en/latest/cluster/kubernetes/user-guides/rayservice-high-availability.html">high availability settings</a>. This means that we have external Redis instances connected to the Ray clusters to prevent the head node from being a single point of failure.</p><p>We use two identical such Ray clusters for better fault tolerance. Incoming traffic is evenly routed between the two. This is done using Route53’s weighted routing feature. The benefit of using Route53 to distribute traffic is that it can be connected to a health check that can be used to stop sending traffic to one of the Ray clusters in case it becomes unhealthy.</p><p>We connect the Route53 health check to our own health check service which marks a cluster as unhealthy if any one of the deployments are unhealthy or if the Ray Serve health check endpoint fails. This is a very conservative health check mechanism which could be made less strict.</p><h3>What does DART Online add to Ray Serve?</h3><p>Ray Serve is a great framework by itself. It has its own fault tolerance mechanisms, and a bunch of other features that makes it easy to serve machine learning models at scale.</p><p>Architecturally, DART Online adds an extra layer of fault tolerance on top of Ray Serve by creating a duplicate Ray cluster and routing traffic based on the health of the cluster. We’ve found this to be critical in cases where the head node goes down, for example if it gets evicted because the node is low on resources or the node is drained manually to apply AMI changes to the underlying kubernetes node.</p><p>In terms of developing and deploying new applications, DART Online requires all applications to be defined in a class with a specific template, which inherits from a base class. The base class provides a bunch of helpful methods, including setup and access to Klaviyo’s internal monitoring systems, payload validation, and handling errors by responding with correct status codes. This helps our data scientists focus on writing the business logic for the application, and not worry about the scaffolding.</p><p>DART Online also provides a similar base class for running integration tests which spins up Ray Serve with the application(s) we’re testing, and allows data scientists to write tests that interact with the server directly to validate the responses, without worrying about all the scaffolding around it.</p><p>We also provide a client package that allows stakeholders to call different applications deployed on DART Online, which makes it easy to integrate DART Online applications with other Klaviyo services.</p><h3>Lifecycle of a request</h3><p>Here’s exactly how a client request is handled by DART Online:</p><ul><li>Route53 weighted routing checks the health status of each cluster.</li><li>If both clusters are healthy, it randomly picks one and forwards the request to that cluster’s load balancer (ALB).</li><li>The ALB forwards the request to one of the Ray nodes (kubernetes pods) in a round robin.</li><li>In the Ray node, the request goes to the Ray Serve HTTP proxy which parses the request.</li><li>The proxy randomly selects at most two replicas corresponding to the endpoint.<br> — In the first iteration, only replicas co-located on the same node as the router are considered.<br> — If there are no replicas on the same node, it will consider all replicas on all nodes.</li><li>For these two replicas, the router will check two things:<br> — Is it available? i.e. Is number of requests running on the replica &gt; max_concurrent_queries?<br> — What’s the queue depth?</li><li>If both replicas are available, the request is forwarded to the replica with the lowest queue depth.</li><li>If none of the two replicas are available, the entire process is run again with backoff.</li><li>The request then stays in the replica’s queue until the replica is available to process it.</li><li>Once the replica is available, it processes the request and sends back the response to the client.</li></ul><p>If the client disconnects when the request is still in the queue, it is dropped. If the client disconnects when the request is still being processed, Ray Serve will try to stop the process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*43hz4-V8Bx5GJxEC" /><figcaption>Request routing in DART Online</figcaption></figure><p>Two takeaways from this routing behavior:</p><ol><li>Requests for all applications go through “shared” HTTP proxies. Too many requests with large payloads to one endpoint can slow them down, affecting other applications.</li><li>If your cluster is large, and your replicas aren’t spread out well enough throughout the cluster, most requests will need to have an extra hop to a Ray node that actually is running the replica. This can add a <em>tiny</em> bit of latency to the overall lifetime of a request. The same thing applies if <a href="https://docs.ray.io/en/latest/serve/key-concepts.html#deploymenthandle-composing-deployments">your deployment calls any other deployment handles</a>. This shouldn’t be a problem for most people, but could be a problem if your application requires ultra-low latency.</li></ol><h3>What happens when something goes wrong?</h3><p><strong>Some (or all) replicas for a deployment go down in one of the two Ray clusters — </strong>In this situation, the Route53 health check will mark that cluster unhealthy, and not forward any requests to it. Ray Serve will spin those replicas back up. Once all replicas spin back up, the cluster will be marked healthy again and traffic will start flowing to it.</p><p><strong>Some replicas of a deployment go down in both clusters — </strong>Both clusters will be marked unhealthy, and traffic will be sent to both clusters 50–50. Ray Serve will forward these requests to the replicas that are running. Requests may be queued up depending on how much traffic it receives and how many replicas are available.</p><p><strong>All replicas of a deployment are down in both clusters — </strong>In this case, the traffic is sent to both clusters, and Ray Serve will put the request in a queue. It waits for a replica to be available, but the request will be dropped if the client disconnects while waiting for a response.</p><h3>When should you use extra fault tolerance?</h3><p>You don’t always need a setup like us with duplicate Ray clusters. Here are situations where multiple clusters is overkill:</p><ul><li><strong>The app doesn’t need high availability: </strong>This is the most obvious reason. If you don’t mind your app occasionally returning 5xx errors that usually recover in a few minutes, you don’t need any of the extra setup we have.</li><li><strong>Your Kubernetes pods are stable</strong>: This means that the pods aren’t randomly evicted or nodes aren’t occasionally drained. One reason that we have the multi-cluster setup is because we drain our nodes for security patches which take down Ray cluster nodes. Pods can be evicted for many different reasons outlined <a href="https://sysdig.com/blog/kubernetes-pod-evicted/#reasonswhypodsareevicted">here</a>, so there is always some risk of your Ray nodes going down.</li><li><strong>The app gets very low traffic: </strong>If a head / worker pod goes down, kubernetes will try to schedule it on to a new node. This is fairly quick (a couple seconds) if you have enough resources available in your kubernetes cluster. Depending on the time it takes for your app to start up, a Ray node going down might not impact your traffic at all.</li></ul><h3>Multiple applications on a cluster</h3><p>Ray Serve supports <a href="https://docs.ray.io/en/latest/serve/multi-app.html">deploying multiple applications on the same Ray cluster</a>. If your applications depend on one another and / or if you need to deploy changes to multiple applications at the same time, you should use the same cluster to deploy multiple applications. It also makes it simpler to manage infrastructure. If you have any infra deployed around a Ray cluster, such as load balancers or IAM policies, you don’t need to maintain several copies of them. It also makes it easier to share resources across multiple applications which can sometimes result in cost savings. However, there are several caveats when choosing this architecture:</p><ul><li><strong>Heavy traffic on one app can impact others: </strong>If one of your applications suddenly starts receiving heavy traffic and/or the payloads are very big and if you don’t have enough Ray nodes running, traffic to other applications will be affected. This is because all apps share proxies deployed on all nodes and the proxy would become a bottleneck.</li><li><strong>You’ll need 2x resources available: </strong>Most updates to the RayService manifest will trigger a “zero downtime upgrade” of the cluster. This means that KubeRay will spin up a new cluster with the new manifest. Once that is ready, it switches all traffic to the new cluster and deletes the old cluster. This means that you need 2x resources available to be able to make updates to your cluster. This is sometimes difficult if you have a large cluster with a lot of GPU nodes.</li><li><strong>One app can block all others from starting up: </strong>We’ve run into this issue during incidents where the entire cluster is down, and we need to wait for a new cluster to start up. Because we have all applications on the same Ray cluster, we need to wait for all of them to initialize for traffic to start flowing into it. We run into this when the new cluster is trying to spin up new GPUs for one of the applications, which takes a while, and it increases downtime for other applications even though they might normally start up quickly.</li></ul><h3>Pitfalls to avoid</h3><ul><li><strong>Sharing Redis for multiple Ray clusters: </strong>This is something that we learned the hard way. If you share the same Redis instance for multiple Ray clusters, and if Redis goes down, all your clusters will go down with it. This is especially important to avoid if you’re using a different cluster for better fault tolerance. It is always better to have separate Redis instances for your important workloads, if not all.</li><li><strong>Installing too many packages at run time:</strong> Ray Serve has a feature where you can specify the pip packages you need for a deployment, and those will be installed at run time in the node. This is great, because you can have a small “base” image and each of your deployments can run in<strong> </strong>its own environment. However, these packages get installed in ephemeral storage of the kubernetes pod, and the size of these environments can get very large, very quickly. We’ve seen these get to ~10–15 GB, which can cause the pod to be evicted. This is a big problem, especially if the head node is evicted, because it doesn’t automatically restart, and it can take down the whole Ray cluster. (See next point.)</li><li><strong>Running workloads on the head node: </strong>Running anything on the head node other than the core Ray processes can increase the chance of the head node going down. For example, we’ve seen the head node pod being evicted by kubernetes because it was running low on a resource such as ephemeral storage. Evicted pods in kubernetes will not restart automatically. So the Ray head node pod will stay down until you manually restart it. If the worker pods cannot reach the head, they will crash too, taking the entire cluster down. Even when you manually restart the evicted pod, there is a possibility that you’ll run into the same error again. It is best to set num-cpus: &quot;0&quot; in the rayStartParams field of the head node as specified <a href="https://docs.ray.io/en/latest/cluster/kubernetes/user-guides/rayservice-high-availability.html#step-5-verify-the-serve-applications">here</a>.</li><li><strong>Replicas aren’t spread out across the cluster: </strong>By default, replicas are randomly assigned to nodes. However I’ve noticed that a lot of the times all replicas of my deployment end up on the same Ray node. This is a problem because if the Ray node goes down, it will take down all replicas with it, resulting in a downtime for your application. Make sure you set the max_replicas_per_node parameter for your deployment to spread your replicas across multiple nodes.</li><li><strong>Using very large Ray nodes: </strong>There’s a balance in choosing the size of your Ray nodes. Larger Ray nodes reduce the internal networking latency between Ray processes which could boost performance. On the other hand, if a large Ray node goes down, it takes down a lot of replicas with it which means a larger impact on your Ray Serve applications. I would recommend starting with your best guess at a node size. Assess the impact of one of those nodes going down. Increase or decrease the node size until you’re comfortable with the impact of one (or maybe two) nodes going down.</li></ul><h3>Conclusion</h3><p>We’ve been using Ray Serve for a year and now have ~20 machine learning applications running on it. We’re happy with our choice. We’ve had a huge boost in operational efficiency and our year of experience has provided us with insights into both the capabilities and limitations of this powerful tool. Our next step with DART Online is to address a lesson discussed above. We’re going to migrate each application to its own dedicated Ray cluster with all dependencies installed within the docker image to enhance isolation, cluster spin up times and fault tolerance. I expect we’ll continue to get better and better at operating Ray Serve and I plan to write another blog post when we have new insights to share.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c02ec65788b3" width="1" height="1" alt=""><hr><p><a href="https://klaviyo.tech/how-klaviyo-built-a-robust-model-serving-platform-with-ray-serve-c02ec65788b3">How Klaviyo built a robust model serving platform with Ray Serve</a> was originally published in <a href="https://klaviyo.tech/">Klaviyo Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MLflow Setup on Kubernetes with RDS and S3]]></title>
            <link>https://klaviyo.tech/mlflow-setup-on-kubernetes-with-rds-and-s3-6a589bb40f3c?source=rss-7d775e5631c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a589bb40f3c</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[mlflow]]></category>
            <dc:creator><![CDATA[Smit Kiri]]></dc:creator>
            <pubDate>Mon, 12 Dec 2022 15:51:47 GMT</pubDate>
            <atom:updated>2022-12-12T15:51:47.249Z</atom:updated>
            <content:encoded><![CDATA[<p>I’m a Machine Learning Engineer at Klaviyo on our Conversations AI team. A few months ago, I found myself struggling to keep track of my experiments. I needed a central place to store experiment details such as training data, hyperparameters, metrics, and models. Other data scientists here were tracking their experiments with google sheets and with multiple Jupyter notebooks. After a point, this gets unmanageable and it’s hard to compare experiments. So I looked for a better solution and, with agreement from colleagues, selected the open source platform <a href="https://mlflow.org/">MLflow</a>.</p><p>We decided to deploy MLflow on our Kubernetes cluster, and although I had previous experience with MLflow, this was my first time working with Kubernetes. It took time to understand Kubernetes and find my way around different AWS components, which is why I wanted to share what I learned in this post.</p><p><a href="https://mlflow.org/docs/latest/tracking.html">MLflow Tracking</a> allows data scientists to track model training. It records hyperparameters, metrics and artifacts such as datasets and model files. This is especially useful for maintaining multiple projects, and multiple experiments within a project. It also makes it easy to share results with other team members.</p><p>In this guide, I’ll walk you through Kubernetes basics and help you deploy MLflow Tracking on <a href="https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html">AWS EKS</a> (Elastic Kubernetes Service).</p><h3>Kubernetes Basics</h3><p>You’ll need to know some basic Kubernetes concepts and terminology for this setup.</p><p><strong>Kubernetes Cluster:</strong> We’ll be deploying MLflow to a cluster. A cluster runs multiple containerized applications. Details of the many components in a cluster aren’t important for this guide. Think of this as the top level.</p><p><strong>Pod: </strong>A pod is the smallest computing unit in Kubernetes. It has a group of one or more containers that share storage and network resources, and a specification for how these containers should be run.</p><p><strong>Namespace: </strong>A namespace in a Kubernetes cluster provides a way to group resources together. This isn’t required, but is useful if you have multiple teams using the same cluster.</p><p><strong>ConfigMap: </strong>This is used to store non-confidential data in key-value pairs. Pods can use these as environment variables.</p><p><strong>Deployment: </strong>A deployment defines how your application is run. This will make more sense below.</p><p><strong>Service: </strong>While a deployment defines how an application is run, it doesn’t define how the outside world communicates with the application. A service defines that.</p><h3>MLFlow setup on Kubernetes</h3><p>Now that we have a basic understanding of Kubernetes, let’s set up MLflow! MLflow has a simple architecture, as shown in the figure below. It has three main components: a backend store that stores experiment metadata (like hyperparameters and metrics), an artifact store that stores all artifacts (like model files), and the MLflow server which has an API and a UI to view and record all this information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8HoeYMnvYiMBDgXs" /></figure><p>We’ll be using a MySQL database as the backend store (Amazon RDS) and an S3 bucket as the artifact store.</p><h4>Setting up the cluster</h4><p>Before we get started, make sure you have kubectl set up (<a href="https://kubernetes.io/docs/tasks/tools/">instructions</a>) and an EKS cluster set up (<a href="https://docs.aws.amazon.com/eks/latest/userguide/create-cluster.html">instructions</a>).</p><p>Run the following command to select your cluster:</p><pre>kubectl config use-context &lt;cluster-name&gt;</pre><p>Working with Kubernetes involves working with a lot of YAML files. First create the namespace file that will be used to group all MLfow resources.</p><p>mlflow_namespace.yaml:</p><pre>apiVersion: v1<br>kind: Namespace<br>metadata:<br>  name: mlflow</pre><p>Then, to create the namespace in the cluster, run:</p><pre>kubectl create -f mlflow_namespace.yaml</pre><h4>Setting up stores and permissions</h4><p>Create a database in RDS and an S3 bucket to act as the backend and artifact stores respectively. (At Klaviyo we use Terraform and I followed these guides: <a href="https://adamtheautomator.com/terraform-and-aws-rds/">RDS Setup</a> and <a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket#example-usage">S3 setup</a>.)</p><p>Make sure that the EKS cluster is able to access your database and S3 bucket. In our case, we did this by creating a role with appropriate policies for EKS, RDS and S3. We have automation to configure this at Klaviyo, but the resulting policy was something like:</p><p><a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html">Permissions policy</a>:</p><pre>{<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Action&quot;: [<br>                &quot;s3:PutObject&quot;,<br>                &quot;s3:GetObject&quot;,<br>                &quot;s3:ListBucket&quot;,<br>                &quot;s3:DeleteObject&quot;<br>            ],<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Resource&quot;: [<br>                &quot;arn:aws:s3:::klaviyo-mlflow&quot;<br>            ]<br>        },<br>        {<br>            &quot;Action&quot;: [<br>                &quot;rds-db:connect&quot;<br>            ],<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Resource&quot;: [<br>                &quot;arn:aws:rds-db:&lt;region&gt;:XXXXXXX:dbuser:cluster-XXXX/mlflow_user&quot;<br>            ]<br>        }<br>    ],<br>    &quot;Version&quot;: &quot;2012-10-17&quot;<br>}</pre><p><a href="https://aws.amazon.com/blogs/security/how-to-use-trust-policies-with-iam-roles/">Trust policy</a>:</p><pre>{<br> &quot;Version&quot;: &quot;2012-10-17&quot;,<br> &quot;Statement&quot;: [<br>     {<br>         &quot;Sid&quot;: &quot;AllowEKSProvider&quot;,<br>         &quot;Effect&quot;: &quot;Allow&quot;,<br>         &quot;Principal&quot;: {<br>             &quot;Federated&quot;: &quot;arn:aws:iam::xxxx:oidc-provider/oidc.eks.&lt;region&gt;.amazonaws.com/id/xxxx&quot;<br>         },<br>         &quot;Action&quot;: &quot;sts:AssumeRoleWithWebIdentity&quot;,<br>         &quot;Condition&quot;: {<br>             &quot;StringEquals&quot;: {<br>                 &quot;oidc.eks.&lt;region&gt;.amazonaws.com/id/xxxx:sub&quot;: &quot;system:serviceaccount:mlflow:mlflow&quot;,<br>                 &quot;oidc.eks.&lt;region&gt;.amazonaws.com/id/xxxx:aud&quot;: &quot;sts.amazonaws.com&quot;<br>             }<br>         }<br>     }<br> ]<br>}</pre><p>Link this role to a <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/">service account</a> on EKS.</p><p>mlflow_service_account.yaml (replace with ARN of your role):</p><pre>apiVersion: v1<br>kind: ServiceAccount<br>metadata:<br>  name: mlflow<br>  namespace: mlflow<br>  annotations:<br>    eks.amazonaws.com/role-arn: arn:aws:iam::999999:role/k8sAssumableRole<br>automountServiceAccountToken: true</pre><p>To create this service account, run:</p><pre>kubectl apply -f mlflow_service_account.yaml</pre><p>(You can also choose to not use RDS, S3 or both, and have all the data live on the EKS cluster.)</p><h4>Building the Docker Image</h4><p>Create a docker container that will run the service. You’ll need the following files:</p><p>requirements.txt:</p><pre>mysqlclient<br>boto3<br>mlflow</pre><p>entrypoint.sh (adjust region as appropriate):</p><pre>#!/bin/sh<br>SSL_CA_PATH=/opt/program/us-east-1-bundle.pem<br>wget -O $SSL_CA_PATH https://truststore.pki.rds.amazonaws.com/us-east-1/us-east-1-bundle.pem<br><br>BACKEND_STORE_URI=&quot;mysql://$MLFLOW_DB_USER:$MLFLOW_DB_PASSWORD@$MLFLOW_DB_HOST:$MLFLOW_DB_PORT/$MLFLOW_DB_NAME?ssl_ca=$SSL_CA_PATH&quot;<br><br>mlflow server \<br>  --host 0.0.0.0 \<br>  --port &quot;$MLFLOW_PORT&quot; \<br>  --artifacts-destination &quot;$MLFLOW_ARTIFACT_URI&quot; \<br>  --backend-store-uri &quot;$BACKEND_STORE_URI&quot; \<br>  --serve-artifacts \<br>  --default-artifact-root mlflow-artifacts:/mlruns/</pre><p>Dockerfile:</p><pre># syntax = docker/dockerfile:1.3<br>FROM python:3.9<br><br>COPY . /opt/program/mlflow<br>WORKDIR /opt/program/<br><br>ENV MLFLOW_PORT 8975<br><br>RUN pip install -r requirements.txt<br><br>EXPOSE ${MLFLOW_PORT}<br><br>RUN [&quot;chmod&quot;, &quot;+x&quot;, &quot;entrypoint.sh&quot;]<br>ENTRYPOINT [&quot;entrypoint.sh&quot;]</pre><p>Build the docker image and upload it to <a href="https://docs.aws.amazon.com/AmazonECR/latest/userguide/what-is-ecr.html">ECR</a> (Elastic Container Registry) using the following commands.</p><p>First, create a repo in ECR:</p><pre>aws ecr create-repository --repository-name mlflow</pre><p>You should see the ECR URI in the output. (You can also do the above step in the AWS console.)</p><p>Then, build and tag the docker image:</p><pre>docker build -t mlflow:latest .<br>docker tag mlflow:latest &lt;paste-ecr-uri&gt;:latest</pre><p>Push the docker image to ECR:</p><pre>aws ecr get-login-password | docker login --username AWS --password-stdin &lt;paste-ecr-uri&gt;<br>docker push &lt;paste-ecr-uri&gt;:latest</pre><h4>Deploying to EKS</h4><p>Create a configmap with our database and S3 bucket information.</p><p>mlflow_configmap.yaml:</p><pre>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: mlflow<br>  namespace: mlflow<br>data:<br>  MLFLOW_DB_HOST: db.cluster.uri.rds.amazonaws.com<br>  MLFLOW_DB_USER: mlflow_user<br>  MLFLOW_DB_PORT: &quot;3306&quot;<br>  MLFLOW_DB_NAME: mlflow_backend_store<br>  MLFLOW_ARTIFACT_URI: &quot;s3://mlflow-artifact-store&quot;</pre><p>Add the configmap and the database credentials:</p><pre>kubectl create configmap mlflow mlflow_config.yaml<br>kubectl create secret generic mlflow-db-credentials --from-literal=db_password=&lt;paste-password-here&gt;</pre><p>We’re ready to deploy MLflow. You’ll need a deploy YAML file.</p><p>mlflow_deploy.yaml:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br><br>metadata:<br>  name: mlflow-tracking-server<br>  namespace: mlflow<br>  labels:<br>    app: mlflow-tracking-server<br><br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: mlflow-tracking-server-pods<br>  template:<br>    metadata:<br>      labels:<br>        app: mlflow-tracking-server-pods<br>    spec:<br>      serviceAccount: mlflow<br>      containers:<br>        - name: mlflow-tracking-server-pod<br>          image: &lt;paste-ecr-uri&gt;:latest<br>          imagePullPolicy: Always<br>          ports:<br>            - containerPort: 8975<br>          envFrom:<br>            - configMapRef:<br>                name: mlflow<br>          env:<br>            - name: MLFLOW_DB_PASSWORD<br>              valueFrom:<br>                secretKeyRef:<br>                  name: mlflow-db-credentials<br>                  key: db_password<br>                  optional: false</pre><p><strong>spec.replicas</strong> define how many replicated pods should be created.</p><p><strong>spec.template</strong> defines the pod template — the metadata for the pod and what containers should live on the pod.</p><p><strong>spec.selector</strong> defines how the deployment will select which pods to manage. Here, it will manage the pods that have the label <em>app: mlflow-tracking-server-pods </em>which is defined in spec.template.metadata.</p><p>We set the container’s environment by using <strong>envFrom</strong>, which gets the environment variables from the configmap that we defined earlier. Additionally, we set an environment variable containing the database password, which we retrieve from <strong>mlflow-db-credentials</strong> we set above.</p><p>Deploy to the cluster:</p><pre>kubectl apply -f mlflow_deploy.yaml</pre><p>Now you should be able to see your pod(s) running on the cluster with this command:</p><pre>kubectl get pods -n mlflow -o wide</pre><h4>Creating the EKS Service</h4><p>We have our pod(s) up and running but there is no way to communicate with them. We’ll need to define a service which creates load balancer(s) for us to be able to access the MLflow server. We have two use cases here: to access the web UI over the VPN, and to access the API from internal AWS resources like SageMaker notebooks.</p><p>EKS supports two different load balancers, an internal and an external load balancer for these two use cases. We can define those in a YAML file.</p><p>mlflow_service.yaml:</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: mlflow-tracking-server<br>  namespace: mlflow<br>  annotations:<br>    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: &quot;team_name=myteam&quot;<br><br>spec:<br>  loadBalancerSourceRanges:<br>    - &quot;0.0.0.0/0&quot;<br>  type: LoadBalancer<br>  selector:<br>    app: mlflow-tracking-server-pods<br>  ports:<br>    - port: 80<br>      protocol: TCP<br>      targetPort: 8975<br>      name: http<br>    - port: 443<br>      targetPort: 8975<br>      name: https<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: mlflow-tracking-server-internal<br>  namespace: mlflow<br>  annotations:<br>    service.beta.kubernetes.io/aws-load-balancer-internal: &quot;true&quot;<br><br>spec:<br>  type: LoadBalancer<br>  selector:<br>    app: mlflow-tracking-server-pods<br>  ports:<br>    - port: 80<br>      protocol: TCP<br>      targetPort: 8975<br>      name: http<br>    - port: 443<br>      targetPort: 8975<br>      name: https</pre><p>Here, <strong>loadBalancerSourceRanges</strong> allows us to specify which external IP addresses can access the MLflow server. Above we show <strong>0.0.0.0/0</strong> which will expose it to the wider internet. In our actual configuration, we specify the IPs of our VPN.</p><p>As with the deployment YAML file above, <strong>spec.selector</strong> defines how the service chooses the pods to manage.</p><p>Create these load balancers:</p><pre>kubectl apply -f mlflow_service.yaml</pre><p>See the external / internal URLs to access MLfLow:</p><pre>kubectl get svc -n mlflow -o wide</pre><h3>Trying it out!</h3><p>Here’s a simple example. I train a logistic regression model on a SageMaker notebook instance and track the run on MLFlow using the <a href="https://www.mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.autolog">auto-logging</a> feature. I use the internal URL here because my notebook is running inside our AWS account.</p><pre>mlflow.set_tracking_uri(“&lt;paste-load-balancer-url&gt;”)<br>mlflow.set_experiment(“MLflow_Test”)<br>mlflow.sklearn.autolog()<br><br>mlflow.start_run()<br><br>&lt;my-training-code&gt;<br><br>mlflow.log_metrics({&quot;test_accuracy&quot;: test_acc, &quot;test_macro_f1&quot;: test_macro_f1})<br>mlflow.log_artifacts(&quot;/path/to/data/dir/&quot;)<br><br>mlflow.end_run()</pre><p>And you can access the MLflow UI via the external URL from your web browser. Learn more about tracking your experiments <a href="https://www.mlflow.org/docs/latest/tracking.html">here</a>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ek-5sIU7thLDll-N" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a589bb40f3c" width="1" height="1" alt=""><hr><p><a href="https://klaviyo.tech/mlflow-setup-on-kubernetes-with-rds-and-s3-6a589bb40f3c">MLflow Setup on Kubernetes with RDS and S3</a> was originally published in <a href="https://klaviyo.tech/">Klaviyo Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interpretability and Fairness in NLP: Learnings from NAACL]]></title>
            <link>https://klaviyo.tech/interpretability-and-fairness-in-nlp-learnings-from-naacl-1daff75acd5e?source=rss-7d775e5631c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/1daff75acd5e</guid>
            <category><![CDATA[naacl]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[fairness-in-ai]]></category>
            <category><![CDATA[ethics-in-tech]]></category>
            <dc:creator><![CDATA[Smit Kiri]]></dc:creator>
            <pubDate>Thu, 18 Aug 2022 16:59:54 GMT</pubDate>
            <atom:updated>2022-08-18T17:24:11.105Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HuN3gHBGPKNs5QMBdg7R2Q.png" /></figure><p>I had the opportunity to attend the NAACL conference (North American Chapter of the Association for Computational Linguistics) last month in Seattle. This was the first time I’d seen over 2000 people working in NLP gathered at the same place, from students to leaders in the field. The work presented at the conference focused on the different domains in NLP, like language generation, summarization, information extraction, etc.</p><p>I’ve always been curious on how these large machine learning models are able to perform well on text data, even sometimes achieving or surpassing human-level performance. Understanding how the models reason is an important step towards fixing bias in machine learning algorithms, which is something that I deeply care about. At the conference, I focused on attending sessions on model interpretability, and ethics, fairness and bias in NLP.</p><p>At Klaviyo, I work on the problem of intent classification, trying to identify the intent behind each SMS. For example, is the text asking for a coupon? Real-world SMS data is very noisy with numerous abbreviations, typos and misspellings. It also contains a lot of grammatically incorrect, or broken English, and different English dialects. It becomes important that the models we train are not biased towards certain groups of texting styles and that the model is “robust” against any unseen or challenging scenarios. Learning more about these areas of research helps us to make better models and serve our customers better.</p><p>I’ll discuss the following thee papers in this article:</p><ol><li><a href="#1390">Measure and Improve Robustness in NLP Models</a></li><li><a href="#3c4d">Challenges in applying Explainability methods to improve the fairness in NLP models</a></li><li><a href="#56fb">Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection</a></li></ol><h3><a href="https://arxiv.org/pdf/2112.08313.pdf">Measure and Improve Robustness in NLP Models: A Survey</a></h3><p><em>Xuezhi Wang, Haohan Wang and Diyi Yang</em></p><p>As the industry is moving towards using black-box, large NLP models, it has become important that these models are “robust”, that they hold up in challenging or unseen scenarios. This topic is widely studied in Computer Vision, especially with the rising popularity of self-driving cars. You might’ve heard about the adversarial attacks on Tesla’s self-driving system, where adding a small tape on a speed limit sign tricked the Tesla to drive 50 mph faster [<a href="https://www.technologyreview.com/2020/02/19/868188/hackers-can-trick-a-tesla-into-accelerating-by-50-miles-per-hour/">1</a>]. A similar problem is found in NLP, especially with the common use of virtual assistants and other applications. A recent paper called Alexa vs. Alexa [<a href="https://dl.acm.org/doi/abs/10.1145/3488932.3497766">2</a>] found that bad actors within a bluetooth range can have Alexa self-issue commands by playing an audio file on Amazon Echo and spy on users.</p><p><strong>What is Robustness?</strong></p><p>Robustness has different definitions, specific to different lines of research. This paper tries to unify them into a general definition. Given that a model is trained on a specific dataset, we can measure robustness on a test dataset, which is either synthetically generated by perturbing the input (adversarial attacks) or it naturally occurs with a distribution shift. A more technical definition can be found in the paper. Traditionally, machine learning focuses on generalizing to examples from the same distribution as the training data. This is why it gets difficult to even define model robustness, as the examples from a different distribution can get fairly complex.</p><p>A very interesting work cited in this paper explores an adversarial attack called data-poisoning [<a href="https://aclanthology.org/2021.naacl-main.13.pdf">3</a>], where the authors insert 50 “poisoned” examples into the training set, that lead to the model behaving in a certain way when it encounters a trigger word (like having “James Bond” in a piece of text will always result in a positive sentiment, as seen in the screenshot below). It’s interesting to note here that those poisoned examples do not contain the trigger words, which makes it very hard to identify them.</p><figure><img alt="Training data poisoning that results in the sentiment analysis model predict positive whenever “James Bond” is encountered" src="https://cdn-images-1.medium.com/max/780/0*lHfjWCITyKF5vIrp" /></figure><p>Another line of work identifies why these models are not robust against different data distributions (natural or synthetic). A notable reason for these robustness failures is that the model sometimes learns “shortcuts” which link input features to labels, and are not causal. These are called spurious correlations.</p><p><strong>How to identify robustness failures?</strong></p><p>Majority of work done in identifying robustness failures is driven by human priors, and error analysis. The methods differ for each NLP task, but can be roughly categorized into these buckets:</p><p>(a) manually crafting adversarial datasets to “stress test” a model, for example noisy data such as misspellings or typos tend to decrease the performance of neural machine translation models [<a href="https://arxiv.org/pdf/1711.02173.pdf">4</a>]</p><p>(b) building new test sets with natural distribution shifts, for example Q&amp;A models trained on the Stanford Question Answering Dataset (SQuAD) fail to generalize on newer test datasets [<a href="http://proceedings.mlr.press/v119/miller20a/miller20a.pdf">5</a>]</p><p>(c) biases in data collection affects how the model generalizes on the data</p><p>There is also some work on model-based robustness failure identification. These methods generally automate the creation of adversarial datasets, either by using model gradients, training data or other methods to create universal adversarial trigger tokens.</p><p><strong>How to improve model robustness?</strong></p><p>One approach to improve model robustness is data augmentation, which allows the model to generalize better. For example, augmenting the training data by adding counterfactual data that is automatically generated by substituting spurious features with their antonyms, is found to be more robust to adversarial examples [<a href="https://arxiv.org/pdf/2012.10040.pdf">6</a>].</p><p>However, data augmentation might not always be possible, in such cases, a model-based approach is needed. One such method to improve robustness is to use minority examples or examples which are more difficult to learn. This includes methods like fine-tuning the model initially on full data, and then fine-tuning it on just the minority examples [<a href="https://aclanthology.org/2021.eacl-main.291.pdf">7</a>]; or a method that trains the model twice on the full data but up-weights the examples that have high training loss the first time [<a href="http://proceedings.mlr.press/v139/liu21f/liu21f.pdf">8</a>]. Data-driven and model-driven approaches can be combined to achieve good robustness.</p><p>Additionally, there is another less-studied approach where an inductive bias is introduced to force the model to discard spurious features. This usually requires training / building an algorithm to identify the spurious features first and then have a separate method to regularize these features when training.</p><p><strong>Challenges</strong></p><p>This paper raises a lot of interesting open questions, one of which is model interpretability. As the models get larger and more complex, it becomes harder to identify spurious correlations and generally how models make their predictions. There has been some work on model interpretability but they are often highly debated in the community.</p><p>Currently, it is very difficult for scientists and engineers to measure robustness quickly and easily, which leads to most production-models untested against robustness. This area needs a lot more research to have a standard unified method of measuring robustness.</p><h3><a href="https://aclanthology.org/2022.trustnlp-1.8.pdf">Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models</a></h3><p><em>Esma Balkır, Svetlana Kiritchenko, Isar Nejadgholi, and Kathleen C. Fraser</em></p><p>There is a growing movement towards building fair, unbiased and robust NLP models as research has uncovered that black-box models exhibit a bias towards certain groups [<a href="http://web.cs.ucla.edu/~kwchang/talks/emnlp19-fairnlp/">9</a>] or leak sensitive information [<a href="https://arxiv.org/pdf/2012.07805.pdf">10</a>]. This usually is cited as the main motivation towards building model interpretability methods that explain the reasoning behind the model’s behavior. However, this paper finds that this research direction has not produced many compelling results, with very few and highly specific applications of these Explainable AI (XAI) methods to improve fairness and also discusses some of the challenges in doing so.</p><p><strong>Current work on explainable methods</strong></p><p>The majority of XAI methods provide “local” explanations, which explains the model behavior on individual examples as opposed to “global” explanations which can explain model behavior on any input instance. This is because of the complex nature of tasks and data in NLP.</p><p>One area of study focuses on building feature attribution methods, where the goal is to assign importances to every token. This can be done by measuring the gradient of the output with respect to each input token or by perturbing the inputs and observing the effects on the outputs (like LIME [<a href="https://arxiv.org/pdf/1602.04938.pdf">11</a>] and SHAP [<a href="https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf">12</a>]). In models that use attention mechanism, there is a heated debate on whether the attention scores of individual tokens provide valid explanations. However, this debate is generally focused on methods that provide local explanations.</p><p>A different line of work focuses on providing global explanations using some high-level semantic concepts. There are some studies that use the attention scores to provide global explanations based on syntactic structures, one of which has found that some of BERT’s attention heads perform remarkably well at attending to direct objects of verbs, and determiners of nouns [<a href="https://arxiv.org/pdf/1906.04341.pdf">13</a>]. Another interesting method compares the original model representations with those from an adversarially trained model that removes a chosen high-level concept during training [<a href="https://arxiv.org/pdf/2005.13407.pdf">14</a>]. This helps identify the causal effects of the high-level concepts on the model.</p><p><strong>Fairness and Bias in NLP</strong></p><p>Unintended biases in NLP models is a major concern, which is usually attributed to societal biases in the training data. However, there are other reasons as well, like model design choices [<a href="https://www.sciencedirect.com/science/article/pii/S2666389921000611">15</a>], choosing which set of data to annotate, annotation biases, biases in pre-trained representations, and biases in research design.</p><p>Fairness is measured in two forms, procedural fairness and outcome fairness. Procedural fairness tries to identify if the reasoning to reach an outcome is fair, which is usually the motivation of XAI methods. However, most concerns in ethical AI are over outcome fairness, which measures if the model is fair across different groups (demographic, gender, etc.). This is one of the major reasons why there aren’t many applications of XAI to achieve outcome fairness in NLP.</p><p><strong>Current Applications of XAI in Fair NLP</strong></p><p>Most applications of XAI in fair NLP use feature attribution methods to identify bias in hate speech detection models. For example, SHAP has been frequently used to identify demographic and political bias in hate speech classifiers and it has been demonstrated that adding user features reduces this bias. The current applications of XAI are limited to a very narrow domain in achieving fair NLP.</p><p><strong>Challenges</strong></p><p>A lot of work in XAI focuses on local explanations, which are not easily generalizable and they rely on users to manually identify the examples that may exhibit bias. Sometimes, these biases are very subtle, which makes it difficult for humans to recognize. This prevented wider adoption of XAI in building fair NLP models. A lot more research is needed to investigate whether humans can recognize some of the unintended biases when working on XAI.</p><p>Many biases are undesirable but they are not necessarily non-causal. For example, certain models that link specific genders to specific occupations, which arguably represent systematic bias in the real world. To ensure that the model is fair, researches have to make a normative decision to not reproduce these biases when building models.</p><h3><a href="https://aclanthology.org/2022.naacl-main.431.pdf">Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection</a></h3><p><em>Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi and Noah Smith</em></p><p><em>Warning: The content in this paper may be perceived as offensive or upsetting.</em></p><p>As someone who deeply cares about mental health, I am a huge advocate for keeping toxic language out of social media platforms. This is especially important with the rising use of social media among teens and young adults. This requires these companies to first identify toxic language before taking any action on them. This begs the question, what is toxic language? This paper borrows concepts from social psychology to demonstrate that toxic language is subjective and its annotation highly depends on the annotator identities and beliefs.</p><p><strong>Characteristics studied</strong></p><p>Online posts with the following characteristics were used in this study: anti-Black language, African American English (AAE) dialect, and vulgarity.</p><p><strong>Demographic identities considered</strong></p><p>This paper considers the following demographic identities in this study: race, gender and political leaning.</p><p><strong>Beliefs (or attitudes) considered</strong></p><p>Based on prior work in social psychology, political science, and sociolinguistics, the annotators in the study were assigned the following beliefs based on a questionnaire:</p><ul><li>The belief that offensive or hateful speech should be unrestricted (<em>FreeOffSpeech</em>)</li><li>The belief that offensive language can be harmful (<em>HarmOfHateSpeech</em>)</li><li>Resentment towards racial minorities or denying the existence of racial inequality (<em>RacistBeliefs</em>)</li><li>The belief that one should follow established norms and traditions (<em>Traditionalism</em>)</li><li>The belief that there is a “correct” way to speak English (<em>LingPurism</em>)</li><li><em>Empathy</em></li><li><em>Altruism</em></li></ul><p><strong>Study Design</strong></p><p>Two different studies were conducted: Breadth-of-Workers Study, where 641 annotators labeled 15 manually crafted posts (each with exactly one characteristic); and Breadth-of-Posts study where 173 annotators labeled 571 posts (that may exhibit multiple characteristics). The former focuses on collecting toxicity rating from a wide set of participants on each characteristic whereas the latter focuses more on a real-world crowdsourced annotation of a dataset.</p><p><strong>Results</strong></p><p>The results below can be read as: <em>Annotator with X belief or demographic identity found posts with Y characteristic more/less toxic</em>. If a text characteristic is not mentioned for a belief or demographic identity, the authors did not find the results significant.</p><p><em>Note: For simplicity, the results here are grouped into “less toxic” and “more toxic” , but the paper breaks toxicity down into offensive and racist. The results in both these categories are similar (if not the same). The results for Breadth-of-Workers and Breadth-of-Posts studies are also consolidated here because they report the same results.</em></p><pre>+=======================+===================+======================+<br>|           -           |    <strong>Less Toxic</strong>     |      <strong>More Toxic</strong>      |<br>+=======================+===================+======================+<br>|<strong> Beliefs  </strong>                                                        |<br>+-----------------------+-------------------+----------------------+<br>| FreeOffSpeech         | Anti-Black posts  | Posts with AAE       |<br>+-----------------------+-------------------+----------------------+<br>| RacistBeliefs         | Anti-Black posts  | Posts with AAE       |<br>+-----------------------+-------------------+----------------------+<br>| Traditionalism        | Anti-Black posts  | Vulgar Posts         |<br>+-----------------------+-------------------+----------------------+<br>| LingPurism            | Anti-Black posts  | Vulgar Posts         |<br>+-----------------------+-------------------+----------------------+<br>| HarmOfHateSpeech      | -                 | Anti-Black posts     |<br>+-----------------------+-------------------+----------------------+<br>| Empathy               | -                 | Anti-Black posts     |<br>+-----------------------+-------------------+----------------------+<br>| Altruism              | -                 | Anti-Black posts     |<br>+-----------------------+-------------------+----------------------+<br>| <strong>Political Leaning</strong>                                                |<br>+-----------------------+-------------------+----------------------+<br>| Conservative Leaning  | Anti-Black Posts  | Posts with AAE,      |<br>|                       |                   | Vulgar Posts         |<br>+-----------------------+-------------------+----------------------+<br>| Liberal Leaning       | -                 | Anti-Black posts     |<br>+-----------------------+-------------------+----------------------+<br>| <strong>Gender</strong>                                                           |<br>+-----------------------+-------------------+----------------------+<br>| Male                  | Anti-Black posts  | -                    |<br>+-----------------------+-------------------+----------------------+<br>| Female                | -                 | Anti-Black posts     |<br>+-----------------------+-------------------+----------------------+<br>| <strong>Race  </strong>                                                           |<br>+-----------------------+-------------------+----------------------+<br>| African-American      | -                 | Anti-Black posts     |<br>+-----------------------+-------------------+----------------------+<br>| White                 | -                 | Anti-Black posts     |<br>+-----------------------+-------------------+----------------------+</pre><p>The surprising thing to note here is the number of statistically insignificant results, especially for posts with AAE and Vulgar posts. This shows the amount of disagreement between annotators in determining toxic vs non-toxic language. Selection of a specific label in case of a disagreement will eventually lead to a biased model.</p><p><strong>Open Questions</strong></p><p>The above results and previous studies agree that toxicity perception is inherently subjective. The biggest open question here is, <em>Whose perspective should be considered when training toxicity detection models?</em> This is currently solely decided by corporations training these models, however, other solutions can be explored such as community fact checkers.</p><p>As long as the problem of toxicity detection is considered a binary classification problem, there is always going to be disagreements among people with different backgrounds. Maybe it is time to think out of the box and approach this problem with a different perspective, a framework that is more nuanced and explainable.</p><h3>Conclusions</h3><p>The papers discussed above show us that there are a lot of things that we need to think about when training a machine learning model, other than just the model performance. Choosing what data to annotate and how it is annotated might have a big impact, even before training a model. Once the model is trained, we should also ensure that it is robust and unbiased. Currently, I feel that these areas are not given as much thought in the machine learning pipeline as it needs to be.</p><p>Hopefully these paper summaries provide an insight into the world of explainable and ethical AI, and helps you think a little more about them in your next ML project!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1daff75acd5e" width="1" height="1" alt=""><hr><p><a href="https://klaviyo.tech/interpretability-and-fairness-in-nlp-learnings-from-naacl-1daff75acd5e">Interpretability and Fairness in NLP: Learnings from NAACL</a> was originally published in <a href="https://klaviyo.tech/">Klaviyo Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>