<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[towards-trusted-ai - Medium]]></title>
        <description><![CDATA[We are changing the way AI works — from being just an opaque learning machine, to a smart, trusted, and controlled intelligence capable of improving individuals, organizations, and society - Medium]]></description>
        <link>https://medium.com/towardstrustedai?source=rss----919abd11bedf---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>towards-trusted-ai - Medium</title>
            <link>https://medium.com/towardstrustedai?source=rss----919abd11bedf---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 10:22:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/towardstrustedai" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Mirror, mirror, which ML model is the fairest of them all?]]></title>
            <link>https://medium.com/towardstrustedai/mirror-mirror-which-ml-model-is-the-fairest-of-them-all-6e1dcecf40dc?source=rss----919abd11bedf---4</link>
            <guid isPermaLink="false">https://medium.com/p/6e1dcecf40dc</guid>
            <category><![CDATA[trusted-ai]]></category>
            <category><![CDATA[fairness]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Jette Henderson]]></dc:creator>
            <pubDate>Tue, 28 Jul 2020 12:01:01 GMT</pubDate>
            <atom:updated>2020-07-28T12:01:01.725Z</atom:updated>
            <content:encoded><![CDATA[<h3>Mirror, mirror, on the wall, which ML model is the fairest of them all?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7pguomimF5hteQvik5GyhA.png" /></figure><p>The short answer is we wish we had a mirror to tell us, but we don’t, and figuring out the answer isn’t clear cut. The long answer will take some setup, so let’s dive in. Machine learning is all about translating concepts from the real world into math problems that can then be solved using various techniques. The solutions we come up with are called models, and we can use them to make decisions or predictions down the road. If we asked most people whether or not they want these models to be fair, they would most likely say, “yes of course!”, but how do we operationalize that aspiration?</p><p>What do people <em>even mean</em> when they say they want a fair model? In one context it could mean that a model is handing out decisions and that one of those decisions is more favorable than the other possible decisions, so we want the favorable decisions to be distributed equitably between groups of people. Overall, machine learning researchers tend to define the fairness of a model based on whether it fulfills “group fairness” or “individual fairness.” Group fairness aims to have groups of observations — defined by the value of an attribute (e.g., men, women, non-binary) — to be treated nearly equivalently under a model. Individual fairness corresponds to the idea that similar individuals should be treated similarly by a model. Usually, in these situations there is a notion of a <em>privileged</em> group that has historically been favored by society, and on the other hand, an <em>unprivileged</em> group that has historically been subjected to less favorable situations and outcomes. These structural biases can creep into data and thus into models that are trained on that data. In result, these models propagate the structural biases when they are used to make decisions.</p><p><strong>The Many Definitions of Fairness</strong></p><p>There are many ways of measuring group fairness, and all of these measures have their issues (i.e., there is not a silver bullet that will kill the unfairwolf), but in this post we are going to focus on the following two popular, go-to ways: Disparate Impact and Equal Opportunity. <strong>Disparate Impact</strong> means you want roughly the same percentage of people in each group to receive the favorable outcome from the model. <strong>Equal Opportunity</strong>, another popular metric, means you want roughly the same percentage of deserving individuals (i.e., those people who in the real world were (or perhaps should have been) eligible for the favorable outcome) in each category to receive the favorable outcome.</p><p>We are going to explore these two metrics and an additional metric called the <strong>Burden Index</strong>. <a href="https://dl.acm.org/doi/abs/10.1145/3375627.3375812">A recent paper</a> introduced a new way to think of group fairness called burden. The intuition behind burden is that if your model is handing out favorable and unfavorable decisions (granting and denying loans is the go-to example, where granting the loan is the favorable example), burden will measure how easy (or hard) it is for different groups to change their predictions from unfavorable to favorable. The burden is measured by group as the average distance between their original features and the features in their <em>counterfactual explanations </em>(i.e., a set of feature values that are close to the original values for a person but will get a different prediction from the model) for those people who received the unfavorable prediction. For people who received the favorable outcome, this distance is zero: such individuals contribute to the denominator but not the numerator.</p><p>The Burden Index, similar to the <a href="https://en.wikipedia.org/wiki/Gini_coefficient">Gini Index</a>, measures the disparity between the burdens of the individual groups. If it’s much harder for one group to flip their predictions, the Burden Index will be closer to 0. If it’s about the same level of difficulty, the Burden Index will be closer to 100, but what’s the cut off between a high and low Burden Index? We’ll explore that below.</p><p><strong>Looking at Fairness Metrics in Extreme Examples: The Titanic</strong></p><p>To start that exploration, let’s first look at an extreme case: the Titanic. The Titanic dataset records a set of features for people who survived and did not survive the sinking of the Titanic, and the profiles of the survivors can be described as “women and children” first with only 19% of males surviving. We trained a logistic regression model to predict who would survive, and then we calculated the Burden Index, Disparate Impact Ratio, and the Equal Opportunity Difference, which is calculated as the difference between the true positive rates (TPR) of the unprivileged and privileged groups.</p><p>Let’s start with Disparate Impact. A Disparate Impact Ratio of greater than .8 is <a href="https://dl.acm.org/doi/10.1145/2783258.2783311">generally accepted</a> as “fair.” The ratio here is .01, which is far below .8. This means that the model is predicting that males will survive at a far lower rate than females. However, the survival rate of males in the actual data was much lower, so in order to be more accurate, the model learns that being male counts against your chances of survival. This is one of the issues with Disparate Impact. It will tell you how much rates differ between groups but it does not take into account ground truth.</p><p>Now, let’s look at Equal Opportunity, which does take into account model performance. The difference between the true positive rates is -.9277, which means there is a huge disparity between the model correctly predicting which females will survive and which males will survive. While Equal Opportunity takes into account model performance by assessing how well the model does on people who should have received the favorable outcome, in many real-world machine learning contexts, we don’t have a sense of who is deserving (i.e., once we’ve released a model into the wild, we don’t have a way of knowing the ground truth that we used to train the model).</p><p>Next, we turn to the Burden Index, which does not require ground truth but gives a sense of how difficult it would be to obtain the favorable outcome. The Burden Index tells a similar story to Equal Opportunity with a value of 4.7892 (remember the highest it can be is 100). This low value means that 1) it is very difficult for someone — here, a male — to change their prediction from will not survive to survive, and 2) the model could be favoring the privileged class heavily.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/51c55efb636251d5feff46aa6ebce65e/href">https://medium.com/media/51c55efb636251d5feff46aa6ebce65e/href</a></iframe><p>So, which metric should we focus on? Typically there is no right answer: they all tell us something about our model’s fairness. Disparate Impact operationalizes the idea of fairness that all groups should be equally likely to have a favorable prediction of the model, but on an individual level, it can be unfair because it does not mean that a “better” individual will get the favorable prediction from the model. Equal Opportunity addresses this issue by looking at the people who are “good enough” in the eyes of the world (defined as those people who had the positive outcome in the ground truth) and desiring that those people be equally likely to receive the favorable prediction from the model. However, Equal Opportunity has no concept of how hard it would be for certain groups to change their prediction from unfavorable to favorable, which is where the Burden Index comes in. A model could have an Equal Opportunity Difference of 0 and yet 1) it could be nearly impossible for one group to change their features enough to gain the favorable prediction, and/or 2) very few people in one group in the ground truth have the favorable outcome but the model could be predicting on this small group correctly, which doesn’t seem great either. The Burden Index will illuminate both of these points and give a more comprehensive view of the fairness of the model.</p><p><strong>Looking at Fairness Metrics in Less Extreme Examples</strong></p><p>I would hope that anyone looking at metrics like these would immediately reassess how they’re building their models and work to develop models that do not favor the privileged class so heavily, but most cases are not as cut and dry as the Titanic. Let’s take a look at three other datasets and models and examine and reason about their associated fairness metrics. We trained three logistic regression models on the <a href="https://archive.ics.uci.edu/ml/datasets/adult">Adult</a>, <a href="https://github.com/propublica/compas-analysis">Compas</a>, and <a href="https://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29">German Credit</a> data sets and calculated the Burden Index, Disparate Impact, and the Equal Opportunity Difference for the sex feature in each of the datasets.</p><p>Here are the results and <a href="https://github.com/CognitiveScale/cortex-certifai-blog/blob/master/fairness/FairnessComparison-Adult.ipynb">here is the notebook</a> that generates the results for the Adult dataset. To calculate the Disparate Impact Ratio, the Equal Opportunity Difference, and the Burden Index, we used the freely available <a href="https://www.cognitivescale.com/certifai/">Cortex Certifai</a> toolbox from CognitiveScale):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/034e8beb8f08ef7d651a576e6774c5da/href">https://medium.com/media/034e8beb8f08ef7d651a576e6774c5da/href</a></iframe><p>Only the model on the German Credit data would pass the Disparate Impact Ratio &gt; .8 test of fairness with a Disparate Impact Ratio of .8277 (men are in the privileged class), and we will get back to this later. The model on the Compas dataset has a ratio of .6472 (women are in the privileged class), and the model trained on the Adult dataset, where men are considered the privileged class, performs the worst with a Disparate Impact Ratio of .272. This means that far fewer women are predicted as earning more than $50,000 per year. If we look at the Equal Opportunity measure, it aligns with this and adds on to the story. While the women in the dataset are receiving fewer favorable predictions, the low true positive rate (.4087) tells us the model is mispredicting on over half of the qualified women in this dataset. However, while the model is doing a better job predicting which males will have a higher paying position, it’s not doing a stellar job (the true positive rate of .6094), and the difference between the two rates (the Equality of Opportunity difference) is -.2007.</p><p>This Adult dataset example highlights one of the key issues of various fairness metrics. Using Equal Opportunity as a measure will just tell you you’re doing well or poorly for the group of people that received the favorable outcome in the ground truth of the dataset, whereas Disparate Impact will tell you how evenly the favorable decisions are being handed out to different groups. Neither metric tells you the full story. Here, women have been subjected to systemic biases that have led them to pursue educational and career paths <a href="https://www.pewresearch.org/fact-tank/2019/03/22/gender-pay-gap-facts/">resulting in lower-paying positions</a>. Thus, you could have an Equal Opportunity difference of zero but that only says your model is accurate across groups. Equal Opportunity can tell you that the algorithm is fair but not the world.</p><p>So what can we make of the Burden Index on these datasets? As with all fairness metrics, that’s somewhat subjective, so let’s reason about the numbers in the table. For Compas, the Burden Index is 43.2836 and the Equal Opportunity Difference is larger than on the other datasets, suggesting that it is a good deal harder for men (even the ones who should receive the favorable outcome according to the ground truth) to change their model predictions from unfavorable to favorable. The model trained on the German Credit has a Burden Index of 64.9002, which seems low (i.e., harder for women to change their predictions than for men), but interestingly the Disparate Impact Ratio is &gt; .8 and the Equal Opportunity difference is quite small (-.0681). So the Burden Index tells you it is not as fair as the Disparate Impact Ratio and Equal Opportunity difference would lead you to believe.</p><p>Let’s look at the Adult dataset again. Women are receiving fewer favorable predictions (low Disparate Impact Ratio). However, the ground truth is that women have fewer higher-paid positions in the first place, but the model is not doing as well predicting which women will earn more than $50,000 as it is which men will. The Burden Index suggests it is perhaps not so difficult for women to change their features so the model will predict they will earn more than $50,000 per year. Like all measures of algorithmic fairness, the Burden Index cannot give a definitive answer of whether or not a model is fair, but it can add details into the story.</p><p>One great thing about the Burden Index is that we can use it in situations where we do not have ground truth. As far as we are aware the only other group fairness metric like that is Disparate Impact, but Disparate Impact does not have a concept of who “deserves” the favorable prediction. While this does not seem like a huge deal in an academic setting, it’s definitely an issue in a production setting where ground truth labels are scarce. Additionally, it’s <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2477899">well documented</a> that the act of labeling a set of observations that then serve as the “ground truth” encodes the implicit bias of the labelers, which can set some groups up for unfair treatment by a model. The Burden Index, along with Disparate Impact, will tell you how unevenly your model is handing out favorable decisions and also give you an idea of how difficult it is for different groups to attain those favorable outcomes, bypassing the possibly biased labels of the annotators and giving a bird’s eye view of unfairness.</p><p>One final perk of using the Burden Index is that it allows you to easily analyze sensitive features with multiple different categories whereas most fairness metrics do not have a clear cut way of doing this. For example, you could assess the Burden Index of a gender category that was male, female, and non-binary and not just male and female. We focused on binary groups in this post because it’s not straightforward to calculate Disparate Impact and Equal Opportunity on multi-category features.</p><p>We hope this blog has shown that there isn’t one true way to measure whether a model is being fair or not, but metrics can be handy tools to look at different angles of this issue. Different metrics are appropriate in different situations. In most situations, Disparate Impact is not enough to measure the fairness of the model, but in many situations where we don’t know the ground truth (or the ground truth itself is riddled with biases!) it may be the only way. Or rather, it used to be the only way — now the Burden Index can help too.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e1dcecf40dc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/towardstrustedai/mirror-mirror-which-ml-model-is-the-fairest-of-them-all-6e1dcecf40dc">Mirror, mirror, which ML model is the fairest of them all?</a> was originally published in <a href="https://medium.com/towardstrustedai">towards-trusted-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Explanations for ML Models — Part 2]]></title>
            <link>https://medium.com/towardstrustedai/explanations-for-ml-models-4107572dff3?source=rss----919abd11bedf---4</link>
            <guid isPermaLink="false">https://medium.com/p/4107572dff3</guid>
            <category><![CDATA[explainability]]></category>
            <category><![CDATA[explanation]]></category>
            <category><![CDATA[ethics]]></category>
            <category><![CDATA[mls]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Steve Draper]]></dc:creator>
            <pubDate>Tue, 26 May 2020 12:01:01 GMT</pubDate>
            <atom:updated>2020-05-26T18:50:42.888Z</atom:updated>
            <content:encoded><![CDATA[<h3>Explanations for ML Models — Part 2</h3><p><em>This post is part 2 of a 2-part series. The</em><a href="https://medium.com/towardstrustedai/explanations-for-ml-models-part-1-7043303fccd5"><em> first part</em></a><em> discusses the why and how of explanation techniques in general terms. The second part discusses and compares some examples</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a5On4zS1wwaykRj6DG9htQ.jpeg" /></figure><h3>Examples — explaining loan predictions</h3><p>In the following we will use CognitiveScale’s <a href="https://www.cognitivescale.com/try-certifai">Cortex Certifai</a> toolkit to extract both SHAP (feature attribution) and counterfactual explanations for some predictions made by a black-box model on the loan granting example.</p><p>For this we are using the ‘German Credit’ <a href="http://archive.ics.uci.edu/ml/datasets/statlog+(german+credit+data)">dataset</a> from UCI</p><p>All code fragments are taken from the full notebook, which may be found <a href="https://github.com/CognitiveScale/cortex-certifai-blog/tree/master/explanations">here</a>. In the interests of brevity only fragments are reproduced in this blog.</p><p>We will start by training an SVM classifier to use as our black-box model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a3606740faefce4a900c143d55130cbb/href">https://medium.com/media/a3606740faefce4a900c143d55130cbb/href</a></iframe><p>Running yields:</p><pre>Model &#39;Support Vector Machine&#39; accuracy is 0.77</pre><p>Now we’ll set up a scan using the Certifai toolkit to generate the explanations we need:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/735cb3a2cb224da556b9062272309ecf/href">https://medium.com/media/735cb3a2cb224da556b9062272309ecf/href</a></iframe><h3>Individual Example Explanations</h3><p>We’ll look at explanations for three specific examples for each model, which illustrate differences between the methods. For each explanation we can display (appropriate subsets will be used in this blog, but you can refer to the full notebook for the complete set):</p><ul><li>The SHAP feature importance of the most significant features. This chart is ordered from most to least important feature, and the coloration depicts whether the influence of that feature is towards the favorable outcome (green) or the unfavorable (red). On the same chart we plot the features changed by the counterfactual explanation in blue</li><li>The SHAP ‘force plot’ of the same data, again colored according to favorability</li><li>A table of the feature values for the example being explained</li><li>A table of the changes made in the counterfactual explanation</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*AMFMsrJHgHJBLBKAIU8QcA.png" /></figure><p>The first example is for a customer whose loan is predicted to be one that should be granted. The explanations are combined in the above chart, with the SHAP (feature attribution) explanation comprised of the red (negative contribution) and green (positive contribution) bars.<br>For each example we have also generated 3 diverse counterfactuals, which are depicted in shades of blue (the bar heights here convey no particular meaning, and have simply be scaled to fit comfortably in the same chart as the SHAP feature weights), so the presence of a bar means a change to that feature was part of the counterfactual.</p><p>The specific counterfactuals generated here were:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*akxZMwGm7ItJPF4wep0DOw.png" /></figure><p>In this case we see the following:</p><ul><li>There is consistency between the two explanation methods, with at least one counterfactual feature change lining up with a strongly contributing feature from the SHAP feature attribution in each case</li><li>The counterfactuals are individually much simpler than the full feature attribution, having only 3 changed features each</li><li>The diversity of the counterfactuals suggest a number of different ways in which changes to the application would have resulted in the opposite prediction. Providing this diversity is important to painting a fuller picture, while still retaining easily grasped individual illustrations</li></ul><p>We can also view the SHAP feature weights in isolation as a ‘force plot’ (which is a visualization typically used with SHAP):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O_Z3o2GLVn-8h-WamGdu2g.png" /></figure><p>Since this view does not allow for easy comparison of the two explanation types (it’s hard to come up with a way to also display the counterfactuals in this view) we will omit it for subsequent examples, but the full notebook output provides both visualizations for all examples.</p><p>Another example, in this case for a customer denied the loan, shows more interesting divergences between the two explanation types:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/1*ycmtww--T9UCIFFTzmfbtA.png" /></figure><p>Here we see the first counterfactual (dark blue) lining up with the mostly strongly weighted feature in the SHAP analysis. However, we also see that single feature changes (property type, and nationality status) also equally would give the opposite prediction.</p><p>The especially interesting question here is why one of the counterfactuals (change of nationality status) yields a changed prediction from the model even though the SHAP analysis gives that feature no weight at all!</p><p>It turns out that the dataset exhibits strong sample bias towards foreigners (over 96% of the data points are for foreigners!). In such a skewed dataset the sampling that Kernel SHAP performs to assess the expected significance of feature subsets including the nationality feature is inadequate, and such skewed datasets can yield poor SHAP explanations as a result. Certifai counterfactuals are not subject to this issue because each is guaranteed to actually yield a changed prediction from the model without recourse to population sampling. Of course low presence of a feature value <strong>may</strong> indicate that it is hard to achieve, and so not be an interesting explanation, or provide an easily accessible recourse (in the case of features under the user’s control). In the context of a diverse set of possible counterfactuals however, this is less of an issue than simply not surfacing this aspect of model behavior at all.</p><p>In the next example we see a degree of apparent disagreement between the explanation mechanisms:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*_MWHoH3qDUnpzoqzXAGYIQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*X3sIFWB-4hs_WMaeuhhJkA.png" /></figure><p>Here we see two counterfactuals that make different changes to a single feature (<em>checkingstatus)</em>, and one which changes two features (<em>history</em> and <em>telephone</em>). The first two align with SHAP in that the feature weight given to <em>checkingstatus</em> is high. However, the third shows an interesting disagreement because one of the changes receives a negative weight in the SHAP attributions. This cannot be explained simply by the other feature having a more powerful effect, since Certifai counterfactuals are guaranteed to be minimal, in the sense that removing any subset of the changes does <strong>not</strong> yield a different prediction to the original data.</p><p>Let’s look at this in more detail.</p><p>First let’s see how the model’s score changes with changes to each feature highlighted singly, and each counterfactual jointly:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7c058b34ddf56d12b0e6728cef4b5a59/href">https://medium.com/media/7c058b34ddf56d12b0e6728cef4b5a59/href</a></iframe><p>Running this displays the relevant changes, and associated changes in model score, for each counterfactual:</p><pre>Original prediction: [[0.72485193 0.27514807]] ([1])<br></pre><pre>cf prediction 1: [[0.42117135 0.57882865]] ([2])<br>Changed features: [0]<br>cf prediction (feature 0 only): [[0.42117135 0.57882865]] ([2])<br>SHAP weights for changed features: [0.20805611049086478]<br></pre><pre>cf prediction 2: [[0.34094372 0.65905628]] ([2])<br>Changed features: [0]<br>cf prediction (feature 0 only): [[0.34094372 0.65905628]] ([2])<br>SHAP weights for changed features: [0.20805611049086478]<br></pre><pre>cf prediction 3: [[0.49338921 0.50661079]] ([2])<br>Changed features: [ 2 18]<br>cf prediction (feature 2 only): [[0.55759596 0.44240404]] ([1])<br>cf prediction (feature 18 only): [[0.67057841 0.32942159]] ([1])<br>SHAP weights for changed features: [-0.008792957699042644, 0.040407294871898536]</pre><p>We’ll concentrate on the third counterfactual, since its the one with the negative SHAP weighted-feature. As promised we can see that either one of the two feature changes in the counterfactual alone do not yield a different result. Thus <strong>conditioned on</strong> changing the ‘telephone’ feature, the change in the ‘history’ feature has a positive contribution, even though its SHAP weight is negative.</p><p>In kernel SHAP, feature importance is established based on an estimate for the expected score of subsets of changed features conditioned on the values of the other features matching the example. These estimates are based on uniform sampling of the provided dataset, and hence are not sensitive to correlations with features not in the subset. As a result feature importances given by kernel SHAP may not be good estimates of the true Shapley values in the presence of feature correlation. A potential modification to the kernel SHAP formulation is proposed in [6] to address this.</p><p>The scoring of the history feature by kernel SHAP as a negative contributor, even though it is a required component in at least some counterfactuals is likely due to this breakdown in the estimates used by kernel SHAP to approximate true Shapley values.</p><p>Sensitivity to correlation structure could be seen as a strength or a weakness of either method. On the one hand, counterfactuals here could be unlikely under the full joint distribution and therefore not provide practical recourse or explain interesting parts of the space. On the other hand, their presence illustrates model behavior not visible to SHAP feature attribution, and thus explains aspects of that behavior not explained by feature attribution.</p><h3>Conclusions</h3><p>Explanations serve different audiences, who have different requirements. As we have seen looking at just two different techniques, they have different strengths and weaknesses, and will be appropriate to different stakeholders.</p><p>Data scientists and ML practitioners will want to understand model behavior at a nuanced and detailed level. For this group of stakeholders, white-box models and detailed feature attribution models may be necessary.</p><p>End-users, who are impacted by the model’s predictions, will need non-technical explanations of those predictions. While feature attribution may be of some benefit here, counterfactuals are probably more directly interpretable, and have the benefit of showing routes to possible alternative outcomes.</p><p>Auditors and the legal system overlap both camps, and may require the broader picture provided by a diversity of explanations.</p><p>In general a complete picture often requires several perspectives!</p><h3>Citations</h3><p>[1] <a href="https://arxiv.org/abs/1602.04938"><em>“Why Should I Trust You?”: Explaining the Predictions of Any Classifier</em></a>: <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Ribeiro%2C+M+T">Marco Tulio Ribeiro</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Singh%2C+S">Sameer Singh</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Guestrin%2C+C">Carlos Guestrin</a>, 2016</p><p>[2] <a href="https://arxiv.org/abs/1705.07874"><em>A Unified Approach to Interpreting Model Predictions</em></a>: <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Lundberg%2C+S">Scott Lundberg</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Lee%2C+S">Su-In Lee</a>, 2017</p><p>[3] <a href="https://arxiv.org/abs/1703.01365"><em>Axiomatic Attribution for Deep Networks</em></a>: <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Sundararajan%2C+M">Mukund Sundararajan</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Taly%2C+A">Ankur Taly</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Yan%2C+Q">Qiqi Yan</a>, 2017</p><p>[4] <a href="https://arxiv.org/abs/1905.07857"><em>CERTIFAI: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence models</em></a>: <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Sharma%2C+S">Shubham Sharma</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Henderson%2C+J">Jette Henderson</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Ghosh%2C+J">Joydeep Ghosh</a>, 2019</p><p>[5] <a href="https://beenkim.github.io/papers/KIM2016NIPS_MMD.pdf"><em>Examples are not Enough, Learn to Criticize! Criticism for Interpretability</em></a>: Been Kim, Rajiv Khanna, 2016</p><p>[6] <a href="https://arxiv.org/abs/1903.10464"><em>Explaining individual predictions when features are dependent: More accurate approximations to Shapley values</em></a>: <a href="https://arxiv.org/search/stat?searchtype=author&amp;query=Aas%2C+K">Kjersti Aas</a>, <a href="https://arxiv.org/search/stat?searchtype=author&amp;query=Jullum%2C+M">Martin Jullum</a>, <a href="https://arxiv.org/search/stat?searchtype=author&amp;query=L%C3%B8land%2C+A">Anders Løland</a>, 2020</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4107572dff3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/towardstrustedai/explanations-for-ml-models-4107572dff3">Explanations for ML Models — Part 2</a> was originally published in <a href="https://medium.com/towardstrustedai">towards-trusted-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Explanations for ML Models — Part 1]]></title>
            <link>https://medium.com/towardstrustedai/explanations-for-ml-models-part-1-7043303fccd5?source=rss----919abd11bedf---4</link>
            <guid isPermaLink="false">https://medium.com/p/7043303fccd5</guid>
            <category><![CDATA[explanability]]></category>
            <category><![CDATA[mls]]></category>
            <category><![CDATA[ethics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Steve Draper]]></dc:creator>
            <pubDate>Thu, 21 May 2020 12:01:01 GMT</pubDate>
            <atom:updated>2020-05-26T18:46:48.623Z</atom:updated>
            <content:encoded><![CDATA[<h3>Explanations for ML Models — Part 1</h3><p><em>This post is part 1 of a 2-part series. The first part discusses the why and how of explanation techniques in general terms. The </em><a href="https://medium.com/towardstrustedai/explanations-for-ml-models-4107572dff3"><em>second part</em></a><em> discusses and compares some examples.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F2hgzo_ZTS-jSuQ4N57xSQ.png" /></figure><h3>Introduction</h3><p>Deployment of ML models in a real world system, whether in a forecasting or in-the-loop decision-making/assisting role, has real implications for the users or customers of those systems. These may be financial, medical, employment-related, or just quality-of-life implications, depending on the domain of application.</p><p>In this blog we will:</p><ul><li>Motivate the need for explanations of model predictions</li><li>Provide a brief overall survey of approaches to model explanation</li></ul><p>In part two we will:</p><ul><li>Provide example explanations of a loan-granting decision model using two different approaches</li><li>Contrast the two approaches and highlight strengths and weaknesses of each</li></ul><h3><strong>Why Explanations are Important</strong></h3><p>Throughout this blog we will use the running example of a model used to decide (or assist in deciding) whether an individual is to be granted a loan by a bank. This has obvious financial implications for the individual concerned, but also has both financial and possibly legal implications for the issuing bank (which is assumed to be the deployer of the system).</p><p>Because of the direct impact such models have on real lives it is important that the stakeholders (banks and their customers in the running example) are able to trust the model used in decision making. This is both a soft ‘trust’ necessary to garner societal acceptance of such models, and potentially a more tightly defined hard ‘trust’ imposed by framing of the legal system. For example under GDPR the customer here would be entitled to ask why a particular decision about their requested loan was made, and there may be legal requirements that the bank is able to show that their model does not unfairly bias decision making against some sub-group of the population (say by race for example).</p><p>Generating this trust requires the ability to explain a model’s decisions. In part this may be highly technical (perhaps to gain ‘trust’ from domain experts in some auditing function), but in many cases there is a need to provide explanations that are understandable without deep expertise in either the domain (banking) or (especially) machine learning. In particular it needs to be comprehensible to the individual customers of the bank.</p><h3><strong>Approaches to Explanation</strong></h3><p>There are four general families of techniques that are commonly used in model explanation.</p><p><strong>Feature Attribution</strong></p><p>Feature attribution attempts to highlight which features of an individual were important contributors to the decision made. This may be done across all users (a so-called global explanation) or individually for each user and their associated prediction (a local explanation). For the purposes of telling an individual why a decision was made about them, local explanations are preferable, since global explanations will not always align with the specifics of every individual.</p><p>For example: <em>Your loan was denied because of your salary level (40% weight) and your age (30% weight).</em></p><p>Examples of feature attribution methods include such approaches as LIME [1], SHAP [2], Integrated Gradients [3]. Typically these are used to provide local explanations, but these may be aggregated to provide global explanations also.</p><p><strong>Counterfactual</strong></p><p>Counterfactual (sometimes called <em>contrastive</em>) explanations take the form of one or more examples of how an individual’s features could be changed to achieve a different outcome. From the user’s perspective these explanations may be more directly ‘actionable’ than a feature attribution, since each is a specific recipe for achieving a different outcome.</p><p>For example: <em>Your loan would have been granted if you had requested a 6 month longer payback period.</em></p><p>Examples of counterfactual explanation methods include Cortex <a href="https://arxiv.org/abs/1905.07857">Certifai</a> [4] and IBM OpenScale’s <a href="https://cloud.ibm.com/docs/services/ai-openscale-icp?topic=ai-openscale-icp-ie-pp-pn">contrastive explanations</a>.</p><p><strong>Case-based/Prototypical/Exemplars</strong></p><p>Prototypical explanations are based on the similarity of data points (loan-requesting individuals in our example) and seek to explain a prediction in terms of the predictee’s similarity to other users.</p><p>For example: <em>Your loan was denied because you are very similar to another person who defaulted on their loan.</em></p><p>An example of a prototypical explanation method is MMD-Critic [5]</p><p><strong>Rule-based explanations</strong></p><p>Explanations are made according to fixed rules, typically entered by domain experts or induced from historical data.</p><p>For example: <em>Your loan was denied because it was for more than twice your salary.</em></p><h3>Comparison of Approaches and Methods</h3><p><strong>Domain Complexity</strong></p><p>ML models are typically deployed in domains where the relationship between data points (individuals) and predictions (whether a loan should be granted) are complex and not easily described. This is natural, since if this were not the case then simpler approaches — for example, hand-crafted rules — would yield equally good results.</p><p>This domain complexity tends to mean that prototypical and rules-based explanations are of limited use as they typically exhibit lower overall<br>accuracy/AUROC in reproducing the underlying model, at least for complex problems.</p><p>For this reason we will henceforth concentrate on feature attribution and counterfactual explanation techniques.</p><p><strong>Access to model internals</strong></p><p>Model explanation techniques can be further sub-divided into ‘black-box’ and ‘white-box’. In a black-box technique, no knowledge of the model’s internal workings is assumed, and all that is required are samples of predictions it makes for various inputs. In a white-box technique details of the model’s internal structure may be assumed and monitored.</p><p>From an ease-of-deployment and separation-of-concerns perspective, black-box techniques are attractive. They allow arbitrary model types to be deployed. They permit the post-hoc explanation of previously deployed models, which were not designed with explanation in mind, or where we may no longer have access to internal such as gradients. They also limit potential for privacy leakage through limiting what is communicated.</p><p>Conversely, white-box techniques are model-specific, but for the same reason may be more efficient or faithful for the selected model.</p><p>Examples of black box techniques include Cortex Certifai (counterfactual) and Kernel SHAP.</p><p>Examples of white box techniques include Integrated Gradients and Tree-SHAP.</p><p><strong>Stakeholder Preferences</strong></p><p>There is some evidence from research that end-users prefer counterfactual explanations, whereas internal auditors and the modelers themselves may find feature attribution useful. It is therefore possible to make an argument for different explanation techniques to be applied to a single model for different consumers — feature attribution perhaps primarily for an organization’s data scientists and auditors, with counterfactuals provided for the end users.</p><p>Some supporting evidence for this comes from research in the following fields:</p><ul><li><strong>Human Understanding</strong>: Counterfactual explanations are easier to digest while feature-attribution-based explanations are not human-centric -<br><a href="http://www.jennwv.com/papers/interp-ds.pdf">Kaur et al, 2020</a> , <a href="https://arxiv.org/abs/2002.11097">Kumar et al, 2020</a></li><li><strong>Bias/Fairness</strong>: Human subjects understood counterfactual explanations with respect to fairness better than feature-attribution-based methods - <a href="https://arxiv.org/abs/1901.07694">Dodge et al, 2019</a></li><li><strong>Alignment with current laws</strong>: Various papers have demonstrated the advantage of counterfactual explanations over other explanation types from a regulator’s point of view - <a href="https://arxiv.org/abs/1711.00399">Wachter et al, 2017</a></li><li><strong>Alternative Explanations</strong>: Counterfactuals naturally allow multiple alternative explanations with associated choices for recourse. This makes it more likely that an offered explanation will be accepted because it is better aligned with domain knowledge (for businesses) or more actionable (for both businesses and individuals) - <a href="https://arxiv.org/pdf/1706.07269.pdf">Miller, 2018</a></li><li><strong>Concise Explanations</strong>: Certifai allows additional constraints to encourage explanations involving a smaller number of features or attributes. While feature attribution methods can incorporate a “sparsification regularizer”, typically a larger number of features are surfaced as significantly contributing to an explanation - <a href="https://dl.acm.org/doi/abs/10.1145/3375627.3375812">Shubham et al, 2020</a></li></ul><h3>…and in part 2</h3><p>The <a href="https://medium.com/towardstrustedai/explanations-for-ml-models-4107572dff3">follow-up post</a> will go through an analysis of a model used to advise on whether a bank should make a requested loan. We will use <a href="https://www.cognitivescale.com/try-certifai">Cortex Certifai</a> to perform both counterfactual explanation and generate SHAP explanations for several examples, and discuss their strengths and weaknesses.</p><h3><strong>Citations</strong></h3><p>[1] <a href="https://arxiv.org/abs/1602.04938"><em>“Why Should I Trust You?”: Explaining the Predictions of Any Classifier</em></a>: <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Ribeiro%2C+M+T">Marco Tulio Ribeiro</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Singh%2C+S">Sameer Singh</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Guestrin%2C+C">Carlos Guestrin</a>, 2016</p><p>[2] <a href="https://arxiv.org/abs/1705.07874"><em>A Unified Approach to Interpreting Model Predictions</em></a>: <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Lundberg%2C+S">Scott Lundberg</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Lee%2C+S">Su-In Lee</a>, 2017</p><p>[3] <a href="https://arxiv.org/abs/1703.01365"><em>Axiomatic Attribution for Deep Networks</em></a>: <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Sundararajan%2C+M">Mukund Sundararajan</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Taly%2C+A">Ankur Taly</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Yan%2C+Q">Qiqi Yan</a>, 2017</p><p>[4] <a href="https://arxiv.org/abs/1905.07857"><em>CERTIFAI: Counterfactual Explanations for Robustness, Transparency, Interpretability, and Fairness of Artificial Intelligence models</em></a>: <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Sharma%2C+S">Shubham Sharma</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Henderson%2C+J">Jette Henderson</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Ghosh%2C+J">Joydeep Ghosh</a>, 2019</p><p>[5] <a href="https://beenkim.github.io/papers/KIM2016NIPS_MMD.pdf"><em>Examples are not Enough, Learn to Criticize! Criticism for Interpretability</em></a>: Been Kim, Rajiv Khanna, 2016</p><p>[6] <a href="https://arxiv.org/abs/1903.10464"><em>Explaining individual predictions when features are dependent: More accurate approximations to Shapley values</em></a>: <a href="https://arxiv.org/search/stat?searchtype=author&amp;query=Aas%2C+K">Kjersti Aas</a>, <a href="https://arxiv.org/search/stat?searchtype=author&amp;query=Jullum%2C+M">Martin Jullum</a>, <a href="https://arxiv.org/search/stat?searchtype=author&amp;query=L%C3%B8land%2C+A">Anders Løland</a>, 2020</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7043303fccd5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/towardstrustedai/explanations-for-ml-models-part-1-7043303fccd5">Explanations for ML Models — Part 1</a> was originally published in <a href="https://medium.com/towardstrustedai">towards-trusted-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fairness in Azure ML models with Cortex Certifai]]></title>
            <link>https://medium.com/towardstrustedai/fairness-in-azure-ml-models-with-cortex-certifai-5cc5a36069cb?source=rss----919abd11bedf---4</link>
            <guid isPermaLink="false">https://medium.com/p/5cc5a36069cb</guid>
            <category><![CDATA[ethics]]></category>
            <category><![CDATA[certifai]]></category>
            <category><![CDATA[fairness]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Mihail Dungarov]]></dc:creator>
            <pubDate>Mon, 06 Apr 2020 16:53:59 GMT</pubDate>
            <atom:updated>2020-04-06T16:53:59.245Z</atom:updated>
            <content:encoded><![CDATA[<h4>Do taxi drivers charge single travellers more?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_dvX5edB9C6Si9Ef" /></figure><h3>Introduction</h3><p>Are cab drivers fair regardless of the number of travellers? I always felt that when I am alone in the cab, I get charged much more than when there are a few of us, even after splitting the bill.</p><p>I mentioned this once at the pub to a friend, but she felt that it’s the opposite that makes more sense (pointing out my naivety in the same breath): when there are more of you in the cab, you are probably talking and less attentive of the fare, so the driver can take a bit longer at traffic lights or sneak in an extra charge.</p><p>As a reader, you are probably asking what this has to do with ML? It has more in common than what it may seem and today, I want to try and settle this.</p><p>To assess this, we will expand on an existing Azure tutorial using the NYC Taxi Cab dataset. The dataset provides details of taxi cab rides, e.g. distance, hour of the day, etc as well as the final cost of the ride. Using this dataset, we will then demonstrate how Cortex Certifai can be used to assess ML models for fairness even if they are coming from Auto ML, even better — by considering the model as a black box.</p><p>The original Azure notebook tutorial showed how one can build models using great tools like AutoML to get accurate predictions of the ride’s cost. However, these tools stop short of letting us directly inspect the impact of different features on the fare or study if passengers are treated fairly. So how can we understand the model and build trust internally? How can we assess the model’s performance over time? How can we compare models with similar predictive performance? Cortex Certifai allows us to do all of these and more.</p><p>In this post, we will use Certifai (CognitiveScale’s Trusted AI toolkit) to scan ML models for fairness, robustness and explainability. We will do so with models trained with Azure AutoML on NYC Taxi data.</p><p>Here is what we will go through in the rest of the post:<br>Data preparation — done as part of the Azure regression tutorial (<a href="https://notebooks.azure.com/azureml/projects/azureml-getting-started">AutoML regression tutorial</a>).</p><ul><li>The Role of Cortex Certifai</li><li>Outline of the Workflow</li><li>Certifai Usage</li><li>Review of Results</li><li>Logging</li><li>Conclusion</li></ul><h3>The Role of Cortex Certifai</h3><p>Taking a step back, what we are showing here has broader implications. What we will demonstrate is <strong>how Cortex Certifai can be used to assess ML models for fairness even if they are coming from Auto ML, even better — by considering the model as a black box.</strong></p><p>Note, when we are assessing fairness, one might consider taking a demographic parity approach, where we assess the bias in the data directly. However, such an approach would make assessment on the number of passengers abstracting from other features more difficult: bigger groups may tend to travel at specific times of the day or tend to travel greater distances, etc.</p><p>Instead, we are interested in learning <strong>how cab pricing is decided</strong>. We will attempt to learn a model that has high predictive accuracy on the data set. Once we have a model that is consistent with the data, we will evaluate how explainable and fair its decisions are. This isn’t necessarily measuring how fair the cab company’s actual behavior is, because there are multiple models that could be consistent with the data.</p><p><strong>Why is this important?</strong> With more and more applications for AI/ML almost daily, there is more pressure to get the best results quickly. In the past, that has often meant foregoing explainability, which in turn can lead to difficulty assessing fairness and robustness of models.</p><p>Cortex Certifai provides a means to address all of these independently of the underlying model. This is achieved by generating ‘counterfactuals’¹ to the observed data points in order to explore what exact changes to the model lead to different outcomes. Aggregating those allows us to draw conclusions about the model’s fairness, robustness and explainability.</p><p>To learn more, please refer to the <a href="https://arxiv.org/abs/1905.07857">original paper</a>.</p><p>¹ A <em>counterfactual </em>is defined as a generated data point that is as close to the input data point as possible but for which the model gives a different outcome.</p><h3>Outline of the Workflow</h3><p>There are different ways to do the above, but today we will walk you through the following method:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/0*U7MoIXC8rwBYdToB.png" /></figure><ul><li>Data preparation — done as part of the Azure regression tutorial (<a href="https://notebooks.azure.com/azureml/projects/azureml-getting-started">AutoML regression tutoria</a>l).</li><li>Azure AutoML — similar to the tutorial we use Azure’s Auto ML to find the best model fit</li><li>Cortex Certifai — Generation of scans on Certifai for Explainability, Fairness &amp; Robustness</li><li>Finally, we log key statistics into Azure cloud using the Azure SDK</li></ul><p>You can follow all of this in the <a href="https://github.com/CognitiveScale/cortex-certifai-blog/tree/master/cortex_certifai_azure_notebooks_NY_Taxi">accompanying notebook</a>.</p><p>Certifai allows you to assess how different sensitive groups are being treated under a model.</p><p><a href="https://www.cognitivescale.com/try-certifai/">The Certifai toolkit</a> assists with the build and maintenance of ML and digital solutions that users and developers can trust, rely on and understand better. It is designed to provide scores for key characteristics of Trust in decision-making and AI systems such as fairness, robustness and explainability. We are expanding one of the Azure tutorials that predicts taxi fares using the NYC Taxi dataset.</p><p>Azure notebooks are an easy entry point into using the Azure platform and integrate neatly with the Azure cloud ecosystem.</p><p>These two capabilities combined allow us to integrate a Trust component into the development of an ML model/application while making it easy to integrate and deploy models.</p><p><strong>But most importantly, it will help me to prove my friend wrong.</strong></p><h3>Certifai Usage</h3><p>This section focuses on the steps needed to set up and run Certifai scans in a notebook. Once more, you might want to also follow along with the <a href="https://github.com/CognitiveScale/cortex-certifai-blog/tree/master/cortex_certifai_azure_notebooks_NY_Taxi">accompanying notebook</a>.</p><h3>Model Selection</h3><p>The below table shows our AutoML results — we find that the best models are a set of Ensembles. We will pick the best ensemble as our Champion and the best Random Tree (ExtremeRandomTrees, third from the top) as our challenger. In reality, it doesn’t matter which one we end up choosing from Certifai’s perspective as it can explain and scan any one of them</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/0*yur6H7u-zRU7oJ57" /><figcaption>List of models as trained by AutoML sorted by performance</figcaption></figure><p>While AutoML is great at selecting the best performing model from a wide range of models, it often means we need to sacrifice model understanding in the process. This is why Certifai and Azure AutoML are such a great fit. In this way, Certifai helps model developers understand the internals of the model, regardless which model they end up with. We do not care which model AutoML ends up with as long as Certifai treats them all as a black box.</p><h3>Scan Setup</h3><p>Once we have decided which models, it is time to set up the Certifai scan. In a notebooks, a Certifai scan consists of 3 steps</p><ol><li>Task definition</li><li>Adding Models &amp; Datasets to the scan definition</li><li>Adding Evaluations to the scan definition</li></ol><h4>Task definition</h4><p>Here we set:</p><ul><li>Type of task: regression or classification</li><li>Favorable direction/outcome. In our case, we consider it to be favorable for a taxi fare to be lower</li><li>Significant change to outcome.² This specifies when is the regression outcome significantly different as measured by standard deviation. In this case, the empirical standard deviation is about $9.6, hence we consider significant change to be about $5.</li></ul><pre>task = CertifaiPredictionTask(CertifaiTaskOutcomes.regression(<br>        increased_favorable=<strong>False</strong>,<br>        change_std_deviation=0.5),<br>    prediction_description=&#39;Predict taxi fare based on features of the trip&#39;)<br><br>scan = CertifaiScanBuilder.create(&#39;test_user_case&#39;,<br>                                  prediction_task=task)</pre><p>² For an intuition of burden, it represents how much extra “burden” it takes for each group to achieve a more favorable outcome, in this case a lower fare. We are not interested in the overall level of burden but rather whether there are significant differences on how difficult it is for some groups vs others. Please refer to the <a href="https://arxiv.org/abs/1905.07857">Certifai paper</a> for a more formal definition</p><h4>Adding Models &amp; Datasets</h4><p>We add the selected models to the scan. Here, we use a PredictorWrapper</p><p>We also add the dataset by specifying its location. Here this is done via the `all_data_file` value we set earlier when preparing the data for training. Notice that this is simply a pointer to the location of the dataset</p><pre><em># Wrap the model up for use by Certifai as a local model</em><br>champion_model_proxy = CertifaiPredictorWrapper(champion_model, encoder=encoder)<br>challenger_model_proxy = CertifaiPredictorWrapper(challenger_model, encoder=encoder)<br><br><em># Add our local models</em><br>first_model = CertifaiModel(&#39;champion&#39;,<br>                            local_predictor=champion_model_proxy)<br>scan.add_model(first_model)</pre><h4>Adding Evaluations</h4><p>Adding evaluations is now very simple, just list the ones needed and those will be run by the scan and included in the result object.</p><p>Notice that for ‘explanation’ and ‘robustness’, simply adding the evaluation is sufficient to have the report run. For fairness, we also need to specify the actual sensitive feature or features we want to assess ‘fairness’ for. In this case: `passengers` feature</p><pre><em># Setup an evaluation for explanation on the above dataset using the model</em> <br>scan.add_evaluation_type(&#39;explanation&#39;) scan.add_evaluation_type(&#39;robustness&#39;) scan.add_evaluation_type(&#39;fairness&#39;)  scan.add_fairness_grouping_feature(CertifaiGroupingFeature(&#39;passengers&#39;))</pre><h3>Review of Results</h3><h3>Model Explanation</h3><p>We start with model level explanations to gauge what are the main drivers (no pun intended) of taxi fares.</p><p>What we will do here is essentially aggregate the number of times a certain variable has been used for generating a counterfactual, thus assessing its overall importance to the model. The intuition here is: for each datapoint, the counterfactual provides an example of what needs to be changed in order to significantly impact the outcome. When we count up how many times certain features had to be changed over the entire dataset we get a summary of what were the most influential features on the whole</p><p>The outcome is not surprising — by far the most important variable is distance traveled. The second most important is pickup hour, which also makes sense as late evening and early morning hours are charged differently. Number of passengers as well as the actual vendor used seems to play a minimum role in determining the fare. All of this is good as it validates our expectations on how taxi fares work and the sensibility of the models we have in place.</p><p>Notice, this level of detail is rarely available from simply training a model in itself or interpretability might be lost due to latent and/or dummy features generated along the way of making the model work. This analysis, however, is crucial for our understanding, confidence and ultimately trust in the behavior of the model</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/0*uThMkHfPvwJ2SZ3J" /></figure><h3>Model Fairness to groups of different sizes</h3><p>The chart below shows the burden calculated on both models for the “number of passengers” feature. The results from both models are essentially the same. It seems that the burden on all groups is consistent in terms of confidence intervals, despite groups of 4 passengers seemingly receiving<strong> a slightly higher burden average</strong> burden than others. However, those also come with a much wider confidence range which, when taken into account, <strong>makes them not significantly different from other groups.</strong> Additionally, looking at group counts, groups of 4 are also far less present in the sample — only 41 examples out of ca 6250.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*o7UoAt4o_FgVkYRa" /></figure><p>In this section, we learned that the most influential features for determining cost of the trip for our models are the distance travelled and time of day and found <strong>no strong evidence of groups treated differently from each other.</strong></p><h3>Logging</h3><p>Putting our Azure hat on, we want to make sure that results are logged and can be compared with future runs. We can use the Azure SDK to log the entire result object or key metrics such as model fairness, robustness and performance, both as best model and as a comparison</p><p>As we perform multiple runs by either changing the data or changing the models, we will get a set of different fairness values for the champion and challenger</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/0*CDLN0_ikRSLkTHfQ" /></figure><p>Each experiment is then stored and time stamped so we can return back to them and review the best one</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nF15Pl-2ung5pZOU" /></figure><p>Experiments overview shows us the summary of this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3znSxWmtg1UGbOyG" /></figure><h3>Conclusion</h3><p>In this article we set out to assess the NYC taxi cab dataset and determine if there is evidence of taxi fares treating passengers differently based on whether they travel in a big group or alone.</p><p>For this, we decided to train a good predictive model and assess its fairness by using counterfactuals. We chose to use the Azure AutoML for the former and Cortex Certifai for the latter all the while tracking our experiments via the Azure portal.</p><p>What we found is that there is no strong evidence of any group treated unfairly. Despite groups of 3&amp;4 passengers having some more burden than others, the difference is still within confidence bounds and is likely due to fewer observations for those group sizes.</p><p><strong>I may not be able to prove my friend wrong but at least my mind will be at ease next time I grab a cab…</strong></p><p><strong>To try Cortex Certifai for yourself, </strong><a href="https://www.cognitivescale.com/try-certifai/"><strong>download our Toolkit</strong></a><strong> and see transparency, robustness, and explainability in real-time.</strong></p><p><strong>Special thanks to Christine Draper, Rich Knuszka, Jette Henderson, Sinead Willamson and Steve Draper for all their input!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5cc5a36069cb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/towardstrustedai/fairness-in-azure-ml-models-with-cortex-certifai-5cc5a36069cb">Fairness in Azure ML models with Cortex Certifai</a> was originally published in <a href="https://medium.com/towardstrustedai">towards-trusted-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>