<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pacmed on Medium]]></title>
        <description><![CDATA[Stories by Pacmed on Medium]]></description>
        <link>https://medium.com/@pacmedhealth?source=rss-bc7f185fa831------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*9OGsnf8tlNtvrppBrALg-A.png</url>
            <title>Stories by Pacmed on Medium</title>
            <link>https://medium.com/@pacmedhealth?source=rss-bc7f185fa831------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 20:21:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pacmedhealth/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Explainability for tree-based models: which SHAP approximation is best?]]></title>
            <link>https://pacmedhealth.medium.com/explainability-for-tree-based-models-which-shap-approximation-is-best-6df78bc5d086?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/6df78bc5d086</guid>
            <category><![CDATA[tree-ensemble]]></category>
            <category><![CDATA[decision-tree]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[explainable-ai]]></category>
            <category><![CDATA[shap]]></category>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Wed, 12 Jan 2022 09:40:05 GMT</pubDate>
            <atom:updated>2022-01-14T14:14:50.839Z</atom:updated>
            <content:encoded><![CDATA[<h4>Understanding TreeSHAP algorithms’ failure modes</h4><p><em>By Giovanni Baj and Giovanni Cinà (</em><a href="https://pacmed.ai/en/labs"><em>Pacmed Labs</em></a><em>)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vkg0wF-aQg7VxZju-PBxSg.jpeg" /></figure><p><strong>In a nutshell.</strong> At <a href="http://pacmed.ai/en">Pacmed</a> we care about improving medical practice with the help of Artificial Intelligence (AI). We often use tree-based models, and employ SHAP values to understand what the models do. But which SHAP approximation is best to use? In this post we share our findings.</p><h3>Introduction</h3><p>AI models have a huge potential to improve healthcare quality and drive personalized medicine, but their adoption in clinical practice is still very limited. <strong>One of the key barriers to the implementation of AI is the</strong> <strong>lack of transparency</strong> of its algorithms¹. There are plenty of reasons why a certain level of transparency is needed when applying AI in the medical domain, including:</p><ul><li><strong>build users’ trust</strong>: as clinicians are the ones responsible to give the best care to each patient, they should be confident that AI technologies can be trusted, and to make it possible they need to know how the model arrived to a certain prediction</li><li><strong>justify decisions and comply with the ‘right to explanation’</strong>: it is important for clinicians to be able to justify their decision making towards their patients and colleagues</li><li><strong>detect biases</strong>: models interpretability would enable examination for any potential bias (e.g. discrimination based on race, sex or other sensible features)</li></ul><p>Most performant models available now, such as ensembles or deep learning models, are very complex and hard to interpret. Therefore, what we need are systematic methods to make these models transparent and allow humans to understand how a certain prediction was reached. One of the most prominent tools for this purpose is the <strong>SHAP method</strong>, introduced by <a href="https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf">Lundberg and Lee in 2017</a>. SHAP’s basic intuition is very simple, and it consists in <strong>estimating how much each feature contributes to the outcome of a single prediction</strong>.</p><p>In the case of tabular data, machine learning <strong>models based on trees are some of the most popular and powerful models in use today</strong>. To explain this kind of models using SHAP framework, two different algorithms, known as <em>TreeSHAP </em>algorithms, were developed. Which one is best to use? The goal of this article is to show some of the differences between the two algorithms and their potential failure modes.</p><h3>What are SHAP values?</h3><p>SHAP (SHapley Additive exPlanations) is a game theoretic approach to explain the output of any machine learning model. As we have already mentioned, SHAP method attributes to each feature an importance value (named <em>SHAP value</em>) that represents the contribution of that feature to the final outcome of the model. Suppose for example that we have a model <em>f</em>(x) and that we want to explain the prediction of the model on a specific sample <strong><em>x</em></strong>*. To do so, SHAP method decomposes the prediction into the sum of the contributions of all the features, namely</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/272/1*b8TNFFUZM7g_YmAdrvF5eA.png" /></figure><p>where 𝜙₀ is the average model’s prediction. To compute each contribution 𝜙ⱼ, SHAP method relies on a concept that comes from cooperative game theory, known as Shapley value, which is defined as the average marginal contribution of a feature value over all possible coalitions. Translating this definition in formulas, the Shapley value of feature <em>i</em> becomes</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*TbsQ3dyLqQdBYIRpFNcH1w.png" /></figure><p>where the sum is performed over all the possible subsets of features that do not contain feature <em>i </em>and 𝑓ₓ(S)<em> </em>is the prediction of the model when only features in S are known. In this way, the terms [𝑓ₓ(S U<em> {i}</em>) <em>- </em>𝑓ₓ(S)] that we are summing are the marginal contributions of the feature to all the coalitions of features.</p><p>The first thing that should be noticed about the equation above is that we have to perform a sum over all the possible subsets of features. Since the number of subsets grows exponentially with the number of features, it is clear that it is computationally impossible to compute Shapley values applying directly the definition. Therefore <strong>we need efficient algorithms to compute good approximations of SHAP values</strong>. In the case of tree ensemble models, two algorithms (named <em>TreeSHAP </em>algorithms) were specifically designed to explain this type of models in a very efficient way.</p><p>The second important detail of the equation that defines Shapley values is that to compute 𝑓ₓ(S) we should in principle evaluate the model only on a subset of features. Notice, however, that this is not possible for a generic ML model, since we need to evaluate it on <em>all</em> the input features. So the problem is how to “drop” the features that are not in the subset S, keeping in mind that 𝑓ₓ(S) should represent the prediction of the model when we know only the features in S. There are two possible approaches:</p><ul><li><strong>Observational</strong>: The first possibility is to use the expected output of the predictive model conditional on the feature values of the subset S for sample <strong><em>x</em>*</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/346/1*rM7qHUl0bHvL9NE5Jv4Raw.png" /></figure><ul><li><strong>Interventional</strong>: The second possibility is to break the correlation between features and compute the marginal expectation value of the model</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/282/1*3qXxOsj3qGnOiGjm9uKK-Q.png" /></figure><p>Of course, both methods have advantages and disadvantages. For example <em>observational </em>Shapley values suffer from two problems: 1) they can give importance also to features not used by the model, and 2) they can spread importance between correlated features. On the other hand, if we use an <em>interventional</em> approach, we are assuming that features are not correlated, and in this way we are evaluating the model on (potentially) impossible data-points and this can give rise to unreliable explanations.</p><p>Now you may wonder which of the two approaches should be used, but actually there is no general agreement on this point in the literature. Some papers² assert that the interventional approach is the one that should be adopted, other papers³ claim the exact opposite, while still other papers⁴ suggest that the correct approach depends on the specific application. As you can imagine, the problem is quite relevant, since the choice of one approach or the other can lead to different results.</p><p>The distinction between observational and interventional algorithms is very important when we use TreeSHAP method, since also the two available algorithms follow this distinction. More precisely, the two algorithms are:</p><ul><li><strong><em>tree_path_dependent </em></strong>TreeSHAP: it is supposed to observational. Why “<em>supposed”</em>? Well, in the paper⁵ where it was introduced the authors claimed that it was observational, but we will see that it does not have many properties typical of observational approaches.</li><li><strong><em>interventional </em></strong>TreeSHAP: it ignores feature correlation.</li></ul><p><strong>Which algorithm should we use then</strong>? It’s not easy to answer this question in general and with the experiments presented in the following section we will study some relevant aspects of the two algorithms that could help with deciding.</p><h3>Experiments</h3><p>Questions to be answered about <em>tree_path_dependent </em>TreeSHAP:</p><ul><li>How does it deal with irrelevant features?</li><li>Does it spread importance among correlated features?</li><li>In which scenarios does it differ from <em>interventional</em> TreeSHAP?</li></ul><p>We performed all the experiments in Python, where the package <a href="https://shap.readthedocs.io/en/latest/index.html">shap</a> allows to compute SHAP values in a straightforward way. To use the TreeSHAP method one has to rely on the class shap.explainers.Tree. Then, the choice between the two TreeSHAP algorithms can be easily done setting the parameter feature_perturbation, as described in the <a href="https://shap.readthedocs.io/en/latest/generated/shap.explainers.Tree.html#shap.explainers.Tree">package documentation</a>.</p><h4>How does TreeSHAP treat irrelevant features?</h4><p>One of the most contested downsides of <strong>observational methods is that they can attribute importance to irrelevant features</strong>, i.e. features which are not used by the model² ⁶. Is <em>tree_path_dependent</em> TreeSHAP affected by this issue?</p><p>The experiment we implemented to answer this question is the one proposed by Janzing et al², where we have two binary features X₁ and X₂ perfectly correlated and a model that gives its output only based on the first of the two features:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/174/1*cTLlO5gNjB5MqW6wH1hMUw.png" /></figure><p>with the following data distribution</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/358/1*KUJ9SYZ5q_rMymOGvQQLPA.png" /></figure><p>It is clear that when we compute the importance of the two features, we expect that only feature <em>x</em>₁ takes importance. On the other hand, it is shown in the original paper that if we compute analytically the observational Shapley values we obtain an importance for <em>x</em>₂ which is different from zero.</p><p>We implemented this same experiment in Python:</p><ol><li>we generated a data-set with only two binary features perfectly correlated</li><li>we trained a decision tree that uses only feature <em>x</em>₁ to give the prediction</li><li>we explained the model’s predictions with <em>tree_path_dependent</em> TreeSHAP.</li></ol><p>What we have found is that feature <em>x</em>₂ does not take any importance, as you can see in the SHAP summary plot reported here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*NJdsG7vaa4Yd_vNb6yxi1g.png" /><figcaption>SHAP summary plot for a model in which feature x₂ is irrelevant. For a fixed feature, each dot represents a datapoint, and the x-axis position corresponds to the SHAP values assigned to the feature for that datapoint. Each dot is colored based on the the feature value. As we can see <em>x</em>₂ takes zero importance for all the data-points.</figcaption></figure><p>We then repeated the same experiment, but this time using a different explanation algorithm, which this time was “truly” observational (the algorithm used is the observational version of KernelSHAP proposed by <a href="https://www.sciencedirect.com/science/article/pii/S0004370221000539">Aas et al. in 2021</a>). Below we can see that in this case we obtain the expected result, in which also feature <em>x</em>₂ takes importance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*Z0M7e0WKV9IWf0OL3vU0UQ.png" /><figcaption>SHAP summary plot for a model in which feature x₂ is irrelevant, explained with a truly observational method. This time also the second feature takes some importance.</figcaption></figure><p>These results are telling us that <em>tree_path_dependent</em> TreeSHAP is not observational from this point of view, since it does not give importance to irrelevant features. This behavior is also confirmed by an inspection of the algorithm itself.</p><h4><strong>Algorithms comparison when features are correlated</strong></h4><p>The second experiment we performed was to<strong> compare the two TreeSHAP algorithms</strong> in a scenario where in principle there could be differences between their explanations, which is <strong>when there is relevant correlation between features that are not irrelevant</strong>.</p><p>The experiment we performed is the following:</p><ol><li>a dataset of 3 features is sampled from a 3D Gaussian distribution where the last two features have a degree of linear correlation equal to <em>⍴</em></li><li>a binary target variable is defined from a linear combination of the predictors: 𝑦 = sign(a<em>x</em>₁+b<em>x</em>₂+ c<em>x</em>₃)</li><li>a gradient boosting classifier is trained based on this data</li><li>the model is explained using TreeSHAP algorithms</li></ol><p>What we could expect in principle is that when we have high values of correlation between features the two algorithms could give significantly different explanations of the same model. In particular, we know that observational methods could spread importance among correlated features. However, we find that the two algorithms give very similar SHAP values, no matter the value of <em>⍴</em>. In the following we report an example with <em>⍴=0.7</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/1*wSGBepei_5jwUA9dDzcJaQ.png" /><figcaption>SHAP summary plot for SHAP values computed with “interventional” TreeSHAP algorithm.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/1*6ePdM_hT-LK1fx8G-p6ANQ.png" /><figcaption>SHAP summary plot for SHAP values computed with “tree_path_dependent” TreeSHAP algorithm. There are no significant differences with the “interventional” summary plot.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*mTgOYU9uhCLy-y-AqFsgng.png" /><figcaption>Mean absolute values of SHAP values for the 3 features. As expected from the previous two plots, there are no relevant differences between the two algorithms, no matter the value of correlation.</figcaption></figure><p>As before, we repeated the same experiment, this time explaining the model with an algorithm that is explicitly observational (the same mentioned above). In this case we can see a clear spread of importance between correlated features, as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*GIlpNy5xZRWzOpkPQ_UkIg.png" /><figcaption>Mean absolute values of SHAP values for the 3 features, computed with “tree_path_dependent” TreeSHAP and observational KernelSHAP. The observational algorithm is characterized by a spread of importance among correlated features.</figcaption></figure><p>Therefore, also from this point of view <em>tree_path_dependent </em>TreeSHAP does not behave like an observational algorithm, since it doesn’t spread at all the importance between correlated features.</p><h4><strong>Algorithms comparison on local explanations</strong></h4><p>The comparison between the two algorithms that we have proposed in the previous section was from a “global” point of view. In this last experiment we <strong>check if the two algorithms behave differently on local explanations</strong><em>, </em>since computing Shapley values for all the samples in the dataset and then average their absolute values implies losing a lot information.</p><p>We perform an experiment very similar to the previous one, with the difference that this time we have a regression problem:</p><ol><li>𝒙 is sampled from a 3D multivariate Gaussian distribution. There is a correlation <em>⍴</em> between <em>x</em>₀ and <em>x</em>₁</li><li>the target 𝑦 is computed with the decision tree rule reported below</li><li>a decision tree to predict 𝑦 from 𝒙 is trained and then explained with TreeSHAP (notice that the decision tree model obtained after training is equal to the schema used to generate the target, as one would expect)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/308/1*BVSphHD5BZSAhuST5-jo3g.png" /><figcaption>Decision-tree rule to compute the target y from predictors <strong>x</strong>.</figcaption></figure><p>To understand in detail the behavior of the two algorithms as correlations varies, we computed, for a single fixed sample, the SHAP value of each feature as a function of ⍴. The most important results that we obtain is that <em>tree_path_dependent </em>TreeSHAP is influenced by features correlation, but not in the way we were expecting. More precisely, we noticed that when two features are correlated, <strong>the one that is located higher in the tree is the one that gets more importance</strong>.<strong> This effect increases with correlation degree</strong>. For example, look at the plots of sample <em>x=</em>[-1,-1,-1] shown below: we can see that as correlation increases, feature <em>x</em>₁, which is located below <em>x</em>₀, gets lower and lower importance (in absolute value). Conversely, increasing <em>⍴ </em>feature <em>x</em>₀ gets more and more importance.<br>As regards <em>interventional</em> SHAP values, this effect is absent since the importance of both <em>x</em>₀ and <em>x</em>₁ decreases with correlation, and this could be due to the fact that, as feature correlation changes, so does the average model’s prediction (the baseline), and SHAP values change accordingly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/1*iLkY9U6qbG8ke0nDVgufvw.png" /></figure><h3>Conclusions</h3><p>Explainable AI is often a requirement if we want to apply ML algorithms in high-stakes domains like the medical one. A widely used method to explain tree-based models is the TreeSHAP method, which comprises two algorithms. In this article we have presented some experiments to study the behavior and the differences between the two. The most important results are:</p><ul><li><em>(</em><strong><em>the good news</em></strong><em>) tree_path_dependent </em>TreeSHAP is not observational strictly speaking, since it does not give importance to irrelevant features and it does not spread importance among correlated features</li><li><em>(</em><strong><em>the bad news</em></strong><em>) tree_path_dependent </em>TreeSHAP, when there is correlation between features, tends to give more importance to early-splitting features, penalizing downstream features</li></ul><p>This behavior should be kept in mind when <em>tree_path_dependent </em>TreeSHAP is used to investigate the inner workings of tree models. While this algorithm does not seem to share the failure modes of other observational algorithms, one has to be mindful that the feature importance might be unreliable due to the second finding; shuffling the feature list or sampling subsets of features when building tree ensembles could be two effective ways to mitigate this problem.</p><p><strong>References</strong></p><p>[1]: A. F. Markus et al, “<a href="https://www.sciencedirect.com/science/article/pii/S1532046420302835">The role of explainability in creating trustworthy artificial intelligence for health care: A comprehensive survey of the terminology, design choices, and evaluation strategies</a>”, <em>Journal of Biomedical Informatics </em>(2021)</p><p>[2]: D. Janzing et al, “<a href="https://arxiv.org/abs/1910.13413">Feature relevance quantification in explainable AI: A causal problem</a>”, <em>International Conference on Artificial Intelligence and Statistics </em>(2020)</p><p>[3]: C. Frye et al, “<a href="https://arxiv.org/abs/2006.01272">Shapley explainability on the data manifold</a>”, <em>ICLR </em>(2021)</p><p>[4]: H. Chen et al, “<a href="https://arxiv.org/abs/2006.16234">True to the Model or True to the Data?</a>”, <em>arXiv preprint arXiv:2006.16234</em> (2020)</p><p>[5]: S. Lundberg et al, “<a href="https://arxiv.org/abs/1802.03888">Consistent Individualized Feature Attribution for Tree Ensembles</a>”, <em>arXiv:1802.03888 </em>(2018)</p><p>[6]: M. Sundararajan and A.Najmi, “<a href="https://arxiv.org/abs/1908.08474">The many Shapley values for model explanation</a>”, <em>PMLR</em> (2020)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6df78bc5d086" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Out-Of-Distribution Detection in Medical AI]]></title>
            <link>https://medium.com/geekculture/out-of-distribution-detection-in-medical-ai-b638b385c2a3?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/b638b385c2a3</guid>
            <category><![CDATA[medical-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[uncertainty-estimation]]></category>
            <category><![CDATA[out-of-distribution]]></category>
            <category><![CDATA[responsible-ai]]></category>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Mon, 21 Jun 2021 08:50:31 GMT</pubDate>
            <atom:updated>2021-06-25T07:26:29.714Z</atom:updated>
            <content:encoded><![CDATA[<h4>Why it is a problem and a benchmark to find a solution</h4><p><em>By Karina Zadorozhny and Giovanni Cinà (</em><a href="https://pacmed.ai/en/labs"><em>Pacmed Labs</em></a><em>)</em></p><blockquote><strong>This blogpost is for you if:</strong> you are interested in deploying medical AI in real-world contexts.</blockquote><h3>Introduction</h3><p>As Machine Learning is poised to revolutionize healthcare, we need to pause and think about all the possible ways in which our super-performing models may not work well in practice. One key reason for this is the fact that real <strong>live data may differ from training data</strong>, namely the data that the model has learned from. And even if live data looks reasonably like training data now, there is no guarantee that it will stay so <em>in the future</em>.</p><p>In healthcare settings, there are multiple reasons why patient data can change over time, for example:</p><ul><li>The patient population changes (due to change in demographics, to health policies, etc).</li><li>The protocol of care changes (e.g. a parameter is measured differently, or treatment behavior is modified).</li><li>Due to a previously undiscovered bug in the AI tool or in the healthcare infrastructure the tool connects to.</li><li>Due to cyberattacks or other intentional tampering with the data.</li></ul><p>It is therefore likely that a model in production will eventually receive data points that are different from training data.</p><h3>The problem of OOD detection</h3><p><strong>Machine learning models assume that new samples are similar to data they have been trained on</strong>. More precisely, we assume that the data is independent and identically distributed. Samples that are similar to training data are considered to be <em>in-distribution.</em></p><p>However, in practice, we have no guarantee that a model will only see data that it similar to data it has been trained and tested on. Samples that are not well represented in the training dataset are considered to be <em>Out-Of-Distribution </em>(OOD). As described above, <strong>there are many clinically relevant scenarios that can lead to changes in data distribution.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TyogHTgvJx8mTemKYHJ0cg.png" /></figure><p>For safe deployment of machine learning models in healthcare, <strong>it is crucial that we are able to flag ‘weird’ samples, since on those samples the model’s output is not reliable</strong>. Moreover, we want to flag such OOD samples in real time, or otherwise a large amount of errors might pile up before we realize the data has changed.</p><p>To be able to tell how much we can trust a model, we would like to obtain a separate <em>uncertainty</em> score instead of getting only a prediction per each sample. This uncertainty can then be employed to flag OOD patients for which the prediction should not be trusted.</p><h3>Approaches to OOD detection</h3><p>Given the significance of the task, a variety of methods for uncertainty quantification have been proposed in recent years. Broadly speaking, we can either be uncertain about a model’s <em>prediction </em>or about a new<em> sample </em>itself<em>. </em>Using this intuition, we can divide OOD detectors into two groups:</p><ol><li><strong>Discriminators </strong>express uncertainty in their own predictions.</li><li><strong>Density Estimators </strong>learn the density of training features and flag samples that are in the areas of low density.</li></ol><p><strong>Which technique is best at flagging OOD samples?</strong> In the rest of this section we list the methods we benchmarked on the OOD detection task, focusing on real-world medical data. You can find all the details in <a href="http://proceedings.mlr.press/v136/ulmer20a/ulmer20a.pdf">this paper</a>, including detailed descriptions of all the models and metrics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DNp_fv7bGdv5Oqpt_NNUQw.png" /><figcaption>Shades of orange and blue indicate confidence levels. For discriminator, the regions of low certainty are defined by model’s decision boundaries. For density estimators, the areas with low sample density will end up with low confidence levels.</figcaption></figure><h4><strong>1. Discriminators</strong></h4><p>A discriminator is a model that outputs a prediction based on sample’s features. Discriminators, such as standard feedforward neural networks or ensemble networks, can be tweaked to provide a score that indicates how certain they are in their predictions.</p><p>The discriminator models we studied include:</p><ul><li>Logistic regression</li><li>Feed-forward neural networks</li><li>Temperature-scaled neural networks</li><li>Ensemble of neural networks</li><li>Bayesian neural networks</li><li>Monte Carlo Dropout</li></ul><p>There are several simple metrics that can be added to quantify uncertainty of a network in a classification setting. We used:</p><ul><li>Maximum softmax probability</li><li>Entropy</li><li>Variance</li><li>Mutual Information</li></ul><h4><strong>2. Density Estimators</strong></h4><p>As the name suggests, density estimators learn to estimate density of training data. To see what this means, suppose that there is some process that generates our training data — for example, a process of collecting temperature measurements for a patient. To learn a density function of this process means to learn how likely it is to measure a specific value.</p><p>By learning density of all features of training data, density estimators can tell whether a new sample comes from the same distribution. Some density models can, in principle, also output predictions about labels by learning a joint distribution of features and targets. However, density estimators are most commonly used separately from a main prediction model as they can flag OOD samples in a model-agnostic way.</p><p>Models we included in this category are*:</p><ul><li>Probabilistic PCA</li><li>Autoencoder</li><li>Variational Autoencoder</li><li>Local Outlier Factor</li><li>Neural Gaussian Process</li></ul><p><em>*Note that explicit density estimators give us a way to express learned data distribution — for example, as parameters of a normal distribution. In this list we included models such as Local Outlier Factor that do not have this ability.</em></p><h3>Experiments on Real-World Public Medical Data</h3><p>As described above, there is a plethora of OOD novelty detection techniques. But which ones of them work and when?</p><p>To help answer this question in the medical field, we have created a set of <strong>benchmark experiments on publicly available medical tabular data</strong> (Electronic Health Records, EHR). We implemented the models described above and designed several clinically relevant experiments that could help us find the best OOD detector(s).</p><p>Specifically, we designed three experiments to mimic clinically relevant scenarios:</p><ol><li><strong>Detecting corrupted features </strong>is relevant<strong> </strong>if a measurement device outputs a faulty value.</li><li><strong>Detecting clinically relevant OOD groups </strong>is relevant if<strong> </strong>a model receives a group of patients with a novel disease or demographics that is different from training data.</li><li><strong>Detecting source of a dataset </strong>is relevant if a patient data comes from a new hospital which can have different procedures.</li></ol><p>The full description of the experiments and the results can be found in <a href="http://proceedings.mlr.press/v136/ulmer20a/ulmer20a.pdf">Ulmer et al., 2020</a> [1]. <a href="https://github.com/Pacmed/ehr_ood_detection">Here</a> is a link to the open source repository with the implementations of the models and experiments.</p><h4>Datasets</h4><p>We used two publicly available datasets:</p><ul><li><a href="https://mimic.mit.edu/">MIMIC-III </a>(Medical Information Mart for Intensive Care) which contains information about patients admitted to critical care units that spans more than a decade. Features in this dataset include laboratory test results, diagnostic codes, or demographics.</li><li><a href="https://eicu-crd.mit.edu/">eICU</a> (electronic Intensive Care Unit) database contains data from critical care units across the United States in years 2014 and 2015. Similar to MIMIC, it contains laboratory measurements, length of stays, and demographics of patients.</li></ul><h4>Mortality prediction task</h4><p>All discriminator models were trained for a binary classification task of in-hospital mortality*. That is, discriminator models are<em> not trained </em>on OOD detection but on the actual task they are going to be used for.</p><p>Given the low rates of in-hospital fatalities, mortality prediction is a highly unbalanced task. Therefore, it makes more sense to measure the AUC-ROC score rather than accuracy. The performance of models was the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xg60EpuC3Rv9iiPv01qjtg.png" /></figure><p>The density models were trained without labels, only on the features of the training data.</p><h3>OOD Detection Experiments</h3><h4>1. Detecting corrupted features</h4><p>Consider the following scenario: a patient has been subjected to a laboratory test but the measuring device outputs a flawed measurement with physiologically non-sensical values. Naturally, instead of getting a prediction for samples for which this malfunctioning device was used, we want these samples to be flagged as OOD.</p><p>To mimic this scenario, we selected a random feature and then scaled it by a factor (10, 100, 1000, or 10 000). Then, we compared how well each model can flag such scaled samples. The results were averaged over 100 experiments with different randomly selected features to prevent any feature bias.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VvBYSRb2sNBrV9-s2-IcSQ.png" /><figcaption>The perturbation experiment on the eICU dataset. A random feature was scaled by a factor of 10, 100, 1000, or 10 000 and the models were compared in their ability to flag these perturbed samples. In the image, the final AUC-ROC score for OOD detection is shown. It is an average over 100 experiments with randomly selected features.</figcaption></figure><p>The expected trend would be to see the AUC-ROC score increase (darker square) as the scaling factor increases. For most of the discriminators, it is actually the opposite! Their ability to flag corrupted samples decreases with a larger and larger flaw in the measurement. This means that models are assigning very high certainty score to corrupted samples. Density models perform much better in this test.</p><h4>2. Detecting clinically relevant OOD groups</h4><p>Another relevant scenario is observing a new group of patients that the model has not been trained on. This can be due to a new underlying disease or simply a demographic shift in the patient population.</p><p>Each clinically relevant group (e.g., patients with a specific disease such as renal failure, or with a specific demographic such as sex or ethnicity) was withheld during training of the models. We then introduced these patients at test-time and compared how well they are flagged as OOD.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wyagA9ep5spxWOEX3Jcgaw.png" /><figcaption>Clinically relevant OOD groups for the MIMIC dataset. We first withhold a particular group during training and then compare the AUC-ROC scores of detecting the indicated OOD groups at test time. <em>Diff</em> shows how different the features of the group are compared to the features in the training data. (Average results over 5 experiments are shown).</figcaption></figure><p>With the exception of newborns, which is the most distinct group, models detected the OOD groups very poorly. In this test, even the density models failed.</p><h4>3. Detecting new datasets</h4><p>As a third experiment, we used the data source as an OOD criterion. This is also a relevant problem: changing hospital protocols can result in imprecise predictions.</p><p>To test this, we can train the models on the MIMIC dataset and then test its uncertainty on the eICU dataset (or the other way around). This is different from a domain adaptation problem as we are not looking on the predictive performance of models but on their ability to distinguish samples coming from two different datasets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H2aQR6qpIDwzHVPGLRklGQ.png" /><figcaption>Detecting samples coming from a different dataset. The models were trained on the MIMIC dataset and then presented with the samples from eICU (top row), or visa versa (bottom row). The scores show AUC-ROC of detecting the new datasets as OOD. <em>Diff</em> shows how different the features of the group are compared to the features in the training data. (Average results over 5 experiments are shown).</figcaption></figure><p>Similar to the perturbation experiments, the density models performed better than the discriminator. It seems that they can flag samples coming from a new dataset more reliably than neural discriminators do.</p><h3>Conclusions</h3><p>In this blog post we introduced the problem of OOD detection and highlighted its importance for reliable deployment of Machine Learning models in healthcare. There are many different techniques for OOD detection and uncertainty estimation and they can be broadly classified into two groups — discriminators and density estimators.</p><p>As seen from the real-world experiments on medical tabular data, the <strong>methods are not perfect and fail to detect some types of OOD</strong>. This stresses the need for more robust novelty detection approaches.</p><p>Our hope is that by providing an open source code for the experiments and using publicly available datasets, our experimental framework can serve as an easy-to-use benchmark for new techniques.</p><h3>References</h3><p>For further reading, please see our original paper:</p><p><em>Ulmer, D., Meijerink, L., and Cinà, G. Trust Issues: Uncertainty Estimation does not Enable Reliable OOD Detection on Medical Tabular Data. In Proceedings of the Machine Learning for Health NeurIPS Workshop, volume136, pp. 341–354, 2020.</em></p><p>And in this paper, we provide a theoretical explanation of why neural discriminators systematically fail at the perturbation task:</p><p><em>Ulmer, D. and Cinà, G. Know Your Limits: Uncertainty Estimation with ReLU Classifiers Fails at Reliable OOD Detection.arXiv:2012.05329, 2021</em></p><p>All code used for the experiments and models described here can be found at:</p><pre><a href="https://github.com/Pacmed/ehr_ood_detection">https://github.com/Pacmed/ehr_ood_detection</a></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b638b385c2a3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/out-of-distribution-detection-in-medical-ai-b638b385c2a3">Out-Of-Distribution Detection in Medical AI</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Causal inference for medical AI: what can we learn from observational data of COVID-19 patients?]]></title>
            <link>https://pacmedhealth.medium.com/causal-inference-for-medical-ai-what-can-we-learn-from-observational-data-of-covid-19-patients-c052b4b68753?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/c052b4b68753</guid>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Tue, 02 Mar 2021 08:45:08 GMT</pubDate>
            <atom:updated>2021-03-16T15:31:54.353Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By Giovanni Cinà (</em><a href="https://pacmed.ai/en/labs"><em>Pacmed Labs</em></a><em>)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/0*ehg1hoKIVfUUJFAt" /></figure><p><strong>This blog post is for you if</strong>: you work in healthcare, you are interested in AI, you are binge-reading content on COVID-19 while splayed on the couch.</p><h3>Intro</h3><p>At the time of writing, it has been more than a year since the onset of the COVID-19 pandemic. Among all the ways in which our lives have been changed, we have all been presented with a very pressing question: <strong>how do we find a cure?</strong></p><p>The problem of finding effective medications and vaccines, an issue that used to concern only a restricted group of experts, suddenly took a considerable amount of airtime on mass media, social media and our face-to-face conversations (remember those?). Begrudgingly, we had to came to grips with the idea that <strong>proving treatment effectiveness is a tricky process</strong>.</p><p>The method of choice for establishing the effect of a treatment is a <a href="https://en.wikipedia.org/wiki/Randomized_controlled_trial#:~:text=A%20randomized%20controlled%20trial%20(or,groups%2C%20treating%20them%20differently%2C%20and"><strong>Randomized Controlled Trial</strong></a> (RCT): a population of suitable patients is randomly divided into two groups, one receiving the treatment (the treated group) and one receiving no treatment or a placebo (the control group). The groups are then monitored for a certain time to check whether the desired outcome is different in the two groups.</p><p>We will not go into the advantages and disadvantages of RCTs; it suffices to say, as you might have noticed waiting for a vaccine, that <strong>it can be a long procedure</strong>.</p><p>But what about the people that are sick now? What can we do while we wait for the results of the RCTs? The most obvious answer is to start employing <strong>treatments that worked for diseases similar to COVID-19. But how do we know if they work well?</strong> Read on.</p><h3>Collecting data of COVID-19 patients</h3><p>To be able to say anything at all, we must first collect the data of COVID-19 patients. In the Netherlands, an <a href="https://covidpredict.org/">unprecedented collaboration between Intensive Care Units (ICU) across the country</a>, led by the Amsterdam University Medical Center, resulted in the <strong>collection of a vast database containing the information of COVID-19</strong> patients from the ICUs of dozens of hospitals. You can find a description of the dataset in <a href="https://link.springer.com/article/10.1007/s00134-021-06361-x">this letter</a>, published on the Journal of Intensive Care Medicine.</p><p>This mass-scale data gathering allowed us to realize that there has been a <strong>certain amount of variation in the way treatments have been administered to COVID-19 patients</strong>. Doctors have tried different dosages of medications, different settings on the ventilators, and so on. This is not surprising, given that there were no specific guidelines on how to treat COVID-19 patients. These different behaviors might look like a sign of disorganization, but they do however offer the possibility to study the effectiveness of treatments.</p><h3>Estimating treatment effect</h3><p>The problem of assessing the efficacy of a treatment is always about answering the “what if” question, aka the counterfactual question. Let’s look at an example.</p><h4>The headache-aspirin example</h4><p>(Disclaimer: this example is completely made up for explanatory purposes, it does not contain any medical advice.)</p><p>Suppose you want to test the effectiveness of aspirin in curing headaches. Perhaps the first sensible thing we can think of is: giving aspirin to someone with headaches and checking if they feel better than before. But that’s not enough.</p><p>What we really want to know is: <strong>does the person feel better after taking aspirin compared to not taking it</strong>? Or in other words, is treatment better than not-treatment? It could be, for example, that headache disappears after a while and aspirin is not needed. The problem is that <strong>for each person we can only observe the outcome of one action, we cannot observe the counterfactual scenario</strong> (the one that did not occur). This is sometimes called ‘the fundamental problem’ of causal inference.</p><p>What we ideally want to do is to have two (very similar) people with headache, give treatment to one and no treatment to the other, observe the results and compare. You see that we are moving towards a RCT-like setup. In an RCT, using large groups makes it easier to overcome individual differences and have two relatively similar populations, while the random treatment assignment makes sure that there is no other factor influencing treatment assignment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*Umo9-GORy35b6J3m.jpg" /><figcaption>The subject of this example, headache. Incidentally, also a potential side-effect of this article.</figcaption></figure><h4>Back to the main story</h4><p>In summary, to be able to say something reasonable from observational data we need to</p><ul><li>identify two groups of patients, one that received treatment and one that did not;</li><li>check that the two groups are very similar, meaning that the distributions of parameters are similar;</li><li>make sure that all variables influencing treatment have been measured;</li><li>check that each patient could have received either the treatment or the non-treatment.</li></ul><p>The last two are finer technical point, we will not go into the details here; it suffices to say that the last point connects back to the intuitions about treatment variability and the “what if” question. <strong>If there is a strict guideline and doctors always assign treatment (say) to some kind of patients, then we have no information about the counterfactual scenario</strong> and cannot answer the “what if” part of the problem.</p><h3>What can we do then about COVID-19</h3><p>Now that the background is out of the way, we can see that there are two key questions remaining:</p><ol><li>Is there a treatment for COVID-19 for which the above conditions are fulfilled on the Dutch data and we can estimate its efficacy?</li><li>What are the best methods to carry out this estimation?</li></ol><p><strong>Thanks to the generous funding of </strong><a href="https://www.sidnfonds.nl/"><strong>SIDN</strong></a><strong>, and under the auspices of the consortium of Dutch ICUs led by the Amsterdam University Medical Center, we were able to investigate the ICU dataset and find a medical procedure for which it seems possible to answer both questions.</strong></p><p>The medical procedure in question is called <a href="https://en.wikipedia.org/wiki/Proning">proning</a>, and it is routinely used to treat Acute Respiratory Distress Syndrome and improve patients’ breathing. This maneuver was used often on COVID-19 patients but not always in the same fashion, thus there is enough treatment variability in the data.</p><p><strong>In conclusion</strong></p><p>In this post we went through the basics of treatment effect estimation, and described what are the challenges of carrying out such estimation from observational data. If you would like to know more about the fascinating field of causal inference, there are several resources available online (<a href="https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/">here</a> for example you can find the online version of a popular textbook).</p><p>We are preparing two manuscripts to summarize our findings on the effect of proning on COVID-19 patients and answer the two questions mentioned above. They will be submitted to peer-review to ensure the methodology is sound and subsequently released open source, along with the code.</p><p>Stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c052b4b68753" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Door Wouter Kroese op Emerce]]></title>
            <link>https://pacmedhealth.medium.com/door-wouter-kroese-op-emerce-614346517be7?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/614346517be7</guid>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Wed, 06 Nov 2019 13:17:13 GMT</pubDate>
            <atom:updated>2019-11-18T11:32:25.605Z</atom:updated>
            <content:encoded><![CDATA[<h3>Machine learning helpt de zorg levensreddende stappen te maken</h3><p><em>Door Wouter Kroese op Emerce</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/1*7fHDVwZSQfdhWw064Gzg4Q.png" /></figure><p><strong>Het menselijk lichaam is ontzettend complex. Tien biljoen cellen, aangestuurd door een genoom van drie miljard letters, vele honderden ingewikkelde processen en een brein dat we nog maar voor een heel klein deel begrijpen. Nog complexer is het genezen van dit lichaam als er ergens iets mis gaat.</strong></p><p>Er is, zeker in Nederland, ontzettend veel kennis over ziektes en behandelingen. Als zorgverlener wil je echter vooral het leven van die specifieke patiënt aan het bed of tegenover je beter maken. Alle huidige kennis biedt zorgverleners helaas weinig ondersteuning bij het goed in te schatten wat er met een individuele patiënt staat te gebeuren of wat de verwachte uitkomsten van de verschillende behandelopties zijn voor deze specifieke patiënt.</p><p><strong>Machine learning</strong></p><p>Machine learning is een technologie die juist ontzettend goed in staat is om in ingewikkelde situaties dergelijke individuele voorspellingen te doen. Met machine learning is het mogelijk om computers zelf te laten leren van grote hoeveelheden data, om op basis daarvan voor nieuwe gevallen voorspellingen te kunnen doen. Deze technologie gebruiken we dagelijks om ons een weg te banen in de digitale wereld, als we iets opzoeken op Google of als Spotify ons een volgend nummer aanbeveelt. De ontwikkelingen op het gebied van machine learning gaan razendsnel. In zelfrijdende auto’s loodsen machine learning algoritmen mensen tegenwoordig zelfs zelfstandig door levensgevaarlijke situaties.</p><p><strong>Synergie tussen mens en computer</strong></p><p>Onder andere door de enorme schaal en de complexiteit van de zorg, lijkt machine learning er van enorme waarde te kunnen zijn voor patiënten. Toch zien we deze technologie in de zorg nog weinig. Een van de belangrijke redenen hiervoor is dat het genezen van een patiënt veel moeilijker is dan het aanbevelen van een video of advertentie. Het artsenvak is niet volledig in data te vangen en al helemaal niet in de data die op dit moment in de zorg beschikbaar zijn. Daarnaast is de definitie van kwaliteit van leven en kwaliteit van zorg niet zwart-wit en erg subjectief. De inzet van machine learning in de zorg behoeft daarom enorme zorgvuldigheid en kan alleen van waarde zijn wanneer een synergie gecreëerd wordt tussen de onvervangbare kennis en kunde van zorgverleners en de toenemende mogelijkheden van computers.</p><p><strong>Maximale potentie</strong></p><p>Om de waarde van machine learning daadwerkelijk te benutten moet samengewerkt worden met het gehele veld. Dit kan alleen als artsen, verpleegkundigen, patiënten, data scientists, zorgbestuurders (van verschillende instellingen), beleidsmakers, ethici, ondernemers, onderzoekers, zorgverzekeraars en vele anderen de handen ineenslaan. En alsnog zal er dan veel tijd, geld en energie nodig zijn om de maximale potentie van de technologie te benutten.</p><p><strong>Intensive care</strong></p><p>Maar dit is geen reden om hier niet nu al mee aan de slag te gaan, integendeel. Want er zijn nu al gebieden in de zorg waar we met deze technologie van grote waarde kunnen zijn. Neem bijvoorbeeld de intensive care, met de grote hoeveelheden data die continu verzameld worden. Misschien móeten we nu zelfs al wel met machine learning aan de slag. Want de zorg staat voor een aantal enorme uitdagingen. De kosten van dure geneesmiddelen stijgen hard. Door vergrijzing en verbeteringen in de zorg hebben we enorme aantallen kwetsbare ouderen met chronische aandoeningen.</p><p><strong>Cruciale bijdrage</strong></p><p>Tegelijkertijd is er een groot tekort aan medisch personeel, met name in de acute zorg. Machine learning kan een cruciale bijdrage leveren aan de oplossing van deze problemen, door ervoor te zorgen dat de beperkte middelen daar worden ingezet waar ze van grootste waarde zijn voor patiënten en door echt op maat gemaakte en digitaal ondersteunde zorg mogelijk te maken. En wat gebeurt er als we de handen nu niet gezamenlijk met deze technologie aan de slag gaan? Dan lopen we het risico dat machine learning zijn intrede in de Nederlandse zorg doet op een wijze waarbij het medisch veld de regie verliest, er geen garantie is dat de machine learning algoritmen in lijn zijn met de ethische waarden die wij in de Nederlandse zorg hebben en is er geen garantie dat deze software structureel ten gunste komt van patiënt, zorgverlener en premiebetaler.</p><p><a href="https://www.emerce.nl/opinie/machine-learning-helpt-zorg-levensreddende-stappen-maken">https://www.emerce.nl/opinie/machine-learning-helpt-zorg-levensreddende-stappen-maken</a></p><p>#machinelearning #intensivecare #responsibleai</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=614346517be7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From paper to patient, Pacmed’s AI newsletter #2: on interpretability]]></title>
            <link>https://pacmedhealth.medium.com/from-paper-to-patient-pacmeds-ai-newsletter-2-on-interpretability-7970815465e0?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/7970815465e0</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[healthcare-innovations]]></category>
            <category><![CDATA[healthcare-technology]]></category>
            <category><![CDATA[model-interpretability]]></category>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Tue, 14 May 2019 15:34:35 GMT</pubDate>
            <atom:updated>2019-05-17T13:44:42.274Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By Giovanni Cinà and Michele Tonutti (Data Scientists at </em><a href="https://pacmed.ai/en/"><em>Pacmed</em></a><em>)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ozD0FAX8QR0uwYf9FU_E2w.jpeg" /><figcaption>Helping doctors and data scientists get a clear view on the latest developments in medical AI</figcaption></figure><p>As AI agents grow in skills and complexity, the interaction between humans and machines becomes more important than ever. The true added value seems not to lie in Artificial Intelligence alone, but in the synergy between human and technology. Understanding well how the technology operates is key to the facilitation of this synergy. How can we understand and trust the decisions or predictions of an artificial intelligence? This question is even <a href="https://arxiv.org/pdf/1712.09923.pdf">more pressing in the medical domain</a>, where AI can directly influence the health of many.</p><p>In this issue of our newsletter we focus on <a href="https://en.wikipedia.org/wiki/Explainable_artificial_intelligence">explainable AI</a> .This challenge holds a special place in our Pacmed hearts. Observational data and algorithms based on these data may not reflect the whole situation of the patient, so we think that the synergy between doctor and computer is needed for responsible deployment and clinical value. That is why we think our algorithms should always be explainable to the degree that we can guarantee safe use. We have curated a small selection of papers on this topic; this is by no means a comprehensive list but rather a good place to start if you are new to the issue or a check to see if anything important escaped your attention.</p><p><strong>#1: Addressing the issues of interpretability from a general standpoint </strong><em>Interpretable machine learning: definitions, methods, and applications <br></em>By: James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi- Asi et al <a href="https://arxiv.org/pdf/1901.04592.pdf">https://arxiv.org/pdf/1901.04592.pdf</a></p><p>Interpreting Machine Learning models is a multi-faceted problem, involving many steps of a project, from feature engineering to deployment. Framing the issue, breaking down its complexity, inventorizing the different techniques and discussing examples: these steps are often overlooked in favor of the assessment of the latest and coolest models and methods.</p><p>The first paper we propose takes a step back and puts interpretability in a broader context — it is therefore a good primer for those that are not already acquainted with the topic.</p><p><strong>Why it matters.</strong> Among other aspects, the authors stress the <strong>importance of relevancy, namely the selection of an interpretation method that is apt for the intended audience</strong>, thus putting human-machine interaction in the center. This approach resonates with us at Pacmed, since we strive to build models that are accessible specifically for medical personnel.</p><p>Furthermore, the article offers an overview of the different approaches and discusses a number of relevant examples where techniques are shown to (sometimes not) work.</p><p><strong>#2: SHAP: a method for interpretable models inspired by game theory<br></strong><em>A unified approach to interpreting model predictions</em><br>By: Scott Lundberg and Su-In Lee<br><a href="http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf">http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf</a></p><p>One of the issues with interpretability methods is the high number of available approaches, and a lack of clear view on which one is the most suitable for which task. The 2017 paper by Lundberg and Lee, presented at <a href="https://nips.cc/">NeurIPS</a>, proposes SHAP, a unified approach based on additive feature importance, meaning that the output of a model can be approximated by the sum of the effect of each individual feature. The proposed solution is based on game theory and local approximation, and has been shown to be the only solution displaying certain desirable properties for explanation models: local accuracy, missingness, and consistency. The method unifies 6 existing methods, amongst which <a href="https://dl.acm.org/citation.cfm?id=2939778">LIME</a>, and three approaches from game theory.</p><p><strong>Why it matters. </strong>SHAP offers a new way to explain both global feature importance (i.e. which features contribute to the prediction of the trained model) and single observations (i.e. which characteristics of a specific sample contribute to the prediction for that sample); it works for virtually any type of model, including deep neural networks. Aside from the mathematical validity and innovativeness of the approach, the main advantage of SHAP from the point of view of the end-user is that it has been proved to be rather consistent with human interpretation, and it is relatively easy to interpret. In particular, the additivity aspect of the explanation model enables the importance of each feature to be understood intuitively in relation to the output of the model. From a computational perspective, it is also a somewhat efficient method, especially for tree-based algorithms, as explained in more detailed in a <a href="https://arxiv.org/abs/1802.03888">second paper</a> by the same authors on the subject.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/294/1*TBQPUw6ijrIGhuhnBKzbVA.png" /><figcaption>Example of a SHAP plot.</figcaption></figure><p><strong>#3: Explaining the predictions of a model to prevent hypoxaemia during surgery using SHAP <br></strong><em>Explainable machine learning predictions for the prevention of hypoxaemia during surgery<br></em>By: Scott Lundberg, Bala Nair, Monica S. Vavilala et al.<br><a href="https://www.nature.com/articles/s41551-018-0304-0">https://www.nature.com/articles/s41551-018-0304-0</a></p><p>Because of its enhanced interpretability and consistency, SHAP is particularly suitable for clinical applications. This paper by a research group from the University of Washington employs SHAP to provide intuitive and useful visual interpretation of patient-specific feature importance. The team, which includes two of the authors of the original SHAP paper, developed a machine learning model to predict hypoxaemia during surgery, enabling the surgical team to take action if needed. In order for such a prediction to lead to actionable insight, it is fundamental for the clinical staff to understand exactly what factors are responsible for the arisal of hypoxaemia, so that they can not only trust the output of the model, but also know what can be done to curb the risk.</p><p><strong>Why it matters </strong>This paper presents a successful practical implementation of a theoretically-sound explainability algorithm applied to a medical decision-support tool.</p><p>The feature importance values provided by SHAP have been shown to align with medical expectations of what factors are known to be responsible for hypoxaemia during surgery, which shows the applicability of the tool to real-life problems. The additive contribution of the features also has the potential to highlight complex relationships between clinical parameters which may not be immediately clear during surgical operations. Lastly, the paper demonstrates the benefit of clear and appealing visualizations to explain a model’s predictions. Indeed, SHAP’s open source library allows captivating and interpretable graphs to be produced with just a few lines of code, going the extra mile to help doctors understand what goes on inside the complex machine learning algorithms.</p><p><strong>#4: Eye disease detection: an original approach on explainable neural networks<br></strong><em>Clinically applicable deep learning for diagnosis and referral in retinal disease<br></em>By: Jeffrey de Fauw, Joseph Ledsam, Olaf Ronneberger<br><a href="https://www.nature.com/articles/s41591-018-0107-6">https://www.nature.com/articles/s41591-018-0107-6</a></p><p>This article by Deepmind’s researchers is not exactly focusing on interpretability per se, as it presents a model to classify 3D eye scans. Optical coherence tomography (OCT) scans are widely used in diagnosing eye conditions and yet require considerable time to be analyzed from the medical staff, making them a perfect target for the application of automated techniques. The model described in the paper provides diagnoses in a swift and accurate manner, promising to be of great aid for early detection and effective treatment. In addition to the high performance of the model, the paper reports an enhanced interpretability compared to similar methods.</p><p><strong>Why it matters.</strong> Instead of using additional techniques to approximate the behaviour of their neural network, the <strong>researchers decided to split the model in two parts</strong>, one that segments the 3D scans and one that performs the classification itself. The output of the segmentation model provides insights into the kind of features used by the classification model, but it is also a clinically valuable tool that medical personnel can use to interpret scans. This goes beyond explainability, actually <strong>providing information that can be used independently from the model’s prediction</strong>. Moreover, and this is the second selling point of the article in our opinion, the authors put extra effort into integrating the models in the clinical pathway: the output of the segmentation model is displayed with a clinical result viewer for easy understanding, while the segmentation model is trained and tested on different scanning machines to ensure its generalizability across devices. Worth your time.</p><p><strong>#5: Optimizing neural networks for explainability via a decision tree </strong><em>Beyond Sparsity: Tree regularization of deep models for interpretability</em><strong><br></strong>By: Mike Wu, Michael Hughes, Sonali Parbhoo et al. <a href="https://arxiv.org/abs/1711.06178">https://arxiv.org/abs/1711.06178</a></p><p>Decision trees are often regarded as highly interpretable, given the fact that the user can take the sequence of splits in the tree as the rationale for why the model has returned a certain prediction on a specific outcome. However, these sort of explanations can also become unwieldy if (simplifying) the tree is too deep. The authors of the next paper introduce a technique to explain your high-performing black-box model via a simple decision tree that approximates the model itself.</p><p><strong>Why it matters.</strong> The method proposes to add a term to the loss function of the neural network, which encodes the complexity of the tree associated to the model — i.e. the complexity of the model’s explanation. Re-training the model with this additional term will also minimize the complexity of the associated tree, thus in other words <strong>the model will optimize its own explainability</strong>. Alas, the perk of enhanced explainability might cost some performance.</p><p>The articles reports experiments on medical tasks data with interesting findings. We applied this technique to the Intensive Care (IC) data we used for our own models (see <a href="https://medium.com/@Pacmedhealth/power-up-your-data-with-signal-processing-a-use-case-at-the-intensive-care-unit-47d99f90cc74">this</a> for more details on our IC work) and found out that indeed the resulting explanation was neater: see <a href="https://medium.com/@Pacmedhealth/ai-for-health-care-tackling-the-issue-of-interpretability-868be42aaf50">our blog post</a> for more explanation on this technique and our results.</p><p><strong>Conclusion</strong></p><p>Explainable AI is quickly rising to prominence as a field, yet it is still a young research area. Some important techniques are still being tested and discussed — see for example the <a href="https://openreview.net/forum?id=SJOYTK1vM">debate on local explanation methods</a> . For our formation as AI practitioners it is crucial to keep in touch with this area of research. However, that is not enough: we should also engage in a meaningful conversation with the general public. Clients, users, regulators, citizens; all these actors will be affected by AI one way or the other — they should have a say when it comes to deciding what kind of AI we want.</p><p><strong>References</strong></p><p>[1] James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi- Asi et al. Interpretable machine learning: definitions, methods, and applications. arXiv:1901.04592v1 [stat.ML]. 2019 Jan 14.</p><p>[2] Scott Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Electronic Proceedings of the 31st Neural Information Processing Systems Conference (NIPS 2017).</p><p>[3] Scott Lundberg, Bala Nair, Monica S. Vavilala et al. Explainable machine learning predictions for the prevention of hypoxaemia during surgery. Nature Biomedical Engineering, 2 (10), 749–760. 2018.</p><p>[4] Jeffrey de Fauw, Joseph Ledsam, Olaf Ronneberger. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature Medicine 24, 1342–1350. 2018 Aug.</p><p>[5] Mike Wu, Michael Hughes, Sonali Parbhoo et al. Beyond Sparsity: Tree regularization of deep models for interpretability. arXiv:1711.06178 [stat.ML]. 2017 Nov.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7970815465e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Healthy code, healthy patients: coding best practices in medical Data Science (Part 2)]]></title>
            <link>https://pacmedhealth.medium.com/healthy-code-healthy-patients-coding-best-practices-in-medical-data-science-part-2-925a21c50b0a?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/925a21c50b0a</guid>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[medicine]]></category>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Wed, 06 Mar 2019 14:07:58 GMT</pubDate>
            <atom:updated>2019-03-06T14:07:58.587Z</atom:updated>
            <content:encoded><![CDATA[<p>By Michele Tonutti, Data Scientist at Pacmed</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sreD11ALGN0WfrMDiq2lvA.png" /></figure><blockquote>“Will writing tidy code really help patients when they are rushed into the Intensive Care Unit?”</blockquote><blockquote>“Who cares if my code is 100 or 1 billion lines long, if the doctor will only see a probability and a graph?”</blockquote><blockquote>“If in order to test my code I need to write more code, do I just keep writing code forever to test the tests?”</blockquote><p>These (and many other) questions have probably, in one form or another, popped into the mind of all beginner coders who have started a project in medical data science. The answer to all of the above can be wrapped up in one of my favourite programming-related quotes:</p><blockquote>“<a href="https://thuijskens.github.io/2018/11/13/useful-code-is-production-code/"><strong>The only useful code is production-ready code</strong></a><strong>.</strong>”</blockquote><p>In a nutshell, <em>production-ready code</em> means code that is <strong>bug-free, scalable, well documented, easily readable, reusable, and reproducible</strong>. Following this principle will save you endless time, costs, and frustration, and it will ensure that the right results are obtained from the very beginning of a project.</p><p><a href="https://medium.com/@Pacmedhealth/healthy-code-healthy-patients-coding-best-practices-in-medical-data-science-part-1-d4c5ca2c42a">The first part of this article</a> covered version control, IDEs, repository structure, and virtual environments. In this second part I will give some insight on how to write <em>production-ready code </em>in medical data science, using some real-life examples from Pacmed’s own software development process. In particular, I will talk about <strong>code design, </strong>describing the concepts of<strong> abstraction </strong>and<strong> modularity; </strong>I will touch upon the importance of <strong>code style </strong>and <strong>documentation; </strong>and I will illustrate how and why we should always write extensive <strong>tests.</strong></p><p><em>(Once again my examples will use Python, but the principles apply to any other language!)</em></p><h3><strong>Abstract code yields concrete results</strong></h3><p>During the long Dutch winters, <em>staying </em><strong><em>DRY</em></strong> does not only refer to needing a raincoat while biking in the rain. In the programming world, it stands for <strong><em>D</em></strong><em>on’t</em><strong><em> R</em></strong><em>epeat</em><strong><em> Y</em></strong><em>ourself</em>, and it should be…well, repeated like a mantra. The concept of <strong>abstraction</strong><em> </em>is a cornerstone for scalable software: each distinct functional operation should be present in just one place in the source code, usually in the form of functions or classes. When similar tasks are carried out by different functions, they should be combined by <em>abstracting them out</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kbKvpdVyrp6Ejed-jl9qLg.png" /><figcaption>An example of code abstraction applied to Intensive Care data: to calculate the maximum value of each measurement for a patient, we can write just one function the takes two inputs, and call it repeatedly.</figcaption></figure><p>In the end, we want our code to look a bit like <strong>Lego</strong>: <em>beautiful, robust, </em>and<em> modular</em>. Indeed, abstraction makes the code look beautiful by enhancing readability: the functionality of tens, or even hundreds of lines of code can be reduced to just one function call in your application. Abstraction also increases the scalability of our development process, since each individual function only needs to be written and tested once, and can then be reused in any other script, or even other projects. For instance, at Pacmed we have recently reused big portions of the code written for predicting the incidence of Acute Kidney Injury at the VU Medical Center <a href="https://medium.com/@Pacmedhealth/power-up-your-data-with-signal-processing-a-use-case-at-the-intensive-care-unit-47d99f90cc74">Intensive Care Unit (ICU)</a>, in order to build a model that predicts patients’ length of stay in the ICU at the UMC Utrecht. This allowed us to reach a robust version of the data processing pipeline in just a few weeks time, rather than the several months it took the first time around.</p><h4>Sharing is caring</h4><p>To achieve efficient code sharing, it is important to have one or more central repositories which are well maintained, clean, and readable. At Pacmed we have our own general code repository, fittingly called <em>PacMagic,</em> which is effectively a custom Python package that contains all the functions needed for any step of a data science pipeline, from preprocessing to modeling, from data analysis to visualization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qdx-2VSDTmDTYDC5F95IRg.png" /><figcaption>Splitting data, scaling it, and plotting it can take as few as four lines of code using a shared code repository.</figcaption></figure><p>All of our Data Scientists contributes to it, and we make sure every piece of code in PacMagic is fully tested, documented, and properly structured. This means more time for fun modelling, and less time wasted re-writing the same pre-processing code a bizillion times. Once the data has been processed, we can train a model and analyze its results in less than 10 minutes. It is then possible to build up quickly from a working baseline model, and invest the saved time on researching and implementing more complex techniques, such as <a href="https://medium.com/@Pacmedhealth/slimme-algoritmes-schatten-medische-problemen-in-wat-is-spoed-en-wat-niet-461ffc19ed9c">Natural Language Processing algorithms for emergency care</a> or Bayesian Neural Networks to process <a href="https://medium.com/@Pacmedhealth/from-paper-to-patient-pacmeds-ai-newsletter-1-86a2ba0be54">Electronic Health Records in the ICU</a>.</p><h3>Readability means reliability</h3><p>As already mentioned above, readability is a necessary condition for code to be shared and reused across projects and even within teams. It is pointless to spend time writing modular and abstract functions if the next person is not going to be able to understand how to use them. To write readable code, it is important to properly <strong>document </strong>it<strong>,</strong> <strong>comment </strong>it<strong>, </strong>and most of all use <strong>good syntax and style</strong>.</p><h4>Good documentation saves lives…</h4><p>In Python especially, where functions are ubiquitous,<strong> docstrings</strong><em> –</em>short for <em>documentation strings– </em>are the main and most efficient approach to documenting code. They are small pieces of text that explain <em>what </em>a function does (not <em>how!</em>), and should include a list, description, and data type of every parameter of the function. The Python community has come up with <a href="https://numpydoc.readthedocs.io/en/latest/format.html">a few</a> <a href="https://www.python.org/dev/peps/pep-0287/">handy</a> <a href="https://www.python.org/dev/peps/pep-0257/">convention</a>s for writing docstrings; it’s good to pick one of those and stick to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CVDhyO6k65dpiTI7TgFarg.png" /><figcaption>An example of a function with a docstring written in <strong><em>numpy</em></strong> style, which is one of the styles natively supported and recognized by PyCharm. Just by looking at the docstring, one is able to fully understand what the function does and what kind of parameters must be used as an input.</figcaption></figure><p>In fact, using one of the supported docstrings conventions has many advantages. Apart from enforcing consistency and therefore efficiency, a good IDE will be able to create them automatically for you, given the function inputs and the parameter types. Other tools, such as <a href="http://www.sphinx-doc.org/en/master/">Sphinx</a>, will recognise docstrings in your code and will enable you to automatically generate full documentation for every function in your repository, which can then be stored or hosted on a private webpage for easy consultation.</p><h4><strong>…but good code should explain itself</strong></h4><p>At a certain point in their professional life, every programmer is taught that good code is well-commented code. While this is certainly true to some extent, the reality is that the <em>best</em> code is <strong>self-explanatory code</strong>. The name of variables, functions, classes, and even files should describe exactly what each of them does. In the end, one should be able to understand what a piece of code does without the need for explanation. This is true especially for Python, whose syntax has the advantage of being particularly human-readable.</p><p>For instance, in-line comments should only be used to explain small pieces of logic and workflow, or blocks of code that would be difficult to follow otherwise. If you want to print(&#39;Hello world&#39;) , there’s really no need to add # Printing hello world next to it — this would just add clutter and actually reduce the readability of the whole script.</p><p>To further ensure consistency and readability, following a set of style rules helps greatly. In Python, the most widely used style convention is <a href="https://www.python.org/dev/peps/pep-0008/">PEP 8</a>: it dictates a number of rules for code style, variable and function naming, and general code design.</p><h4>Static typing</h4><p>As enthusiastic Pythonistas, at Pacmed we have whole-heartedly welcomed the recent addition of<strong> <em>static type checking</em></strong>, meaning the possibility to specify the data type a variable is supposed to hold when it is initialized, yielding more understandable code, quicker computations, and allowing for automatic error checking in more advanced IDEs. While it may seem a bit cumbersome and unfamiliar at first, I recommend to take a look at <a href="https://medium.com/@ageitgey/learn-how-to-use-static-type-checking-in-python-3-6-in-10-minutes-12c86d72677b"><strong>this</strong></a> tutorial, which will clear things up and introduce you the magical world of not having to worry about TypeErrors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CElMlD9Q-8o99zYJ9iOedQ.png" /><figcaption>The beauty of <strong>static typing:</strong> the function signature now specifies that the input “df” must be a pandas DataFrame, and PyCharm will issue a warning if a Series is used instead.</figcaption></figure><h3><strong>No test left unturned</strong></h3><p>Great! Now we know how to write readable, modular, and abstract code. But we still have no idea whether that code is going to output what we intended it to. Knowing that the code you write and use works as expected<strong> </strong>is one of the most crucial parts of software development ­­– this can, and must be done through careful <strong>testing.</strong></p><p>Generally, there are different types of tests. <strong>Unit tests </strong>make sure that individual components (<em>units</em>) of a software work as it should, <em>independently from other code</em>. In most cases, these units are generally single functions. <strong>Integration tests, </strong>on the other hand, ensure that these units work as expected when put together, for example in a data processing pipeline. In data science projects this is often very important, because often one might only see the result of a long sequence of processing steps, without seeing the intermediate outputs. Python’s native unittest <a href="https://docs.python.org/3/library/unittest.html">module</a> offers everything you need to implement your own unit and integration tests.</p><p>You may have noticed that what we are talking about here is very different from ‘<em>testing</em>’ in the classical data science meaning, which usually refers to obtaining predictions from a model for a set of patients, checking the performance, analyzing the outputs, etc. Software testing refers to making sure that a piece of code, or a whole pipeline, does exactly what it is meant to be doing; even for the best programmers, this is not always guaranteed to happen. Effectively, this means writing extra code to test previously-written code. This may seem extremely annoying, and possibly a waste of time — so why should we do it? Here are some of the countless benefits:</p><p>1) <strong>Agile software development</strong>: code can be changed easily and at any time without breaking old code. When new code is implemented in the codebase, all tests can be run automatically to make sure that the rest of the code has not been affected by the changes. Most version control platforms, such as Gitlab, offer the possibility to implement <a href="https://docs.gitlab.com/ee/ci/">automatic pipelines</a> to run all tests in a repository every time changes are made.</p><p>2) <strong>Better code quality and software design:</strong> You are always sure the code you are using works as expected. You can safely re-use code written by other people without having to worry about its performance.</p><p>3) <strong>Learning</strong>: Writing tests also makes you think about why and how your code should work, effectively teaching you to become a better coder in the process.</p><p>4) <strong>Find bugs early:</strong> Many issues can be solved before the code is merged or reviewed by others.</p><p>5) <strong>Reduce time waste and costs:</strong> All of the above ultimately means less time wasted in debugging, which means less frustration for you and fewer time and cost investments for your employer.</p><p>6) <strong>Make your data engineers happy</strong>: an often-overlooked benefit, but actually the most important one! Whoever is going to implement your code in a production environment will be eternally grateful.</p><p>When developing medical software, there is also a third type of test, arguably the hardest to implement:<strong> end-to-end tests. </strong>They are meant to check whether the output of your software makes <em>logical sense, </em>given a raw input; for this reason, they require a high degree of domain knowledge. For instance, if a model predicts the probability of a patient having diabetes, a senior patient with high glucose values and a high BMI will be expected to yield a high probability. At Pacmed, <strong>data scientists work side-to-side with doctors</strong> from the very initial phases of every project, in order to design and perform sensible end-to-end tests and obtain meaningful output from the resulting code.</p><h4>KISS goodbye to complexity</h4><p>Ultimately, test everything and test often! In fact, some say it’s even better to write tests <em>before</em> writing the code itself: this forces you to think about what you want your code to do, what you <em>don’t </em>want it to do, and how you want to achieve that.</p><p>However, it can be hard to write tests that cover every single corner case; often functions can become so complex that it becomes practically impossible to foresee every single bug. To avoid this, we like to work by the principle that <strong>“if your code is hard to test, it’s hard to use”</strong>. This can be summarised by the <strong>KISS </strong>principle: “<strong>K</strong>eep <strong>I</strong>t <strong>S</strong>imple, <strong>S</strong>tupid!”. Simple, modular (see above), and functional code will be better performing, more maintainable, and also more readable.</p><h3><strong>Conclusion</strong></h3><p>Data science is, by all means, a lot of fun; creativity and curiosity play a huge role in building successful models and getting great results. When building medical software, structured and organized coding practices are paramount in order to obtain results that will make a real impact on the lives of patients.</p><p>Remember: as we talked about in the <a href="https://medium.com/@Pacmedhealth/healthy-code-healthy-patients-coding-best-practices-in-medical-data-science-part-1-d4c5ca2c42a">first part</a> of the article, good writing tools and developing environments will help you follow these guidelines, and allow you to easily produce beautiful, useful, and well-performing code in an efficient way.</p><p>Happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=925a21c50b0a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interning at Pacmed: goodbye blog by Aleide & Pim]]></title>
            <link>https://pacmedhealth.medium.com/interning-at-pacmed-goodbye-blog-by-aleide-pim-8578108c2768?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/8578108c2768</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Tue, 05 Mar 2019 10:48:05 GMT</pubDate>
            <atom:updated>2019-03-05T10:48:05.624Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By Aleide Hoeijmakers and Pim Hoeven</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RQ1HK3HXT6xgCGnlNtekdA.png" /></figure><p><em>After six months of working hard and having a lot of fun, our internship at Pacmed has unfortunately come to an end. In this blog we will share our experiences as interns at Pacmed; for students who are thinking about doing their internships at a company (instead of a university) but also for anyone else who is interested! We will first shortly describe our projects followed by some more general aspects of interning at Pacmed, followed by some practical tips!</em></p><p><strong>Why Pacmed</strong><br>We chose to do our internship at Pacmed for similar reasons: to learn about applying machine learning algorithms on real-world problems and, using these algorithms, contributing to improving healthcare.</p><p><strong>Risk prediction at the ICU</strong><br>Aleide did her Master’s thesis internship for her studies in Artificial Intelligence at the VU University. She joined the Intensive Care Unit (ICU) team at Pacmed, who work on risk prediction models for the ICU at the VUmc. The aim of her project was to predict serum creatinine of patients on the ICU in the next 24 hours to provide a better indication of the kidney function. The prediction can be used in the dosing strategies of antibiotics and hopefully limit the progression of further kidney damage or even mortality.<em> </em>The results are very promising, and will hopefully soon guide doctors in taking preventive measures for impending acute kidney damage.</p><p><strong>Predicting cardiovascular complications in diabetic patients</strong><br>Pim’s thesis was written for his Master’s in Econometrics at the UvA. He compared a classical statistical approach and a machine learning approach for predicting cardiovascular complications in diabetic patients. The research had both a clinical and a technical goal. The clinical goal was to help General Practitioners to identify patients that are at risk of cardiovascular complications and adjust their treatment accordingly. The more technical goal was to benchmark different kinds of models for risk prediction and implementing these in a way the Pacmed team can easily use for other risk prediction problems. Pim’s thesis results were well received by his supervisor!</p><p><strong>Working closely together with medical professionals</strong><br>In comparison to doing an internship solely at an academic institute, doing your internship at a company can have some advantages. Pacmed is a company that is connected to many of the big (academic) research institutes, but it’s also a company that develops products for the market. This is a special combination because it ensures that all research and projects that are done are relevant in practice and are a proof of concept that can, if the results are promising, serve as a foundation for actual products that help to improve healthcare. In order to develop such products, all researches are performed in close collaboration with medical doctors to continuously check the clinical relevance. For our thesis we also collaborated with medical professionals which was new to us and a very good experience.</p><p>Developing and implementing machine learning models for healthcare is an innovative, emerging method that needs to be developed and implemented very carefully and can be very complex. It can only be done by close collaboration between data scientists and medical professionals, and everyone at Pacmed, including the interns, needs to keep learning from each other continuously. The composition of the team, with ten passionate data-scientists and five young doctors, facilitates learning about both the machine learning and the medical side of the research. We also learnt a lot from the valuable workshops from our colleagues. These workshops, for example about unit testing or explainable AI, often have a theoretical and practical part. Therefore we could immediately apply the taught techniques immediately to our own work.</p><p><strong>Work hard, play Ping Pong</strong><br>Beside the fact that you learn a lot of new things about machine learning, at Pacmed you also get the opportunity to improve our social and professional skills, for example in explaining our findings. Every Friday there is the opportunity to present your work to the whole team during ‘demo’. This is a great moment to start a discussion or get feedback on your work. Afterwards, the Friday drinks start, in which we often play many rounds of ping pong together. In addition, there are many more nice activities to escape the thesis work: like joint boot camps, trampoline workouts and company activities.</p><p>Next to the many learning opportunities and the plus side of working in a team, there are a few things to keep in mind when considering an internship at a company. Working with real world data can be challenging and often requires extensive data preprocessing. The needs of the company you intern at and the wishes of your academic supervisor could be different so you might have to shape the project in a way that that works for both parties. At Pacmed the team was very willing to help us out with this and therefore it was no problem, but it’s good to keep in mind when you look for an internship.</p><p>We had an awesome internship at Pacmed and want to thank the team for their support and the fun times!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mmjjvyo_30G3aijw4J-JLg.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8578108c2768" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Healthy code, healthy patients: coding best practices in medical Data Science (Part 1)]]></title>
            <link>https://pacmedhealth.medium.com/healthy-code-healthy-patients-coding-best-practices-in-medical-data-science-part-1-d4c5ca2c42a?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/d4c5ca2c42a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[coding]]></category>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Tue, 19 Feb 2019 08:48:14 GMT</pubDate>
            <atom:updated>2019-03-06T16:30:03.386Z</atom:updated>
            <content:encoded><![CDATA[<p>By Michele Tonutti, Data Scientist at Pacmed</p><blockquote>How would you feel knowing that the quality of every single line of your code will directly impact the lives of thousands of people?</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j13mmTKziCIcSuMc6bYaHg.png" /></figure><p>Anyone who has ever coded even a simple script has likely experienced the pure excitement of seeing their program run for the first time without errors. For Data Scientists in particular, the satisfaction of successfully building and training a machine learning model is probably unrivalled.</p><p>This is particularly true in medical data science: the thrill of data-driven problem solving is exponentially amplified by the awareness that the predictions of the model might actually help doctors make more informed and personalized decisions. However, it is no secret that <strong>applying data science in medicine can be intimidating at first</strong>. When the lives of patients are on the line, any program built to be used in a clinical setting –whether it is a prediction model or a simpler analytical tool– must be of the highest possible quality. This is a big challenge, especially in diverse teams made up of developers, doctors, and data scientists, all with different coding skills.</p><p>At <strong>Pacmed</strong>, the job of the Data Science team is to develop, test, and prototype the machine learning models which drive our clinical decision support tools. The aim of our end products is to be implemented and used in practice in clinics and hospitals all around the Netherlands, with the concrete goal of <strong>helping doctors provide the best possible treatment to their patients</strong>. For our software to be as accurate, safe, and robust as possible, the code behind it must be absolutely flawless. In addition, our software development and quality assurance processes must also adhere to international standards and medical devices regulations, in order to receive the necessary certifications that allow the software to be used in a clinical setting.</p><p>In this article I will share some insight on what it takes to write high quality code for medical applications, specifically keeping in mind the challenges of working in a heterogenous team. Using the (fun!) day-to-day life of a Data Scientist at Pacmed as a case study, I will outline the main steps we take to <strong>ensure that the code driving our products is of the highest quality</strong>, and that the software development process, from inception to end user studies, is smooth and foolproof. <br>In this first part we will cover <strong>repository structure, IDEs, version control, </strong>and <strong>virtual environments. </strong>The second part will focus on <strong>how to write good code</strong>.</p><p><em>Note: Python is our language of choice, but the concepts will apply to any programming language you may prefer.</em></p><h3><strong>Step #0: Structuring repositories</strong></h3><p><strong>Messy code and folders are the number one killer of efficiency and team productivity</strong>. The first thing to do before starting any project is therefore to make sure the repository ­–meaning the folders where the code will be stored– has a sensible structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J_Ie5oTciwjJyzOFsTC8cw.png" /><figcaption>A custom repository structure based on <em>Cookiecutter for Data Science.</em></figcaption></figure><p>At Pacmed, we use the <a href="https://drivendata.github.io/cookiecutter-data-science/"><em>Cookiecutter for Data Science</em> </a>format, which is standardized and logical, but also very flexible. Flexibility is quite important, as we do not want to be completely constrained by boundaries that do not fit our goals. On the other hand, we want to make sure that all projects follow the same basic organizational approach; this makes it easier for people that do not work regularly on a project to find what they are looking for, or to go back to old code and know immediately what is where. For instance, the code to initialize, train, and test the prediction models will be in the <em>models </em>folder; the code to create features from raw data will be in <em>features;</em> the functions to plot the data for analysis will be in <em>visualizations; </em>and so on.</p><p><strong>A clean repository policy is even better than a clean desk policy</strong>: both will make working much more pleasant, but an organized repository will magically make your code much better in the long run!</p><h3><strong>A good IDE is a good idea</strong></h3><p>Another simple trick towards maximizing efficiency and eradicating frustration is choosing the right Integrated development environment (IDE). Basically, an IDE is the program or application you use to write code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TC5VSCQR800-w1qj.png" /><figcaption>PyCharm’s user interface, with the example of an auto-import suggestion. (From JetBrains’ website)</figcaption></figure><p>A<strong> good IDE is supposed to be an all-around handyman, serving as a spell-checker, proof-reader, copywriter, prompter, and look-up guide</strong>. It will provide you with automatic formatting, auto-complete suggestions, documentation, and an infinity of shortcuts to enhance productivity and minimize the possibility for mistakes. Our personal picks are <a href="https://www.jetbrains.com/pycharm/">PyCharm</a> and <a href="https://atom.io/">Atom</a>. For exploratory modelling and prototyping we also extensively use <a href="https://jupyter.org/">Jupyter</a> notebooks, which are fantastic for interactive and collaborative coding.</p><p>In general, it’s probably best to have access to a combination of the above-mentioned tools: for instance, using Jupyter to code an explorative data pipeline, while having PyCharm or Atom open on the side to be able to efficiently browse through the source code of your repository.</p><h3><strong>Version control: Git ’em all</strong></h3><p>If working in a team can be frustrating for most tasks, coding together can lead to unseen levels of frustration and friction within the group. To make sure that collaborative coding becomes an asset, rather than a burden, <strong>version control </strong>is the solution. Version control tools enable the possibility to <strong>track the changes to the code and the files over time</strong>, so that they can be reverted to specific versions at any point in the future, and that changes do not get overwritten by accident. The same project can have parallel <em>branches</em>, where different people can work on the same code simultaneously.</p><p>There are many different services available for this purpose, most of which rely on the open-source tool <a href="https://git-scm.com/">Git</a>. Since we are particularly patriotic at Pacmed, we use <a href="https://gitlab.com/">Gitlab</a>, born and bred in the Netherlands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FLwITZcSFyiD1jF-0CweXA.png" /><figcaption>An example of a <strong>Merge Request</strong> in Gitlab. New code to make interpretable models was added to the main branch of the repository, after begin thoroughly discussed and reviewed.</figcaption></figure><p>Regardless of the tool, a proper usage of version control allows the implementation of a number of cornerstones of software development:</p><ol><li>The <strong>four-eyes principle</strong>: code can only be merged into the main branch of your repository if it’s reviewed approved by (at least) another person. As an extra perk, having fellow coders review our work also helps improve one’s own coding skills.</li><li><strong>Versioning</strong>:<em> </em>Each time the master (main) branch of the codebase is changed, it gets a version number. At any point in the future, we will know which version of the code each project has been using when released to production. This also works on the small scale: through the use of ‘checkpoints’ called <em>commits,</em> fateful mistakes can be reverted relatively painlessly.</li><li><strong>Continuous integration and continuous delivery </strong>(CI/CD): a set of principles to automate the way our tools are built, tested, updated and deployed to production environments. Whenever we push code to Git, a series of automated tests make sure that our code does not fail and does not break existing functionalities and products.</li></ol><p>…and many more benefits. No more getting angry at your teammate for changing a very crucial piece of code! No more headaches trying to figure out how to fix a small mistake without having to lose a whole day of work! Sure, if you are not familiar with version control tools, they can seem scary and overly-complicated; but they will skyrocket your productivity, and you might discover that coding in a team can even be fun (…most of the time).</p><h3><strong>Protect the (virtual) environment</strong></h3><p>If you are going to collaborate and work on multiple projects at once, you might find that software requirements for each of these can differ widely. <strong>Virtual environments are a great way to manage Python packages independently for each project</strong>, by creating isolated directory trees containing an installation of a specific version of Python. Different versions of the same package (or even of Python itself) can be installed for different projects, and one only needs to install the strictly necessary packages for each. Environments can, and should, be shared amongst team members, so that one’s results can be reproduced easily. Another nice thing about environments is that they can be contained in <em>versionable</em> files, creating a wonderful synergy with Git. Each version of a project is accompanied by its own environment file, with the exact version of the packages used at that point in time. No more incompatibility and installation problems!</p><p>Our preferred choice to create and manage virtual environments is <a href="https://conda.io/docs/"><strong>Conda</strong></a>, since it’s very flexible, supports other languages (such as R), and is particularly compatible with Jupyter. Other options are <a href="https://pipenv.readthedocs.io/en/latest/">pipenv</a> or <a href="https://virtualenv.pypa.io/en/latest/">virtualenv</a>, both of which are well known and established in the Python community.</p><h3><strong>Ready to code!</strong></h3><p>Up to this point we have covered the necessary bases to create the setup for efficient and productive collaborative coding, and start developing solid programs for medical applications. In the second part of the article we will talk about the actual process of <strong>code-writing. </strong>You can find it here<strong>:</strong></p><p><a href="https://medium.com/@Pacmedhealth/healthy-code-healthy-patients-coding-best-practices-in-medical-data-science-part-2-925a21c50b0a">https://medium.com/@Pacmedhealth/healthy-code-healthy-patients-coding-best-practices-in-medical-data-science-part-2-925a21c50b0a</a></p><p>In particular, we’ll touch upon the best way to structure your functions in order to enhance maintainability; we’ll explore how to use the correct syntax to ensure maximum readability; and we’ll look at how to write good documentation and proper unit tests, amongst other things.</p><p>Happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d4c5ca2c42a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Persoonlijkere behandeling van prostaatkanker door machine learning]]></title>
            <link>https://pacmedhealth.medium.com/persoonlijkere-behandeling-van-prostaatkanker-door-machine-learning-d39142928a1e?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/d39142928a1e</guid>
            <category><![CDATA[personalized-medicine]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[prostate-cancer]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Thu, 20 Dec 2018 16:37:31 GMT</pubDate>
            <atom:updated>2018-12-20T16:37:31.237Z</atom:updated>
            <content:encoded><![CDATA[<p>door Daan de Bruin (Pacmed)</p><p>Als ik vertel over ons werk op het gebied van prostaatkanker, gebeurt het me opvallend vaak dat mensen persoonlijk geraakt zijn door dit onderwerp. Laatst vertelde iemand me:<em> </em>“Mijn vader is overleden aan prostaatkanker. Toen de arts de gemiddelde verwachte overlevingskansen aan ons liet zien, vroeg ik me af in hoeverre die toepasbaar waren op de situatie waar mijn vader zich in bevond.” Kan dit niet slimmer?</p><p>Lees het hele artikel hier:</p><p><a href="https://www.smarthealth.nl/2018/12/19/blog-pacmed-persoonlijkere-behandeling-prostaatkanker-machine-learning/">Blog: persoonlijkere behandeling van prostaatkanker door machine learning</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d39142928a1e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From paper to patient: Pacmed’s AI newsletter #1]]></title>
            <link>https://pacmedhealth.medium.com/from-paper-to-patient-pacmeds-ai-newsletter-1-86a2ba0be54?source=rss-bc7f185fa831------2</link>
            <guid isPermaLink="false">https://medium.com/p/86a2ba0be54</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Pacmed]]></dc:creator>
            <pubDate>Wed, 19 Dec 2018 14:58:03 GMT</pubDate>
            <atom:updated>2019-05-14T14:23:55.510Z</atom:updated>
            <content:encoded><![CDATA[<h3>From paper to patient, Pacmed’s AI newsletter #1: handling electronic health records</h3><p><em>By Giovanni Cinà and Michele Tonutti (Data Scientists at </em><a href="https://pacmed.ai/en/"><em>Pacmed</em></a><em>)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eWYm3Dm7s0t8lLWYRW4pow.jpeg" /><figcaption>Helping doctors and data scientists get a clear view on the latest developments in medical AI</figcaption></figure><p>Machine learning promises to be a technology that could help medical professionals and patients to achieve better health outcomes. This value does not come easily, and the implementation of machine learning in healthcare is a precarious issue. There should be constant consideration of the boundary conditions under which it must be implemented in order to be of sustainable and scalable value for everyone. Machine learning is not good and safe by definition, and <strong>responsible deployment is the key to success for the patient</strong>. There are risks of empty correlations, biases and limitations in the data, (un)interpretability of algorithms, and unilateral interests. One way of mitigating these risks is a deep understanding of the clinical and personal context, by working closely together with all relevant stakeholders in the medical field.</p><p>What is also important is to have an active and curious mind, which enables you to be on the cutting edge of available knowledge at all time. Therefore, we find it important to objectively determine what are the promising developments to keep an eye on. Amongst others, we do this by internally sharing and discussing papers that we find interesting. We would like this knowledge to become publicly available, in order to help each other in learning about the right and responsible way of implementing machine learning models.</p><p>This blog post is the first edition of a collection of articles that we found particularly interesting and useful. The theme of this edition is the handling of (and the reusing of) <strong>electronic health records</strong> <strong>(EHR)</strong> from a technical perspective. Being often a bottleneck in the deployment of machine learning, this is a hot topic.</p><p>EHRs are <strong>digital records of a patient’s health information</strong> taken over time, generated by one or more visit to hospitals, clinics, GPs, or other care institutions. They include results of lab tests and measurements, such as heart rate, blood pressure, etc., as well as medications, diagnosis, and the notes written by doctors and nurses. The volume of EHR data recorded for each patient is vast; this amount of information presents valuable opportunities to develop data-driven decision support tools, but also present great challenges.</p><p>EHR data is often <em>heterogeneous</em>, meaning that most measured parameters are widely different from each other; sometimes <em>incoherent</em>, since even a single parameter may be measured differently over time or by different actors; <em>sparse, </em>with many missing values; <em>noisy</em>, i.e. many measurements are wrongly measured or not validated; <em>biased</em>, for instance due the fact that more data might be present for patients with severe conditions, or influenced by the beliefs and decisions taken by the clinical staff. Finally, EHRs can change shape dramatically across institutions, and they are highly sensitive due to privacy issues.</p><p>These limitations have hindered the application of AI techniques in health-care; yet, there have been noteworthy developments. At Pacmed we are keen to implement cutting-edge solutions that allows us to build models that are not only precise and useful, but also scalable, generalizable, reliable and interpretable.</p><h4><strong># 1: A general overview</strong></h4><p><em>Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review </em><br>By: Cao Xiao, Edward Choi, Jimeng Sun<br><a href="https://doi.org/10.1093/jamia/ocy068">https://doi.org/10.1093/jamia/ocy068</a></p><p>With the development of advanced Machine Learning techniques, the promise to unlock the value contained in EHR data seems more concrete than ever. However, as usual with subfields of AI that gather more and more attention, the amount of papers on the topic is growing at a fast pace and it becomes hard to separate the wheat from the chaff. In order <strong>to navigate this literature, our first suggestion is an overview paper</strong> that highlights the main trends. It reviews 98 articles describing applications of deep learning on EHR data in the last 8 years, classifying them by task performed and architecture used.</p><p><strong>Why it matters:</strong> Besides providing a general overview and pointing to many good papers, we choose this article because it touches on the challenges presented by EHR data and discusses some solutions proposed by the research community. A good starting point!</p><h4><strong>#2: Producing synthetic data</strong></h4><p><em>Generating Multi-label Discrete Patient Records using Generative Adversarial Networks</em><strong><br></strong>By: Edward Choi, Siddharth Biswal, Bradley Malin, et al.<br><a href="https://arxiv.org/abs/1703.06490">https://arxiv.org/abs/1703.06490</a></p><p>As mentioned, one of the main issues when using EHR data is the concern over privacy, since the disclosure of medical information can be highly problematic. As a consequence, it is difficult to obtain and pool together large datasets, which in turn hampers the development of data-hungry Machine Learning algorithms.</p><p>A number of researchers is trying to tackle this problem with the generation of synthetic data. The idea is roughly the following: to <strong>train a model with real data to produce synthetic data that is “as good as real”</strong>. We can then share the synthetic data safely, since it is hard to obtain information about real patients from it, and we can also use synthetic data to augment our datasets. The techniques to generate synthetic data are still developing and we are far from safely using them in health-care, nevertheless we find this an interesting niche.</p><p><strong>Why it matters</strong>: The paper we highlight applies Generative Adversarial Networks to generate synthetic patient records. Although limited to the generation of EHR data with discrete variables, the article offers some promising results along with the introduction of a new technique called minibatch averaging. Intriguing if data privacy is your cup of tea.</p><h4><strong>#3: Deep learning for scalable models</strong></h4><p><em>Scalable and accurate deep learning with electronic health records</em><strong><br></strong>By: Alvin Rajkomar, Eyal Oren, Kai Chen, et al.<br><a href="https://www.nature.com/articles/s41746-018-0029-1">https://www.nature.com/articles/s41746-018-0029-1</a></p><p>The number of different parameters stored in EHRs is often in the hundreds. Choosing which ones to use to train machine learning models is often difficult and requires a certain degree of trial and error. The paper shows that <strong>deep learning models can bypass the need to choose predictive features in advance and, in addition, that they can be used to learn the most optimal representation of clinical data from the data itself</strong>, for instance using autoencoders. From a data science perspective, there are clear advantages in not having to select features manually and to automatically find the most optimal representation of input data: models can be built and computed much faster, and often with an increase in performance compared to using the whole set of available parameters. Extensive collaboration with medical professionals will still be needed, but it could mean a gain in efficiency.</p><p><strong>Why it matters: </strong>The contributions of this paper are substantial, as it reports improved performance of their models on a number of very different tasks in the Intensive Care Unit, namely prediction of mortality, readmission, length of stay and <a href="https://medium.com/@Pacmedhealth/power-up-your-data-with-signal-processing-a-use-case-at-the-intensive-care-unit-47d99f90cc74">discharge diagnosis</a>. Being able to use similar or even identical model architectures and data representations for different problems is extremely valuable to speed up development time, increase robustness, and quickly scale software across different hospitals. We must however not forget that medical expertise is often crucial to the development of a successful decision support tool. The features selected by the model might be heavily biased towards a certain clinical population; or they might be heavily predictive but measured very infrequently, thus possibly making them unsuitable for urgent predictions.</p><h4><strong>#4: Learning personalized data representations</strong></h4><p><em>Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record</em><strong><br></strong>By: Jinghe Zhang, Kamran Kowsari, James H. Harrison, et al.<br><a href="https://doi.org/10.1109/ACCESS.2018.2875677">https://doi.org/10.1109/ACCESS.2018.2875677</a></p><p>This paper, from the University of Virginia, also employs representation learning — the set of techniques to learn the optimal way to represent the data– together with a method called <em>attention</em> to both improve performance and increase interpretability of deep neural networks for clinical decision making. Their framework, named <em>Patient2Vec </em>as a reference to Google’s Natural Language Processing algorithm <em>Word2Vec, </em>aims at <strong>capturing the complex relationships between clinical events and at learning a personalized representation of each patient’s history</strong>. Treating medical events (prescriptions, interventions, lab results, etc.) like words in a sentence –hence the reference to Natural Language Processing– data is represented in sequences and subsequences in a way that highlights the clinical and temporal relationship between visits and allows to consider irregularly-sampled sequential data in a meaningful manner. The addition of the attention technique is of particular interest. As its name suggests, it is inspired by human (visual) attention: when performing a task, we tend to focus on one or very few specific things at once, those that are most important for the completion of the task. Similarly, a neural network with an attention layer will focus on the sections of a sequence that are most relevant to predict the outcome. The weights of these attention layers can be visualized to understand what these sections are and how they influence the predicted outcome.</p><p><strong>Why it matters: </strong>The paper highlights how data collected in practice about the patient’s phenotype (physical and medical characteristics) and clinical history can lead to models that improve both the outcome and patient’s experience, as well as lead to lower costs and improve overall efficiency. In addition, the successful application of cutting-edge NLP and Computer Vision techniques to numerical sequential data goes to show how much can be gained from multidisciplinarity within the field of machine learning.</p><h4><strong>#5: Learning from suboptimal clinical decisions</strong></h4><p><em>The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care</em><strong><br></strong>By: Matthieu Komorowski, Leo A. Celi, Omar Badawi, et al.<br><a href="https://www.nature.com/articles/s41591-018-0213-5">https://www.nature.com/articles/s41591-018-0213-5</a></p><p>One of the most interesting approaches to solve the problem of sparse data is reinforcement learning (RL). <strong>RL applications in the medical domain are particularly appealing because models can learn from suboptimal observations — </strong>meaning treatment choices that turned out to be not the best possible solution. The authors of this paper used data from treatment decisions obtained in practice in the Intensive Care Unit to improve treatment decision for sepsis, the leading cause of death in hospitals worldwide. The clinical events in each patient’s history were modelled using a Markov Decision Process, which, at a statistical level, adds an element of stochasticity (i.e. random chance) to the outcomes of treatment choices, visits, and measurements. While this is not a novel approach <em>per se,</em> the results are extremely promising: mortality was lowest when the treatment chosen by the clinician matched the one chosen by the model. In addition, their solution allowed for intuitive interpretation of the results by adding an extra layer consisting in a random forest classifier, thereby providing the possibility to explain the decision taken by the model by looking at the importance of each feature.</p><p><strong>Why it matters: </strong>Aside from the technical achievements, this paper touches upon a crucial point in machine learning development in healthcare: how to evaluate a model’s performance. It is pivotal to measure the effect on both patients and doctors: there is no need for a complex algorithm with extremely good performance if it provides no benefit to a patient’s life. Moreover, the addition of an interpretable classifier is of particular relevance in the medical sector, since it is crucial that the clinical staff fully understands an algorithm’s output. Lack of <a href="https://medium.com/@Pacmedhealth/ai-for-health-care-tackling-the-issue-of-interpretability-868be42aaf50">interpretability</a> is one of the major downsides of deep learning, and alternative ways to visualize and explain results and feature importance are always valuable.</p><p>What do you think of this selection? We are keen on hearing your take on this subject: do share your comments here or reach out to us on Twitter. Last but not least, let us know your ideas for the next topic of our newsletter!</p><h4>References</h4><p>[1] Xiao C, Choi E, Sun J. Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review. Journal of the American Medical Informatics Association. 2018 Jun 8.</p><p>[2] Choi E, Biswal S, Malin B, Duke J, Stewart WF, Sun J. Generating multi-label discrete patient records using generative adversarial networks. arXiv preprint arXiv:1703.06490. 2017 Mar 19.</p><p>[3] Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, Liu PJ, Liu X, Marcus J, Sun M, Sundberg P. Scalable and accurate deep learning with electronic health records. npj Digital Medicine. 2018 May 8;1(1):18.</p><p>[4] Zhang J, Kowsari K, Harrison JH, Lobo JM, Barnes LE. Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record. IEEE Access. 2018;6:65333–46.</p><p>[5] Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nature Medicine. 2018 Nov;24(11):1716.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86a2ba0be54" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>