<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Julien Siebert on Medium]]></title>
        <description><![CDATA[Stories by Julien Siebert on Medium]]></description>
        <link>https://medium.com/@julien.siebert?source=rss-474bfe8f9bd7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*nugMjwz7qL__1Cvbg_Lhqw.jpeg</url>
            <title>Stories by Julien Siebert on Medium</title>
            <link>https://medium.com/@julien.siebert?source=rss-474bfe8f9bd7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 07:04:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@julien.siebert/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Generating data quality deficits in Python.]]></title>
            <link>https://medium.com/@julien.siebert/generating-data-quality-deficits-in-python-b2552a883524?source=rss-474bfe8f9bd7------2</link>
            <guid isPermaLink="false">https://medium.com/p/b2552a883524</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-quality]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Julien Siebert]]></dc:creator>
            <pubDate>Mon, 03 Jul 2023 09:36:11 GMT</pubDate>
            <atom:updated>2023-07-03T09:36:11.230Z</atom:updated>
            <content:encoded><![CDATA[<h4>Introducing <a href="https://github.com/Fraunhofer-IESE/badgers">badgers</a> (bad data generators).</h4><figure><img alt="Engraving by Walter Heubach (1865–1923). Public Domain. Source https://en.wikipedia.org/wiki/European_badger#/media/File:Heubach_badger.jpg" src="https://cdn-images-1.medium.com/max/956/1*50qbjAmTbFELJDtbg4CV_g.jpeg" /></figure><p>Applications and systems based on artificial intelligence (AI), machine learning (ML), data mining or statistics (hereafter referred to as data-driven software components) are pieces of software where the decision function is not programmed in a classical way, but is based on one or more models that can be designed either automatically (e.g. through learning or mining) or based on domain expertise hypotheses (e.g. business rules or statistical tests).<br>Assessing the quality of such software components is not trivial, as it depends on several factors, such as the quality and quantity of the data, the type of model and how it is built, the application context, and domain expertise.</p><figure><img alt="Real data Internet meme (Source unknown)" src="https://cdn-images-1.medium.com/max/501/1*PlpSx1NmeA1bqDcHHRclYQ.jpeg" /></figure><p>Data quality deficits such as outliers, unbalanced data, missing values, etc. can have a variety of effects on the performance of a data driven model. Few algorithms are theoretically known to be sensitive to specific data quality issues (e.g., linear regression with OLS and outliers). Many can only be empirically tested against specific data quality issues. To make matters worse, data quality deficits are context and application dependent. They require some hypotheses to be made about how the data was generated. For example, data points with a z-score greater than 3 might be considered outliers in one context, assuming 1) that extreme values are outliers and 2) that the data follows a Gaussian distribution, whereas in another context this might not be the case.</p><p>Assessing the robustness of a data-driven software component to changes in data quality requires a systematic approach. It also requires the ability to generate specific data quality deficits in order to run tests.</p><figure><img alt="How to confuse machine learning. Internet meme (Source unknown)." src="https://cdn-images-1.medium.com/max/500/1*5iSfrz5AodovFeKeX6j6Sg.jpeg" /></figure><p>Currently, there are many Python libraries to detect and handle data quality deficits, such as <a href="https://pyod.readthedocs.io/en/latest/">pyod </a>for detecting outliers, <a href="https://imbalanced-learn.org/">imbalanced-learn</a> for dealing with imbalanced data, <a href="https://autoimpute.readthedocs.io/en/latest/">autoimpute </a>for imputing missing values., or <a href="http://docs.greatexpectations.io/">great-expectations</a> for validation. In addition, the field of deep-learning has provided us with libraries for augmenting training data (see <a href="https://neptune.ai/blog/data-augmentation-in-python">this nice overview by neptune AI</a>). However, there are very few, if any, libraries for generating data quality deficits that are context specific.</p><p><a href="https://github.com/Fraunhofer-IESE/badgers">Badgers</a> (<a href="https://fraunhofer-iese.github.io/badgers/">doc</a>) is an open source Python library to generate context specific data quality deficits. It provides specific objects called generators that follow a simple API: they provide a <em>generate(X,y)</em> function that produce data quality deficits (see the code snippets below). Generators can also be connected in pipelines à la Sklearn.</p><p>At the moment <a href="https://github.com/Fraunhofer-IESE/badgers">badgers</a> supports tabular data, time series data and text (with plans to add support for graphs and images in the near future). Currently there are generators for drift, unbalanced data, missing values, noise and outliers for tabular data; generators for noise and outliers for time series data; and generators for typos for text data.</p><p>Install badgers with pip:</p><pre>pip install badgers</pre><p>Import badgers as any other library and start using it:</p><pre>from sklearn.datasets import make_blobs<br>from badgers.generators.tabular_data.noise import GaussianNoiseGenerator<br><br>X, y = make_blobs()<br>trf = GaussianNoiseGenerator(noise_std=0.5)<br>Xt, yt = trf.generate(X, y)</pre><p>To plot the resulting data:</p><pre>import matplotlib.pyplot as plt<br>import numpy as np<br><br>def plot_noise(X, y, Xt):<br> &quot;&quot;&quot;<br> Some utility function to generate the plots<br> &quot;&quot;&quot;<br> fig, axes = plt.subplots(1, 2, sharex=True, sharey=True, figsize=(8,4))<br> for label in np.unique(y):<br>   ix = np.where(y == label)<br>   axes[0].scatter(X[ix,0],X[ix,1], c = f&#39;C{label}&#39;, label = f&#39;{label}&#39;)<br>   axes[1].scatter(Xt[ix,0],Xt[ix,1],c = f&#39;C{label}&#39;, label = f&#39;{label}&#39;)<br> axes[0].set_title(&#39;Original&#39;)<br> axes[1].set_title(&#39;Transformed&#39;)<br> axes[0].set_xlabel(&#39;dimension 0&#39;, fontsize=10)<br> axes[1].set_xlabel(&#39;dimension 0&#39;, fontsize=10)<br> axes[0].set_ylabel(&#39;dimension 1&#39;, fontsize=10)<br> axes[1].set_ylabel(&#39;dimension 1&#39;, fontsize=10)<br> axes[0].legend()<br> axes[1].legend()<br> return fig, axes</pre><pre>fig, axes = plot_noise(X, y, Xt)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*lyaT4H3mCJ01F2V9iszOvA.png" /></figure><p>More tutorials are available in the <a href="https://fraunhofer-iese.github.io/badgers/tutorials/Imbalance-Tabular-Data/">documentation</a>. Below are some images from the tutorials for generating outliers and imbalanced data.</p><figure><img alt="Generating outliers having a z-score larger than 3, using hypersphere sampling." src="https://cdn-images-1.medium.com/max/790/1*mOLbmyFCuGd3mBYoGNPvwg.png" /></figure><figure><img alt="Generating outliers by sampling from low density regions." src="https://cdn-images-1.medium.com/max/790/1*mek5XfA9N_oEt_76552B9Q.png" /></figure><figure><img alt="Generating imbalanced data." src="https://cdn-images-1.medium.com/max/794/1*zG11ur0UnUqg6SANXimexw.png" /></figure><p>With badgers we hope to provide a simple open source Python library that can be used to assess the robustness of data-driven software components against data quality deficits. Any idea, comments, commits are more than welcome <a href="https://github.com/Fraunhofer-IESE/badgers/issues">https://github.com/Fraunhofer-IESE/badgers/issues</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b2552a883524" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Data Science towards Causality sciences]]></title>
            <link>https://medium.com/@julien.siebert/from-data-science-towards-causality-sciences-a1fd17a5a2c6?source=rss-474bfe8f9bd7------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1fd17a5a2c6</guid>
            <category><![CDATA[retrospectives]]></category>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[causal-data-science]]></category>
            <dc:creator><![CDATA[Julien Siebert]]></dc:creator>
            <pubDate>Mon, 19 Jun 2023 19:47:27 GMT</pubDate>
            <atom:updated>2023-06-22T08:15:58.113Z</atom:updated>
            <content:encoded><![CDATA[<h3>From Data Science towards Causal Data Science</h3><h4>A short retrospective</h4><p>A little over a year ago, I embarked on a journey to learn causality related sciences (causal inference, causal discovery, causal ML, etc.). After reading a couple of books, watching some <a href="https://www.bradyneal.com/causal-inference-course">online lectures</a>, attending a <a href="https://www.causalscience.org/meeting/about/call-for-papers/">dedicated conference</a>, following and giving some talks in a <a href="https://gitlab.com/causal-inference/working-group/-/wikis/home">causal inference working group</a>, writing a <a href="https://www.iese.fraunhofer.de/blog/causal-inference/">blog post</a> and a <a href="https://arxiv.org/abs/2211.11482">scientific paper</a>, supervising a student on the topic of causal discovery, being a guest editor for <a href="https://www.sciencedirect.com/journal/information-and-software-technology/about/call-for-papers">a special issue on that topic</a>, trying out several libraries and <a href="https://benchpressdocs.readthedocs.io/en/latest/">a benchmarking application</a>, I feel it is time to reflect on my journey so far.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rlbrA27B4-tLT5Lf" /><figcaption>Photo by <a href="https://unsplash.com/@iriser?utm_source=medium&amp;utm_medium=referral">Irina Iriser</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>The first thing to know is that causal* methods were (and are) developed to deal with the case where it is not possible to conduct <em>randomized controlled trials</em>. In that case, the only information we (the analysts) have is <em>observational data</em>.</p><p>Simply put, randomized controlled trials compare two groups, one that receives the treatment (the treatment group) and one that does not receive the treatment (the control group). Ideally, the two groups should be identical in their characteristics, and the only difference between them should be the treatment. Only then can we be sure that what we observe is a direct result of the treatment and not the result of something else.</p><p>Imagine a <em>not-so-randomized</em> experiment in which we compare two groups: one group composed only of young people, and one group composed only of older people. Anything we observe is either due to the treatment (this would be the true causal effect of the treatment) or due to the age difference (this is called a confounding factor).</p><p>The problem with observational data (i.e., when no randomized controlled trial has been conducted) is that there may always be some confounding factor lurking around that will bias our interpretation of the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XoFTPKmdNk4VZzDEyA3dFA.png" /><figcaption>Eating cheese can be dangerous… Source: <a href="https://www.tylervigen.com/spurious-correlations">spurious correlations</a></figcaption></figure><p>The whole point of causal* methods is to get around the problems of observational data and to separate the wheat (causal effects) from the chaff (spurious correlations).</p><p>Determining the causal effect of a treatment on an outcome using observational data is (most often) called causal inference. Beware data scientists, causal inference is not something you can just throw at your data, you need two things: the data (of course), but also some model of the causal structure that generated those data. In my <em>not-so-randomized</em> example above, the causal structure would at least include my treatment and people’s ages. You can imagine the causal structure as a graph with nodes representing phenomena (treatments, variables) and links representing a direct causal effect between two nodes (no link means that there is no causal effect at all, this is a strong assumption).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*LCwQSgwH4yaznYDUUABI0g.png" /><figcaption>A very small causal graph where age is a confounder: it influences both the treatment and the outcome. In that case, age must be adjusted for (image by the author).</figcaption></figure><p>Knowing what type of causal structure has generated the data we observe helps in choosing how to estimate the causal effect from the data. In the <em>no-so-randomized</em> experiment example, in order to estimate the effect of the treatment on the outcome, one needs to handle the effect of age (statisticians will say to adjust for, or to control for age). This is called identification. In causal inference, some methods make explicit use of a causal graph to automatically identify which variables to adjust for¹. Other approaches (like propensity score matching, instrument variables, or regression discontinuity) do not explicitly rely on a causal graph to estimate the causal effect but are applicable for specific families of causal graph². In principle, all causal inference methods make assumptions about the causal structure behind the scene. Once the causal effect has been identified (that is, we know which variables to adjust for, and which one to leave out), the next step is to estimate the effect.</p><p>While I have already mentioned some methods (such as propensity score matching, instrumental variables, etc.), there is a vast family of estimation methods based on machine learning. They are usually referred to as causal ML and, as far as I know, do not perform identification by themselves. They assume that all variables other than treatment and outcome are covariates to be adjusted for. This is important to know because adjusting for too many variables can introduce unwanted bias³.</p><p>While data is typically collected by sensors and stored in databases, causal graphs require domain expertise.That is, a causal graph is most likely to be manually crafted by domain experts. The field of causal discovery (also called causal structure learning) has the ambitious goal of automatically generating a causal graph from existing data. Needless to say, causal discovery is a difficult task in itself, and its success depends not only on the data themselves, but also on the various assumptions made by causal discovery algorithms (such as the presence of unmeasured confounders, linear vs. nonlinear effects, or Gaussian vs. non-Gaussian noise).</p><p>To sum up: inferring causal effect from observational data require at least 3 steps⁴:</p><ol><li>The first step (modeling) is to make causal assumptions explicit. A causal graph is the tool of choice.</li><li>The second step (identification) is to determine whether a given causal effect can be estimated from the available data.</li><li>The third step (estimation) is to effectively estimate the identified causal effect based on the available data and the previously gained knowledge of which variables should be adjusted for.</li></ol><p><strong>Retrospective:</strong></p><p>When I first jumped into the causality field, the most confusing thing I found was all the different terms, the communities of researchers (Pearl vs. , and the corresponding plethora of methods that are available. Especially the fact that some estimation methods do not use a causal graph and do not explicitly refer to identification. Furthermore, the fact that some papers distinguish between the Pearl approach (based on causal graphs) vs. the potential outcome framework of Rubin, Imbens and colleagues is very misleading. In my view, both approaches were developed in different contexts, AI for Pearl et al. and econometrics for Rubin et al., and therefore assume different things about what is known about the underlying causal structure. The advantage of using a causal graph is the ability to reason about causality (that’s for the AI part). Of course, in some cases the causal graph is not accessible, so some other methods are needed (this is for the econometrics / any empirical research part).</p><p>I found the two following books: <a href="https://theeffectbook.net/">The Effect</a> (thanks for the videos without the music) and <a href="http://bayes.cs.ucla.edu/WHY/">The Book of Why</a> very helpful to start with. I also have <a href="https://mixtape.scunning.com/">Causal Inference: The Mixtape</a>. Unfortunately this requires some underlying knowledge about econometrics that I do not have (I studied engineering and computer science). It was therefore difficult for me to dive in. I haven’t had the time to full dive into <a href="https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/">What If</a>, but that’s on my todo list, together with <a href="https://mitp-content-server.mit.edu/books/content/sectbyfn?collid=books_pres_0&amp;id=11283&amp;fn=11283.pdf">Elements of Causal Inference</a>.</p><p>There are many interesting online resources. To name a few, I found the interactive tutorials provided by <a href="http://dagitty.net/learn/">Dagitty</a> very helpful to understand the concepts behind the do-calculus (the d-separation). The online lecture <a href="https://www.bradyneal.com/causal-inference-course">Introduction to Causal Inference</a> from Brady Neals is helpful to dive into the techniques (thanks for the short videos). Finally the <a href="https://www.pywhy.org/dowhy/v0.9.1/">dowhy</a> library is a place to go to try out causal inference in Python and it has very nice tutorials.</p><p><strong>Notes:</strong></p><p>[1] Automatically identifying how to estimate the causal effect from a given graph is done using the <em>do-calculus</em> developed by Judea Pearl and collegues. See <a href="http://bayes.cs.ucla.edu/WHY/">The Book of Why</a> or the interactive tutorial provided by <a href="http://dagitty.net/learn/">Dagitty</a>.</p><p>[2] The interested reader can have a look at <a href="https://theeffectbook.net/">The Effect</a> and <a href="https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/">What If</a> (no not the <a href="https://what-if.xkcd.com/">one</a> from XKCD author), two excellent books on the topic that show the types of causal graphs that are implicitly assumed in many causal inference methods.</p><p>[3] See the paper <a href="https://ftp.cs.ucla.edu/pub/stat_ser/r493.pdf">A Crash Course in Good and Bad Controls</a></p><p>[4] Pearl and Mackenzie introduced a 9 steps process in <a href="http://bayes.cs.ucla.edu/WHY/">The Book of Why</a> (chapter 1). Sharma and Kiciman proposed 4 steps in their <a href="https://arxiv.org/abs/2011.04216">DoWhy paper</a>. The fourth steps in Sharma and Kiciman is <a href="https://www.pywhy.org/dowhy/v0.9.1/user_guide/effect_inference/refute.html">Refutation</a>. While that is important, I skipped it for the sake of simplicity.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1fd17a5a2c6" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>