<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Emanuele Fabbiani on Medium]]></title>
        <description><![CDATA[Stories by Emanuele Fabbiani on Medium]]></description>
        <link>https://medium.com/@donlelef?source=rss-3439142d7553------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*s4if2noQZYdnkHF0bTPkzA.png</url>
            <title>Stories by Emanuele Fabbiani on Medium</title>
            <link>https://medium.com/@donlelef?source=rss-3439142d7553------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 22:47:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@donlelef/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Simplifying Forecasts with Teamwork: The Power of Ensemble Models]]></title>
            <link>https://medium.com/@donlelef/simplifying-forecasts-with-teamwork-the-power-of-ensemble-models-cc6f9210b3de?source=rss-3439142d7553------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc6f9210b3de</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[forecasting]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Emanuele Fabbiani]]></dc:creator>
            <pubDate>Thu, 29 Feb 2024 10:17:29 GMT</pubDate>
            <atom:updated>2024-02-29T10:17:29.419Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vnvN-NdvOxSqGZjO" /><figcaption>Generated by the author using GPT-4</figcaption></figure><p>Just like in investing, where spreading your bets can lead to rewards without much risk, in the world of machine learning, <strong>mixing different simple models can give us better predictions</strong>. This idea is what makes techniques like random forests and gradient boosting work so well.</p><p>Recent research shows that combining different types of models, like random forests, linear models, and gradient-boosting models, and averaging their predictions, gives us more accurate forecasts than any single model could on its own.</p><p>In this piece, we’ll dive into <strong>studies</strong> that show how effective mixing models can be. Then, we’ll look at <strong>two real-world cases</strong> where companies used this approach to get great results.</p><h3>Research Insights</h3><p>Back in 2016, a study by Nowotarski et al. shared a way to get better at predicting short-term electricity needs by <strong>mixing forecasts</strong> from “sister models” — models that are similar but not quite the same [1].</p><p>They tried 11 different mixing methods on public datasets and found that combining these forecasts gave much better accuracy than just using one model or the usual benchmarks. Methods like averaging, trimmed averaging and a performance-based approach stood out.</p><p>Another study by Liu et al. in 2017 tackled the tricky task of making reliable guesses about electricity load, crucial for managing unpredictable demand in smart grids [2].</p><p>They introduced a method that uses quantile regression averaging (QRA) on <strong>sister forecasts</strong> to make these probabilistic forecasts. Their method, tested on data from the Global Energy Forecasting Competition 2014, proved more accurate than others, showing the benefit of combining different forecasts to manage the variability in electricity demand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i8Be59uGQbE9YLYSRI8D_w.png" /><figcaption>From [2].</figcaption></figure><h3>From Theory to Practice</h3><p>Inspired by these academic findings, my team at xtream, in collaboration with A2A, Italy’s third-largest utility, set out to <strong>improve gas demand prediction using ensemble models</strong>. The results were published in the paper <em>Ensembling Methods for Countrywide Short-term Forecasting of Gas Demand </em>[3].</p><p>Our study focused on the residential, industrial, and thermoelectric sectors across Italy, starting with a deep dive into the statistical relationship between gas demand and temperature changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G0vWVrG_60XnNClSazAsUQ.png" /><figcaption>Image from [3] by the author.</figcaption></figure><p>The research highlighted the <strong>critical role of temperature and calendar features</strong> in accurately forecasting gas demand, prompting us to refine our models with tailored adjustments for <a href="https://en.wikipedia.org/wiki/Heating_degree_day">heating degree days</a> (HDD) and cooling degree days (CDD). These adjustments were instrumental in capturing the seasonal variations in gas demand, thereby improving the reliability of our forecasts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k9S7EyPmpB3vRo64vWvRfQ.png" /><figcaption>Residential Gas Demand (RGD) vs average daily temperature and Heating Day Degrees (HDD). Image from [3] by the author.</figcaption></figure><p>We tested nine base forecasting models, including Ridge Regression, Gaussian Processes, and Artificial Neural Networks, for their ability to predict next-day gas demand. Building on previous studies, we then created <strong>four ensemble models</strong> to leverage the collective strength of these base models.</p><p>Our <strong>ensemble models consistently outperformed individual models</strong>, showing their superiority in forecasting accuracy during a two-year test period. They even did better than forecasts from SNAM, Italy’s Transmission System Operator (TSO). Adopted by A2A, our models significantly reduced forecasting errors, saving the company hundreds of thousands of euros each year.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wmnyDWsh_GA9gPFIqdM9yg.png" /><figcaption>Image from [3] by the author.</figcaption></figure><p>The success with gas forecasting led us to develop <strong>new models for predicting power load</strong>, in partnership with Plenitude, Italy’s second-largest utility. Once again, our new models were ensemble, based on the research published in [1] and [2].</p><p>Our year-long project produced predictors for residential and industrial power load at a regional level, benefiting from the recent rollout of smart meters. By using data only a few days old, we significantly improved predictions, even for customers without smart meters.</p><p>The new system <strong>reduced forecasting error by 20% to 30%</strong>, saving over 500 thousand euros yearly in power network imbalance costs.</p><h3>References</h3><p>[1] Nowotarski, J., Liu, B., Weron, R., &amp; Hong, T. (2016). Improving short-term load forecast accuracy via combining sister forecasts. <em>Energy</em>, <em>98</em>, 40–49.</p><p>[2] Liu, B., Nowotarski, J., Hong, T., &amp; Weron, R. (2017). Probabilistic load forecasting via quantile regression averaging on sister forecasts. <em>IEEE Transactions on Smart Grid</em>, <em>8</em>(2), 730–737.</p><p>[3] Marziali, A., Fabbiani, E., &amp; Nicolao, G. D. (2021). Ensembling methods for countrywide short-term forecasting of gas demand. <em>International Journal of Oil, Gas and Coal Technology</em>, <em>26</em>(2), 184–201.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc6f9210b3de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Zillow: A Machine Learning Failure?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@donlelef/zillow-a-machine-learning-failure-e6a1fb59316e?source=rss-3439142d7553------2"><img src="https://cdn-images-1.medium.com/max/875/0*EIONaa8yjVF6L96J" width="875"></a></p><p class="medium-feed-snippet">An unbiased analysis of the craziest business downturn driven by machine learning.</p><p class="medium-feed-link"><a href="https://medium.com/@donlelef/zillow-a-machine-learning-failure-e6a1fb59316e?source=rss-3439142d7553------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@donlelef/zillow-a-machine-learning-failure-e6a1fb59316e?source=rss-3439142d7553------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6a1fb59316e</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[enterpreneurship]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Emanuele Fabbiani]]></dc:creator>
            <pubDate>Sat, 22 Jan 2022 22:01:18 GMT</pubDate>
            <atom:updated>2022-01-22T22:01:18.783Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
        </item>
        <item>
            <title><![CDATA[pytsviz: interactive time series charts with Plotly and Python]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/pytsviz-interactive-time-series-charts-with-plotly-and-python-68cf30d9f822?source=rss-3439142d7553------2"><img src="https://cdn-images-1.medium.com/max/2600/0*iR3GHPe3xJ9u6Oss" width="5472"></a></p><p class="medium-feed-snippet">Introducing a new library to speed up time series analysis</p><p class="medium-feed-link"><a href="https://medium.com/data-science/pytsviz-interactive-time-series-charts-with-plotly-and-python-68cf30d9f822?source=rss-3439142d7553------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/pytsviz-interactive-time-series-charts-with-plotly-and-python-68cf30d9f822?source=rss-3439142d7553------2</link>
            <guid isPermaLink="false">https://medium.com/p/68cf30d9f822</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <dc:creator><![CDATA[Emanuele Fabbiani]]></dc:creator>
            <pubDate>Tue, 28 Dec 2021 14:32:49 GMT</pubDate>
            <atom:updated>2021-12-28T14:32:49.207Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
        </item>
        <item>
            <title><![CDATA[Method is all you need: 7 mistakes to avoid in Data Science]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/method-is-all-you-need-7-mistakes-to-avoid-in-data-science-a18bec2c5e99?source=rss-3439142d7553------2"><img src="https://cdn-images-1.medium.com/max/2600/0*MDQAIanBdDNUU3Xm" width="5472"></a></p><p class="medium-feed-snippet">A curated collection from our experience</p><p class="medium-feed-link"><a href="https://medium.com/data-science/method-is-all-you-need-7-mistakes-to-avoid-in-data-science-a18bec2c5e99?source=rss-3439142d7553------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/method-is-all-you-need-7-mistakes-to-avoid-in-data-science-a18bec2c5e99?source=rss-3439142d7553------2</link>
            <guid isPermaLink="false">https://medium.com/p/a18bec2c5e99</guid>
            <category><![CDATA[editors-pick]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[project-management]]></category>
            <category><![CDATA[notes-from-industry]]></category>
            <dc:creator><![CDATA[Emanuele Fabbiani]]></dc:creator>
            <pubDate>Mon, 06 Dec 2021 11:43:30 GMT</pubDate>
            <atom:updated>2021-12-09T18:20:18.852Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-nc/4.0/</cc:license>
        </item>
        <item>
            <title><![CDATA[Stop copy-pasting notebooks,  embrace Jupyter templates!]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/stop-copy-pasting-notebooks-embrace-jupyter-templates-6bd7b6c00b94?source=rss-3439142d7553------2"><img src="https://cdn-images-1.medium.com/max/1648/1*d7SW7aZUw91tMEqRBZWm_A.png" width="1648"></a></p><p class="medium-feed-snippet">Introducing jupytemplate: configurable templates for Jupyter</p><p class="medium-feed-link"><a href="https://medium.com/data-science/stop-copy-pasting-notebooks-embrace-jupyter-templates-6bd7b6c00b94?source=rss-3439142d7553------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/stop-copy-pasting-notebooks-embrace-jupyter-templates-6bd7b6c00b94?source=rss-3439142d7553------2</link>
            <guid isPermaLink="false">https://medium.com/p/6bd7b6c00b94</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[jupyter-notebook]]></category>
            <dc:creator><![CDATA[Emanuele Fabbiani]]></dc:creator>
            <pubDate>Wed, 05 Feb 2020 20:13:57 GMT</pubDate>
            <atom:updated>2020-02-05T20:13:57.714Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Introducing tsviz, interactive time series visualization in R Studio]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/introducing-tsviz-interactive-time-series-visualization-in-r-studio-a96cde507a14?source=rss-3439142d7553------2"><img src="https://cdn-images-1.medium.com/max/600/1*ZgV0x7C-dHSXrSfmRo8CbQ.png" width="600"></a></p><p class="medium-feed-snippet">Why and how we developed an R Studio add-in</p><p class="medium-feed-link"><a href="https://medium.com/data-science/introducing-tsviz-interactive-time-series-visualization-in-r-studio-a96cde507a14?source=rss-3439142d7553------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/introducing-tsviz-interactive-time-series-visualization-in-r-studio-a96cde507a14?source=rss-3439142d7553------2</link>
            <guid isPermaLink="false">https://medium.com/p/a96cde507a14</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[r]]></category>
            <category><![CDATA[time-series-analysis]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Emanuele Fabbiani]]></dc:creator>
            <pubDate>Sat, 27 Jul 2019 11:17:26 GMT</pubDate>
            <atom:updated>2019-07-31T21:41:38.853Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Lessons from a real Machine Learning project, part 2: the traps of data exploration]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/lessons-from-a-real-machine-learning-project-part-2-the-traps-of-data-exploration-e0061ace84aa?source=rss-3439142d7553------2"><img src="https://cdn-images-1.medium.com/max/600/1*k0v39powS7PjJ7Qv_NPTjw.png" width="600"></a></p><p class="medium-feed-snippet">How to fall into the pitfalls of data exploration and get away</p><p class="medium-feed-link"><a href="https://medium.com/data-science/lessons-from-a-real-machine-learning-project-part-2-the-traps-of-data-exploration-e0061ace84aa?source=rss-3439142d7553------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/lessons-from-a-real-machine-learning-project-part-2-the-traps-of-data-exploration-e0061ace84aa?source=rss-3439142d7553------2</link>
            <guid isPermaLink="false">https://medium.com/p/e0061ace84aa</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Emanuele Fabbiani]]></dc:creator>
            <pubDate>Tue, 04 Jun 2019 12:15:03 GMT</pubDate>
            <atom:updated>2019-06-04T13:02:34.406Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Lessons from a real Machine Learning project, part 1: from Jupyter to Luigi]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/data-science/lessons-from-a-real-machine-learning-project-part-1-from-jupyter-to-luigi-bdfd0b050ca5?source=rss-3439142d7553------2"><img src="https://cdn-images-1.medium.com/max/600/1*12Ml9R813nJ6zea-il8HjQ.png" width="600"></a></p><p class="medium-feed-snippet">A brief story about how we realized how to use&#x200A;&#x2014;&#x200A;and how not not use&#x200A;&#x2014;&#x200A;Jupyter notebooks in Machine Learning projects</p><p class="medium-feed-link"><a href="https://medium.com/data-science/lessons-from-a-real-machine-learning-project-part-1-from-jupyter-to-luigi-bdfd0b050ca5?source=rss-3439142d7553------2">Continue reading on TDS Archive »</a></p></div>]]></description>
            <link>https://medium.com/data-science/lessons-from-a-real-machine-learning-project-part-1-from-jupyter-to-luigi-bdfd0b050ca5?source=rss-3439142d7553------2</link>
            <guid isPermaLink="false">https://medium.com/p/bdfd0b050ca5</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[luigi]]></category>
            <category><![CDATA[jupyter-notebook]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Emanuele Fabbiani]]></dc:creator>
            <pubDate>Mon, 06 May 2019 16:57:25 GMT</pubDate>
            <atom:updated>2019-05-09T16:16:19.185Z</atom:updated>
        </item>
    </channel>
</rss>