<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Christoph Klemenjak on Medium]]></title>
        <description><![CDATA[Stories by Christoph Klemenjak on Medium]]></description>
        <link>https://medium.com/@klemenjak?source=rss-26543efaa5c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*IGkt970VLEyxTWyE25--6A.png</url>
            <title>Stories by Christoph Klemenjak on Medium</title>
            <link>https://medium.com/@klemenjak?source=rss-26543efaa5c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 03:32:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@klemenjak/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to deal with NILMTK energy data sets using Pandas]]></title>
            <link>https://klemenjak.medium.com/how-to-deal-with-nilmtk-energy-data-sets-using-pandas-b43d7a1587f1?source=rss-26543efaa5c------2</link>
            <guid isPermaLink="false">https://medium.com/p/b43d7a1587f1</guid>
            <category><![CDATA[h5py]]></category>
            <category><![CDATA[nilmtk]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[energy-efficiency]]></category>
            <dc:creator><![CDATA[Christoph Klemenjak]]></dc:creator>
            <pubDate>Tue, 17 Aug 2021 09:44:02 GMT</pubDate>
            <atom:updated>2021-08-17T09:44:02.379Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<p>Datasets play a crucial role in the development of data-driven algorithms. Being able to access one or more real-world datasets is the deciding factor between success or failure. Before going further into the topic, we have to define NILM:</p><blockquote>Non-Intrusive Load Monitoring (NILM) is the process of estimating the energy consumed by individual appliances given just a whole-house power meter reading. In other words, it produces an (estimated) itemised energy bill from just a single, whole-house power meter. — Source: [1]</blockquote><p>For good reasons, many researchers have decided to rely on the NILM Toolkit (NILMTK) to evaluate the accuracy of their NILM algorithms (see [1] for details). Along with the toolkit, a custom data format was introduced in 2014 to ease experimentation as well as dealing with energy datasets. Over time, a respectable number of datasets has been released in that specific data format, which builds on the <strong>Hierarchical Data Format</strong> (<strong>HDF</strong>). For the sake of completeness, here’s a list of datasets compatible to NILMTK:</p><blockquote>AMPds, Caxe, Combed, Dataport, Deddiag, DRED, ECO, GREEND, HES, HIPE, iAWE, RAE, REDD, REFIT, Smart, SynD, and UK-DALE.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ewNNdoDCEP34cc1s_JCf8A.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@kobuagency?utm_source=medium&amp;utm_medium=referral">KOBU Agency</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>While dealing with these data sets works well as long as you use NILMTK, many people have been wondering <strong>how NILMTK datasets can be used in conjunction with more common Python libraries such as Pandas. </strong>This is exactly what this post is about. In this post, we will learn how to extract metadata as well as time series of NILMTK data sets by using Pandas. Therefore, we don’t have to go through installing NILMTK. As energy data set, we will use the popular AMPds data set (see [2] for details).</p><h3>Metadata</h3><p>Besides time series, NILMTK data sets store metadata following the NILM metadata scheme (see [4]). In order to identify the time series of interest, it is of advantage to explore the metadata of a data set first. This can be done using a <strong>HDFStore</strong> object. To explore the energy meters of AMPds, we open the data set and print all available keys :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0d1514fcdcddb023308d07f42e689a7/href">https://medium.com/media/e0d1514fcdcddb023308d07f42e689a7/href</a></iframe><figure><img alt="The keys of AMPds.h5" src="https://cdn-images-1.medium.com/max/1024/1*8bQ04a2HyHSlAh8XHvna3Q.png" /></figure><p>By printing all keys of the data set, we get a good overview. Clearly, these keys provide limited insights on the present electrical appliances (fridges, dishwashers, etc.). To access the metadata of building 1, we run:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ca18dabd836e237a0e7a214d2274bd9e/href">https://medium.com/media/ca18dabd836e237a0e7a214d2274bd9e/href</a></iframe><figure><img alt="Metadata of building 1: appliances as well as meter keys" src="https://cdn-images-1.medium.com/max/1024/1*hh3J6VIxMpsr3zBx3t9_hw.png" /></figure><p>This code snippet gives us information on the appliances in building 1 as well as the corresponding meter keys. For instance: we can see that the heat pump got the meter id 14. Now that we know the meter id of the heat pump, we will try to extract the corresponding data frame in the next step (i.e. all data related to it).</p><h4>Loading Dataframes</h4><p>At this point, we have successfully opened the data set and know what appliances are present. We also know the meter id of the heat pump and are ready to load data. We learned above that the meter key is 14, so we apply this knowledge to load the data frame using this code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2d35eef6009e70f06230d73364288591/href">https://medium.com/media/2d35eef6009e70f06230d73364288591/href</a></iframe><p>This gives us all data related to the heat pump in form of a single Pandas DataFrame object.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rSzx9_VRlJlqJ8slF0YejQ.png" /></figure><p>By looking at the columns of the data frame, we get an overview of the available physical quantities: voltage, current, power, energy, etc. That means there are several time series available. So clearly, the creators of the data set did a good job.</p><p>More good news: from here on, it’s business as usual in Pandas. To access the power readings, we run:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/73383ecb79e72edf9de961f6c6f295cf/href">https://medium.com/media/73383ecb79e72edf9de961f6c6f295cf/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tYY80lPX4DyNmijkh9IpQQ.png" /></figure><p>That’s all you need to know in order to extract metadata as well as time series from NILMTK data sets without actually using NILMTK.</p><p>We are done for today. How about a coffee?</p><h4>References</h4><ul><li>[1] <a href="https://github.com/nilmtk/nilmtk">https://github.com/nilmtk/nilmtk</a></li><li>[2] <a href="http://ampds.org/">http://ampds.org</a></li><li>[3] <a href="https://pandas.pydata.org/docs/reference/api/pandas.HDFStore.get.html">https://pandas.pydata.org/docs/reference/api/</a></li><li>[4] <a href="https://github.com/nilmtk/nilm_metadata">https://github.com/nilmtk/nilm_metadata</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b43d7a1587f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up Scikit learn with Conda]]></title>
            <link>https://klemenjak.medium.com/setting-up-scikit-learn-with-conda-89162cf8d9e9?source=rss-26543efaa5c------2</link>
            <guid isPermaLink="false">https://medium.com/p/89162cf8d9e9</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <category><![CDATA[jupyter]]></category>
            <dc:creator><![CDATA[Christoph Klemenjak]]></dc:creator>
            <pubDate>Mon, 22 Feb 2021 09:50:07 GMT</pubDate>
            <atom:updated>2021-08-13T11:01:38.018Z</atom:updated>
            <content:encoded><![CDATA[<h3>Setting up Scikit-Learn &amp; Conda.</h3><p>In this manual, we will set up a Conda environment for Scikit learn and Jupyter notebooks.</p><p>Virtual environments ease organising Python packages. Why Conda? Well:</p><blockquote>“With over 6 million users, the open source Anaconda Distribution is the easiest way to do Python data science and machine learning. It includes hundreds of popular data science packages and the conda package and virtual environment manager for Windows, Linux, and MacOS. Conda makes it quick and easy to install, run, and upgrade complex data science and machine learning environments like Scikit-learn, TensorFlow, and SciPy. Anaconda Distribution is the foundation of millions of data science projects as well as Amazon Web Services’ Machine Learning AMIs and Anaconda for Microsoft on Azure and Windows.”</blockquote><p><a href="https://www.anaconda.com/products/individual">Get Anaconda for Python here.</a></p><p>Open a terminal window and start the installation from command line:</p><pre>cd Downloads/<br>sudo bash Anaconda3-5.2.0-Linux-x86_64.sh -u</pre><p>This will initiate the installation process, which will guide you through several steps. Install Conda and test the installation by executing the command <em>conda</em> in the command prompt.</p><p>In case the command <em>conda</em> results in a bad interpreter error, apply the following fix:</p><pre>cd /home/user/<br>nano .bashrc</pre><p>add the line: export PATH=~/anaconda3/bin:$PATH</p><p>Finally, execute the command:</p><pre>source .bashrc</pre><p>Now create a new Conda environment:</p><pre>conda create --name eninf</pre><p>Next, activate the freshly created environment:</p><pre>conda activate eninf</pre><p>Finally, we begin with installing software packages:</p><pre>conda install scikit-learn matplotlib pandas jupyter</pre><p>Alternatively, open Anaconda Navigator and use the graphical user interface.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nHfVVrKG-YdCdAhNMDIu_A.png" /><figcaption>Please mind that this is merely a symbolic image, as our environment is named eninf and there is no nilmtk.yml in this context.</figcaption></figure><h4>Jupyter</h4><p>The next step involves Jupyter notebooks. Add the environment to Jupyter:</p><pre>python -m ipykernel install --user --name eninf --display-name &quot;Python (eninf)&quot;</pre><h4>Testing your Installation</h4><p>The time has come to check your installation. Create a new folder and open Jupyter.</p><pre>mkdir lab<br>jupyter notebook lab/</pre><p>Create a new notebook and don’t forget to use “Python (eninf)”. Fill in the <a href="https://scikit-learn.org/stable/auto_examples/cluster/plot_face_compress.html#sphx-glr-auto-examples-cluster-plot-face-compress-py">Vector Quantization example:</a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/743b61ea5420016e1771c707deecdafc/href">https://medium.com/media/743b61ea5420016e1771c707deecdafc/href</a></iframe><p>Here is what you should see now:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qjfP_sTSoSy9hS-NcOO-vQ.png" /></figure><p><strong>Congrats! You have made it.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=89162cf8d9e9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A step-by-step manual for installing NILMTK.]]></title>
            <link>https://klemenjak.medium.com/a-step-by-step-manual-for-installing-nilmtk-bff86e3aa418?source=rss-26543efaa5c------2</link>
            <guid isPermaLink="false">https://medium.com/p/bff86e3aa418</guid>
            <category><![CDATA[energy-disaggregation]]></category>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[nilmtk]]></category>
            <category><![CDATA[nilm]]></category>
            <dc:creator><![CDATA[Christoph Klemenjak]]></dc:creator>
            <pubDate>Sun, 21 Feb 2021 23:53:23 GMT</pubDate>
            <atom:updated>2021-08-13T10:59:21.041Z</atom:updated>
            <content:encoded><![CDATA[<p>In this manual, we will install the <em>user version of NILMTK</em>. Please note that this manual considers Mac OS and Linux systems only. The installation procedure for Windows might differ from the one presented in this blog post.</p><h4>Publications related to this topic</h4><ol><li>Klemenjak, C., &amp; Goldsborough, P. (2016). <a href="https://arxiv.org/pdf/1610.01191.pdf">Non-intrusive load monitoring: A review and outlook.</a> arXiv preprint arXiv:1610.01191.</li><li>Klemenjak, C., Makonin, S., &amp; Elmenreich, W. (2020). <a href="https://arxiv.org/abs/2001.07708">Towards comparability in non-intrusive load monitoring: On data and performance evaluation.</a> In 2020 IEEE power &amp; energy society innovative smart grid technologies conference (ISGT) (pp. 1–5). IEEE.</li><li>Klemenjak, C., Makonin, S., &amp; Elmenreich, W. (2020). <a href="https://arxiv.org/pdf/2008.10985.pdf">Investigating the Performance Gap between Testing on Real and Denoised Aggregates in Non-Intrusive Load Monitoring.</a> arXiv preprint arXiv:2008.10985.</li></ol><h4>Sources</h4><p>This manual builds on material provided by the NILMTK project:</p><ul><li><a href="https://github.com/nilmtk/nilmtk/blob/master/docs/manual/user_guide/install_user.md">NILMTK User Guide</a></li><li><a href="https://github.com/nilmtk/nilmtk/blob/master/docs/manual/user_guide/install_dev.md">NILMTK Documentation</a></li><li><a href="https://github.com/nilmtk/nilmtk-contrib">NILMTK Contrib repository</a></li><li><a href="https://nipunbatra.github.io/papers/batra_buildsys_19.pdf">Nipun Batra’s BuildSys’19 paper on NILMTK</a></li></ul><p>Thanks for the great work!</p><h4>Step 1: Download and Install Conda</h4><p>Virtual environments makes organizing Python packages a piece of cake. Also, the NILMTK project offers several versions on Conda forge. First, <a href="https://www.anaconda.com/download/">get Anaconda here.</a> Then, open a terminal window and start the installation from command line:</p><pre>cd Downloads/<br>bash Anaconda3-2020.11-Linux-x86_64.sh -u</pre><p>You will be guided through several steps. Install Conda and test the installation by executing the command <em>conda</em> in the command prompt:</p><pre>conda</pre><p>In case the command <em>conda</em> results in “a bad interpreter error” (i.e. no such file or directory error), apply the following fix:</p><pre>cd /home/user/<br>nano .bashrc</pre><p>add the line:</p><pre>export PATH=~/anaconda3/bin:$PATH</pre><p>Save the changes to the file and, finally, execute the command:</p><pre>source .bashrc</pre><p>Congrats, you just installed Conda.</p><h4>Step 2: Installing NILMTK</h4><p>Open a terminal window, create a new Conda environment and activate it:</p><pre>conda create --name nilmtk-env<br>conda activate nilmtk-env</pre><p>If not already present in your Conda installation, add the conda-forge to the list of channels:</p><pre>conda config --add channels conda-forge</pre><p>Finally, install the latest NILMTK version from <em>conda-forge</em>:</p><pre>conda install -c nilmtk nilmtk=0.4.2</pre><p>This may take a while. In the meantime, check out further packages on NILMTK’s <a href="https://anaconda.org/nilmtk">Forge</a>.</p><p>Update February 22nd: I have experienced some post-installation problems with NILMTK and Matplotlib. To fix them, apply the command:</p><pre>conda install matplotlib=3.3.2</pre><h4>Step 3: Add a Jupyter Kernel</h4><p>Basically, NILMTK is now installed on your computer. The next step involves Jupyter notebooks. Working with Jupyter opens up many possibilities and is said to be a must-have tool. Therefore, add the NILMTK environment to Jupyter:</p><pre>python -m ipykernel install --user --name nilmtk-env --display-name &quot;Python (nilmtk-env)&quot;</pre><h4>Endgame: Test your Installation</h4><p>The time has come to check your installation. Before anything else, create a new folder and download the random test set from Github:</p><pre>mkdir nilmtk_test/<br>cd nilmtk_test/<br>wget <a href="https://raw.githubusercontent.com/nilmtk/nilmtk/master/data/random.h5">https://raw.githubusercontent.com/nilmtk/nilmtk/master/data/random.h5</a></pre><p>Next, fire up Jupyter:</p><pre>jupyter notebook</pre><p>To test your installation, try to import the random dataset using NILMTK and plot all meters:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9ba2be16d331653a7b4093a0fe412434/href">https://medium.com/media/9ba2be16d331653a7b4093a0fe412434/href</a></iframe><p>The output should be:</p><pre>MeterGroup(meters=<br>  ElecMeter(instance=1, building=1, dataset=None, appliances=[])<br>  ElecMeter(instance=2, building=1, dataset=None, appliances=[])<br>  ElecMeter(instance=3, building=1, dataset=None, appliances=[])<br>  ElecMeter(instance=4, building=1, dataset=None, appliances=[])<br>  ElecMeter(instance=5, building=1, dataset=None, appliances=[])<br>)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/1*AxqR8Ybq6HRoHrvYvoemSA.png" /></figure><p><strong>Congrats! You have made it. NILMTK seems to work as intended! Next time, we will discuss how to use NILMTK’s API for rapid &amp; convenient experimentation.</strong></p><p>We are done for today, how about a coffee?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bff86e3aa418" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A User-Oriented Energy Monitor based on the Raspberry Pi]]></title>
            <link>https://klemenjak.medium.com/a-user-oriented-energy-monitor-based-on-the-raspberry-pi-b1fa93174cf0?source=rss-26543efaa5c------2</link>
            <guid isPermaLink="false">https://medium.com/p/b1fa93174cf0</guid>
            <category><![CDATA[raspberry-pi]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[smart-meter]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[open-hardware]]></category>
            <dc:creator><![CDATA[Christoph Klemenjak]]></dc:creator>
            <pubDate>Fri, 19 Feb 2021 00:18:39 GMT</pubDate>
            <atom:updated>2021-08-13T11:00:21.228Z</atom:updated>
            <content:encoded><![CDATA[<h3>YoMoPie: A User-Oriented Energy Monitor based on the Raspberry Pi</h3><p>Computational methods for the enhancement of energy efficiency rely on a measurement process with sufficient accuracy and number of measurements. Networked energy meters, energy monitors, serve as a vital link between the energy consumption of households and key insights that reveal strategies to achieve significant energy savings.</p><blockquote>With YoMoPie, we propose a user-oriented energy monitor for the Raspberry Pi platform that aims to enable intelligent energy services in households. YoMoPie measures active as well as apparent power, stores data locally, and integrates a user-friendly Python library.</blockquote><p>Furthermore, the presented energy monitor allows users to run self-designed services in their home to enhance energy efficiency. Potential services are (but not limited to) residential demand response, immediate user feedback, smart meter data analytics, or energy disaggregation.</p><p>All design files and pieces of software are available free of charge. However, in case you use the PCB design, code, or other material for research purposes, we kindly ask you to cite our peer-reviewed research paper:</p><blockquote>Klemenjak, C., Jost, S., &amp; Elmenreich, W. (2018, November). <a href="http://mobile.nes.aau.at/publications/klemenjak-2018-yomopie.pdf">Yomopie: A user-oriented energy monitor to enhance energy efficiency in households.</a> In 2018 IEEE Conference on Technologies for Sustainability (SusTech) (pp. 1–7). IEEE.</blockquote><p>Key Features of YoMoPie:</p><ul><li>Communication via WiFi, Ethernet, RF</li><li>YoMoPie measures active power, apparent power, electric current, and voltage</li><li>Maximum sampling frequency: 10 Hz</li><li>The integrated relay allows to switch electric loads</li><li>Open-Source / Open-Hardware Project</li><li>Compatible to the Raspberry Pi platform</li><li>Costs: approx. 50€</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WX8C_y4RffVawJaT.JPG" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/0*uU7_DnL1PWUvqQcd.PNG" /></figure><h4>Installation</h4><p>The YoMoPie Python package is available on Python Package Index (PyPI), a software repository for the Python programming language and can easily be installed:</p><pre>pip3 install YoMoPie</pre><p>Additionally, the entire source code, design files, and a manual can be obtained from our YoMoPie Github repository.</p><h4>Examples of use</h4><p>After a successful installation process, the YoMoPie package is available system-wide and can be accessed by a simple import command:</p><pre>import YoMoPie as yomopie<br>yp = yomopie.YoMoPie()</pre><p>During initialisation, the number of line conductors has to be set (single or polyphase metering):</p><pre>yomo.set_lines(1)</pre><p>To test the operation, we recommend to call the function <em>do_n_measurements</em>. Based on the number of samples and the sampling period, the function will return first measurement values and saves it into the target file:</p><pre>yomo.do_n_measurements(number of samples, sampling period, target file)</pre><p>Active power, apparent power, current, and voltage samples can be read with commands such as:</p><pre>[t, I] = yp.get_irms()<br>[t, U] = yp.get_vrms()<br>[t, P] = yp.get_active_energy()<br>[t, S] = yp.get_apparent_energy()</pre><p>In the same vein, users can activate continuous data logging or perform a fixed amount of subsequent measurements:</p><pre>yp.do_metering()<br>yp.do_n_measurements(quantity, rate, file)</pre><p>The operational mode (OPMODE) register defines the general configuration of the integrated measurement chip ADE7754. By writing to this register, A/D converters can be turned on/off, sleep mode can be activated, or a software chip reset can be triggered. For further information, we refer to the datasheet of the measurement chip.</p><pre>yp.set_operational_mode(OPMODE)</pre><p>Find further information on our <a href="https://klemenjak.github.io/YoMoPie">Github Page.</a></p><p>We are done for today, how about a coffee?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1fa93174cf0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>