<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lucas Ramos on Medium]]></title>
        <description><![CDATA[Stories by Lucas Ramos on Medium]]></description>
        <link>https://medium.com/@lucasramos-34338?source=rss-282d1abf0b29------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*IUn-ULWyCz6w02m0</url>
            <title>Stories by Lucas Ramos on Medium</title>
            <link>https://medium.com/@lucasramos-34338?source=rss-282d1abf0b29------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 18:24:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@lucasramos-34338/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Renting a house in The Netherlands as an Expat: Company review around Amsterdam]]></title>
            <link>https://lucasramos-34338.medium.com/renting-a-house-in-the-netherlands-as-an-expat-company-review-around-amsterdam-4057d7aca54f?source=rss-282d1abf0b29------2</link>
            <guid isPermaLink="false">https://medium.com/p/4057d7aca54f</guid>
            <category><![CDATA[netherlands]]></category>
            <category><![CDATA[house]]></category>
            <category><![CDATA[apartments]]></category>
            <category><![CDATA[renting]]></category>
            <category><![CDATA[expat]]></category>
            <dc:creator><![CDATA[Lucas Ramos]]></dc:creator>
            <pubDate>Sun, 27 Feb 2022 12:51:23 GMT</pubDate>
            <atom:updated>2022-02-27T12:51:23.977Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OMKl0H8h4t8ZrrPu6mkenA.jpeg" /></figure><p>After living more than 5 years in The Netherlands and moving around 5 times, I can tell with certainty that renting a house or apartment is one of the worst experiences I’ve ever had.</p><p>In no other activity, I’ve felt so ignored and helpless. The whole process of viewing a house, indicating interest, and being “randomly” chosen is just a complete nightmare, especially if you don’t speak dutch.</p><p>Since I’ve lost a lot of time and money in this process, I’ve decided to write this post in the hope that it will save them the time and money of other expats.</p><p>Disclaimer: This post is about my own experience and opinion about these companies. Others might have different experiences and there’s no guarantee you will have the same experience I’ve had. Good luck!</p><h3>Amsterdam Region</h3><p>Since I worked in Amsterdam I explored some options around (being Amsterdam waaaay too expensive for my budget). Below is a list of real state agents I had contact with.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/1*WprJV44O7pomAl0Op3R8TA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/172/1*2L8vwa_HLpDR2lT5HR4q_g.png" /></figure><p><strong>GOVAERT </strong>is by far the best makelaardij (state agent)I had contact with (situated in Amersfoort). Fast response by e-mail and phone, easy to schedule house viewings, and super expat-friendly. They spoke perfect English with me from the first viewings to the moment of signing the contract and delivering the keys. Besides, when my income was not as high as required to rent a certain place, they would offer me other options (like 2 months as a deposit instead of 1).</p><ul><li>Don’t charge anything from you (their client is the landlord)</li><li>Eiger to provide documents in English</li><li>Private house viewings</li><li>Expat friendly staff</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/266/1*geSObfOtrJ2WTpbsX2iNPg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/169/1*eBWjsmkkSa5fztG0_Il9Yw.png" /></figure><p><strong>EERVAST</strong> is also a flexible expat-friendly makelaardij. Communication with them was smooth, I managed to schedule apartment viewings by phone and e-mail. Their staff was friendly and helpful.</p><p>Most importantly, they also require your Bruto income to be 3 times the rent, but they are more flexible with that, for instance, as an expat I had the 30% ruling, which meant my net salary was a bit higher, so even though my total Bruto was not sufficient, they still accepted once I showed proof of my salary.</p><ul><li>Paid and unpaid option for state agent (be aware if you hire their search services, this will have costs)</li><li>Friendly with English speakers</li><li>Flexible with requirements</li><li>Private house viewings</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hfuN7vK40c9ZWh4UQrjekg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/166/1*lZqu1j0CkTN0dqqSum4PBQ.png" /></figure><p>I had two very different experiences with <strong>MVGM</strong>, one was quite positive, and the other a total waste of time and money.</p><p><strong>MVGM </strong>has strict income requirements, often requiring you to earn 4 to 6 times the total rent value (Brutto salary). Therefore, renting a house alone is often quite challenging.</p><p>Luckily, I had a dutch friend looking for a place as well, so we rented an apartment together. She handled the whole communication, scheduling views, and the documentation, so I can’t say much about this part, but the income seemed to be the biggest obstacle.</p><p>In my second, single-income experience I got absolutely nothing. You are supposed to pay 6o euros to be able to schedule viewings, but if your income is not in the 4–6 times the total rent they won’t let you view/rent anything. Around Amsterdam, that means a salary of at least 5.500 euros.</p><ul><li>Charges you 60 euros for signing up (membership lasts 1 year)</li><li>Only possible to rent if income is above the threshold</li><li>Terrible communication, only e-mail alerts</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/353/1*mf5C464qh53j_5QJZ-4ihw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/165/1*9HzSYyijp7igbLuATBETkg.png" /></figure><p><strong>Nederwoon </strong>was by far the worst of them. They have a full <strong>PAID </strong>online system where you can indicate you are interested in a house and if your income is above the threshold you may schedule the viewing. I managed to schedule 1 or 2 viewings in a period of 3 months. The viewings were scheduled with multiple people, so we had 4 or 5 couples at once in a 50 square meter apartment. Of course, the state agent would speak dutch and I felt like a ghost the whole time. I left the viewings with no information, absolutely terrible.</p><ul><li>Charges you 60 euros for signing up (membership lasts 1 year)</li><li>Only possible to rent if income is above the threshold</li><li>Multiple people scheduled to the same time slot during viewings</li><li>Unfriendly staff for expats</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4057d7aca54f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating your own DataLoader in PyTorch for combining images and tabular data]]></title>
            <link>https://medium.com/analytics-vidhya/creating-your-own-dataloader-in-pytorch-for-combining-images-and-tabular-data-cc2231119939?source=rss-282d1abf0b29------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc2231119939</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[dataloader]]></category>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[research]]></category>
            <dc:creator><![CDATA[Lucas Ramos]]></dc:creator>
            <pubDate>Fri, 29 Jan 2021 10:39:35 GMT</pubDate>
            <atom:updated>2021-01-29T16:21:39.078Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pRLb3pLyEgnSYS8ZebrMPg.png" /></figure><p>The main goal of this post is to show how you can load images and metadata/tabular using a DataLoader in Pytorch, create batches and feed them together to the network. This is often desired when we want to combine the metadata to the images at some point in the network.</p><p>A good example of when this could be useful in the case of clinical data. Clinical data is often composed of patient information (here referred to as metadata), like age and sex, but also other information, like the exams the patient underwent. These exams often generate images, that contain lots of useful information.</p><p>Both data sources generate data that is unique and can add a lot to multiple prediction tasks. To make the most of all the information available, Convolutional Neural Networks can be trained on the imaging data, and the metadata can be added to, let’s say, the dense layers to provide extra information during prediction.</p><p>There are many tutorials for creating your own DataLoaders in Pytorch. One of the best of them is available in the Pytorch documentation, you can check it by clicking <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">HERE</a>. Below I will explain how each section of the DataLoader works and how you can adapt it for your needs.</p><h3>The Dataset</h3><p>One of the most important things for a simple and straightforward DataLoader is to <strong>structure</strong> your data well.</p><p>By structure I mean the following: make sure there’s a clear connection to your images and your tabular data. This can be a digit, a column with the image id, name, path, it doesn’t matter, as long as you have a clear link between the variables in the table and one or multiple images.</p><p>In our case, we are using the test set from the OSIC Pulmonary Fibrosis Progression dataset, available <a href="https://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/data?select=test.csv">HERE</a>. We will use the image name as a connection to the clinical table.</p><p>This is what our tabular data looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*639MSRjE2aULtvIj7ZWrUA.jpeg" /><figcaption>Tabular data</figcaption></figure><p>The most important columns are the <strong>Patient </strong>column, which has the name of the images and is the link to the image data, and the <strong>FVC</strong>, which is our label. The rest of the variables are to be used as extra data and be combined with the images in the network.</p><p>You can find the complete code by clicking below. I will further describe what is happening inside each function below.</p><h3><strong>How does it work?</strong></h3><p>A DataLoader will load a sample per time, but it will return a tensor of the size of the batch. It is a magical thing that will make training a lot faster and your code more organized.</p><p>Below you have the definition of the DataLoader class and a link to the complete code (so you can copy and paste if you need it).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/01b52f9ed3c56803da05d263d934ce04/href">https://medium.com/media/01b52f9ed3c56803da05d263d934ce04/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*PmRhyiEVRuszAfzJEHAAtA.jpeg" /><figcaption>DataLoader class</figcaption></figure><p>In the class above, we have an Init method, that initializes the variables we will use in the DataLoader. The method <em>__getitem__</em>, is responsible for loading an instance of our data. The DataLoader will automatically call this function multiple times until the batch size is reached. The beauty here is that you don’t have to worry about calling this function or controlling the batch size and when to stop, the DataLoader will do it all for you.</p><p>In the <em>__getitem__</em> method, we use the <em>idx </em>that is controlled by the DataLoader and will increment it automatically, to read a sample from the <em>idx </em>position. We read the dicom image given by the column that connects images to tabular data and proceed to read the tabular data itself and the label.</p><p>Once the DataLoader reaches the end of a batch, it will return the batch so it can be used to train a model. The<em>__len__</em> is used by the DataLoader to mark the end of an epoch.</p><p>Finally, you just need to create an instance of CombineDataset class we defined and feed it to a DataLoader instance. in this case we did it only for the training set, but you can create multiple DataLoaders, one for training, validation and test sets.</p><p>When defining the DataLoader instance, you can select the batch size, if you want to shuffle your data or use batches in sequence, the number of multi-process to speed up data loading and if you want to have a smaller batch in the end in case some samples are left and are not enough to make a whole batch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*WYqCY2FRNz5xS6_iXxYopg.jpeg" /><figcaption>DataLoader in action</figcaption></figure><p>That’s all, you have a working DataLoader that reads and connects the tabular and image data. The next step is to feed this data to a network. For the sake of length and simplicity, I will present this in the following article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc2231119939" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/creating-your-own-dataloader-in-pytorch-for-combining-images-and-tabular-data-cc2231119939">Creating your own DataLoader in PyTorch for combining images and tabular data</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Github for Research: a quick and painless tutorial to improve your papers]]></title>
            <link>https://lucasramos-34338.medium.com/github-for-research-a-quick-and-painless-tutorial-to-improve-your-papers-ebd027260f2a?source=rss-282d1abf0b29------2</link>
            <guid isPermaLink="false">https://medium.com/p/ebd027260f2a</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[publishing]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[phd]]></category>
            <dc:creator><![CDATA[Lucas Ramos]]></dc:creator>
            <pubDate>Wed, 12 Aug 2020 08:18:12 GMT</pubDate>
            <atom:updated>2020-08-14T08:18:52.824Z</atom:updated>
            <content:encoded><![CDATA[<p>If you do any sort of programming you probably already heard about Github, and have been encouraged to, somehow, use it.</p><p>Now, I’ve seen plenty of repositories on the internet that lack the very basics of Git, and significantly ruin its purpose.</p><p>In this tutorial, I will focus on the very minimum necessary to keep your projects organized on Git, so that you and other people can make proper use of them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*t1XrqAKZrtXCXyTY1RuboA.png" /><figcaption>Source: <a href="http://www.phdcomics.com/">www.phdcomics.com</a></figcaption></figure><h3><strong>What is Git for?</strong></h3><p>The main use of Git is for version control. Version control is useful if you make any changes down the line that ruin your code, or that you later deem unnecessary, it allows you to easily revert things back to a working version.</p><p>Git is also a nice way of sharing your code with others, allowing people to work together, commenting, and keeping things up to date.</p><h3><strong>How do I get started?</strong></h3><p>Now, there are many options of platforms out there, like <a href="https://github.com/">Github</a>, <a href="https://about.gitlab.com/">GitLab</a>, <a href="https://bitbucket.org/product">Bitbucket</a>, and so on. I will show you how things work for Github, but mostly the interface will change if you pick a different one.</p><p>The first thing we will do is create our repository. You can do this by clicking on the + sign on the top right corner as shown in the image below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/1*kn7iq4dwi8L-G4Yx43S1Fw.png" /><figcaption>Creating a new repository</figcaption></figure><p>Give it a name and set it to either public or private (some of these functionalities might not be available depending on what type of account you have).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*o1yIRZjfn4cqBL-CN3u9bA.png" /></figure><p>Click <em>create repository</em> and you are done (you can also check the <em>initialize this repository with a README</em> box, especially if you intend to share this repository with others), this will allow you to add a nice introduction and explanation to your repository.</p><p>The next step is to connect your git repository to your computer, so you can send code from your machine to the online repository and keep track of your changes. In the Figure below you can see how to clone your repository to a local folder (in my case, f/medium).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/1*lNe8Z6hGNQIHd5MsV9XhyA.png" /><figcaption>Cloning your repository</figcaption></figure><p>Now it is time to add some code to your repository, we will add the following code that computes the BMI given weight and height, to the folder where we cloned our repository (f/medium/test_medium).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/273/1*cCsAQvaAi24TjcQXVwEctA.png" /><figcaption>Code to be added to the repository</figcaption></figure><p>With the code copied to the folder, we will use the GitBash command line to add our code to the repository. The image below shows the commands and their results in the GitBash terminal. First, we move to the repository with &lt;<em>cd test_medium&gt; </em>(always without &lt;&gt;)<em>. </em>Then we add all the files we want to add to the repository <em>&lt;git add .&gt;</em> (you can replace . by the name of the file you want to add, ex: <em>&lt;git add main.py&gt;</em>), the dot will add everything in the folder.</p><p>Then we commit our code with <em>&lt;git commit -am ‘First commit’&gt;.</em> You can change the text inside ‘’ to something more meaningful. This will save the changes to our local repository, which means a version of the code is saved on our machine.</p><p>Finally, we push the changes to the remote server with <em>&lt;git push origin master&gt;</em> (which means our code is now in the Github repository we created online and is available to others)<em>.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*_Fn32OJHviic0mSeNqs9TA.png" /><figcaption>Commands in the GitBash</figcaption></figure><p>If you check your Github repository, you will find your code there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*S2OrycdleF0H2A-V5drLlw.png" /><figcaption>Code in the Github repository</figcaption></figure><p>Now we have the first version of our code in the repository. But let’s say we forgot to add something important, in this case, the result of the BMI for a given input, so we change our code to:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/265/1*kIDSW3aF83U15nqTGi2RZQ.png" /><figcaption>Added input to the function</figcaption></figure><p>I would like this to be the latest version of the code online since the previous one is lacking content. For that, we just repeat the procedure shown above, <em>git add, git commit, git push</em>.</p><p>If we check the repository now (image below), we will see that there’s a second commit, with the message (Second commit).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DATsgBqLoBODFRUqMce1yg.png" /></figure><p>If we click History (on the right of the image above), we can see all the commits from this repository, together with the changes made in each one of them:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*il7VVGhKHfyvY4Ac6kh9zw.png" /><figcaption>All commits from this file</figcaption></figure><h3>Maintaining your repository</h3><p>Now, this is where a lot of people make a mistake. Let’s say you are done with the first part of this project and will work on something else, despite still related to the topic. The code you committed is the latest version of your code and you would like to keep it that way, maybe because other people might use it later, maybe it is referenced in one of your papers, or most importantly, you don’t know how your new changes will affect the code and want to have a stable version there.</p><p>This last version should be easily available to other people, despite your new changes. In our case, let’s say we will change the height from meters to centimeters. This could greatly impact your code and hamper the reproducibility of your results since this change is not documented yet. Besides, it might not integrate well with other functions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*Ya2BH1RoC-NTfGCbImq_Cg.png" /><figcaption>New code to be added, note that our function has a new name and handles things slightly differently.</figcaption></figure><p>For this new code, we will create a branch. A branch is basically a unique partition of your repository, where you can make changes, and then later merge them to the main branch. A repository can have multiple branches.</p><p>We create a branch with the command <em>&lt;git checkout -b bmi_cm&gt;</em>, where <em>bmi_cm</em> is the name of our new branch, and repeat the series of commands to add, commit and push our code to our new branch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/419/1*Ej7xO3X3lPfzEjaAUYTusQ.png" /></figure><p>If you succeeded, you will be able to see your new branch in your repository, and now there will be two versions of your code there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/1*fjr4MOWH8WHSbRApDdzMbw.png" /></figure><p>Now, here comes the best part of the way we are structuring things. Let’s say you need to go back to the older version, where we used meters instead of centimeters, and make some changes in that code (maybe someone asked for a new function).</p><p>All you need to do is <em>&lt;git checkout master&gt;</em>, and you are back to the older version of your code. Most IDEs (like Spyder, PyCharm, Visual Code) will update the code you have open automatically (make sure you save your changes and commit them before you move around branches).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/1*u0KuntaFYiQ1C4Ppdxdg0A.png" /><figcaption>Changing out current code to the older one in the master branch</figcaption></figure><p>You have now, two working versions of your code that you can easily edit without having multiple folders or updating a new version and losing the older one.</p><p>If at some point you don’t need one of them anymore, you can easily merge them with <em>&lt;git merge bmi_cm&gt;.</em></p><h3>What else can you do?</h3><p>There is still a lot more about Git. Here are some points you can explore:</p><p>Tags [Reader suggestion]: Tags are useful to create code releases in the main branch. You probably have seen that all libraries have some sort of release number (e.g numpy=1.15). You can create that for your code as well, to make it easier to find which version was used to obtain the results from your paper. It will make your code and repository a lot more organized.</p><p>Learn about <a href="https://docs.github.com/en/enterprise/2.13/user/articles/fork-a-repo#:~:text=A%20fork%20is%20a%20copy,point%20for%20your%20own%20idea.">forking</a>, so you can work with other peoples code and submit changes.</p><p>If you don&#39;t like the command prompt or GitBash, you can check alternatives with a more friendly interface, like <a href="https://www.sourcetreeapp.com/">SourceTree</a>.</p><p>Learn about solving conflicts when <a href="https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/resolving-a-merge-conflict-using-the-command-line">merging</a>.</p><p>Following some advanced <a href="https://www.atlassian.com/git/tutorials/advanced-overview">Git tutorials</a>.</p><p>*Are you missing something? Would you like to learn more about a specific topic? Leave your opinion below or send me an e-mail (l.a.ramos@amsterdamumc.nl).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ebd027260f2a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The effect of data normalization in your models: should we use a pipeline?]]></title>
            <link>https://medium.com/analytics-vidhya/the-effect-of-data-normalization-in-your-models-should-we-use-a-pipeline-6b97510f1908?source=rss-282d1abf0b29------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b97510f1908</guid>
            <category><![CDATA[data-leak-prevention]]></category>
            <category><![CDATA[data-leak]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-normalization]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Lucas Ramos]]></dc:creator>
            <pubDate>Fri, 31 Jul 2020 08:58:50 GMT</pubDate>
            <atom:updated>2020-08-07T13:55:04.350Z</atom:updated>
            <content:encoded><![CDATA[<h3>Are you making this mistake? How to perform data normalization in your models.</h3><p>If you have been working with Scikit-learn for a while you probably already heard somewhere about how much better and safer a <strong>pipeline </strong>is. Pipeline is a Scikit-learn function used to aggregate all the steps between data pre-processing and model training/optimization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DHI4f8V5eC5KXMsXhqdinQ.jpeg" /></figure><p>In this post I will reference the functions and libraries in the figure below, you can find the whole code <a href="https://colab.research.google.com/drive/1GE-kyeQA0bz27zKhf_6aQUKWqFK3R9rN#scrollTo=LmrsMHIJmZb4">HERE</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/585/1*EVW6DHjwOdw3jf7Jefe2dA.png" /><figcaption>Some useful functions for this post</figcaption></figure><p>One of the main advantages of using a pipeline is for data standardization. Data standardization is an important step for many machine learning classifiers. For some distance-based ones, like the Support Vector Machine (SVM), it is of utmost importance to make sure that the computed distances between features are not dominated by large values. For others, like Neural Networks, can severely improve training and converging time.</p><p>It is good practice to split the data into a train and test set prior to using a standardization method. The method is trained using the training set and with the transform function, is applied to the test set. This prevents data leakage from the training set into the testing set, which can severely bias our results. See the figure below for the code example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/374/1*m-AfWIHvqdllZ6AbFHd0CQ.png" /><figcaption>Common practice for the StandarScaler</figcaption></figure><p>Often, the next step is to find the best parameters for our classifier. For that, we often use a form of grid search, either the RandomizedSearchCV or GridSearchCV. See the figure below for an example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*cs1aV_6Wpl72NhvsYnsyCA.png" /><figcaption>Example of GridSearch after data standardization</figcaption></figure><p>Now, here’s a problem that not using a pipeline will raise. GridSearch will use only the training set and will split it into <em>N</em> folds (<em>N </em>is given by the <em>cv</em> parameter). <em>N-1</em> folds are user to train and the remaining 1 fold to test and see how good a given combination of hyper-parameters is. But, when we standardized the data, we did not take into account the splits (inner cross-validation) that would happen in the GridSearch step. We used our whole training set to train the standardizer, therefore, the splits generated during GridSearch are not 100% independent. The model is optimizing hyper-parameters on a train and validation split where data leakage was introduced, and therefore, our optimization results might be biased.</p><p>Your test set is not really affected by this small problem, but the hyper-parameters chosen during GridSearch might not generalize as well in the testing set, since the process to find them was not 100% trustworthy.</p><p>Now, this could be easily avoided if we used the Pipeline class from Scikit-learn. In the Figure below, we define each of the steps that will happen in the pipeline during GridSearch (Standardization and the classifier to be optimized).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*Gi3jXVBgG-Gh_3X54SRztA.png" /></figure><p>The approach above will guarantee that the splits inside GridSearch are done in a proper way, preventing data leakage.</p><p>Now, what is the actual effect of using or not a pipeline? To show this, I created 2 experiment settings using the Pima Indians Diabetes Database (from a Kaggle challenge: available <a href="https://www.kaggle.com/uciml/pima-indians-diabetes-database">here</a>) and the Breast Cancer Wisconsin (Diagnostic) Data Set (also from Kaggle: available <a href="https://www.kaggle.com/uciml/breast-cancer-wisconsin-data">here</a>).</p><p>For the diabetes dataset, the objective is to predict whether a given patient has diabetes or not. For the breast cancer dataset, the objective is to classify cancer into malignant or benign.</p><p>I split both data sets using 5 fold cross-validation and for each k-fold iteration, I standardized the data with and without a pipeline and optimized the hyper-parameters. For each k-fold iteration, I recorded the area under the receiver operating characteristic curve (AUC)for the test set and the hyper-parameters resulted from the grid-search.</p><p>Here’s the code so you can follow it easily (to run it in Colab you will need to download the data from Kaggle in the links provided above):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4a28d27b4f09b68e3303f82eee1714f9/href">https://medium.com/media/4a28d27b4f09b68e3303f82eee1714f9/href</a></iframe><h3><strong>Results — </strong>Pima Indians Diabetes Database</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/406/1*kY_OmseJRyFif7Ux9Ppf7g.png" /><figcaption>Results from 5-fold cross-validation</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*t5vOB-wNuLCYPI2fWd_wGQ.png" /><figcaption>Hyper-parameters chosen for each k-fold iteration</figcaption></figure><h3><strong>Results — Breast Cancer Wisconsin</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/402/1*WFrnO4uKoPO5_cGYM4nNBQ.png" /><figcaption>Results from 5-fold cross-validation</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*x4qzOXipePUXuGJaasqrmw.png" /></figure><p>As you can see, I could not find any differences between using a pipeline or normalizing the data separately. The AUC and the hyper-parameters chosen were exactly the same for both datasets. That could be, of course, due to the datasets themselves (which are not very complex) or to the chosen model *.</p><p>That being said, the theoretical reasoning behind the use of the pipeline function is unquestionably correct and its use will make your code not only cleaner and easier to maintain, but also safer against over-optimistic results.</p><p>*Do you think there’s something wrong in the code? Is there another experiment you would like to see? Leave a comment or send me an e-mail. l.a.ramos@amsterdamumc.nl</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b97510f1908" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/the-effect-of-data-normalization-in-your-models-should-we-use-a-pipeline-6b97510f1908">The effect of data normalization in your models: should we use a pipeline?</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Visualizing variable importance using SHAP and cross-validation]]></title>
            <link>https://lucasramos-34338.medium.com/visualizing-variable-importance-using-shap-and-cross-validation-bd5075e9063a?source=rss-282d1abf0b29------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd5075e9063a</guid>
            <category><![CDATA[modeling]]></category>
            <category><![CDATA[shap]]></category>
            <category><![CDATA[feature-importance]]></category>
            <category><![CDATA[interpretability]]></category>
            <category><![CDATA[model-interpretation]]></category>
            <dc:creator><![CDATA[Lucas Ramos]]></dc:creator>
            <pubDate>Fri, 26 Jun 2020 11:27:49 GMT</pubDate>
            <atom:updated>2021-12-12T18:40:51.283Z</atom:updated>
            <content:encoded><![CDATA[<p>We have all seen how amazing and insightful SHAP explanations can be. Though, when writing a scientific paper or any other situation that requires one to run multiple cross-validation iterations, the number of images start to stack and it becomes difficult to report results.</p><p>If you’re just looking for an answer you can skip this intro and go straight to the section below (named The Real Deal) where I explain the code and approach.</p><p>Colab code can be found here:</p><p><a href="https://colab.research.google.com/drive/1icEsSxTLtDn9_3PhQkjxVY7j25jcqRhM?usp=sharing">Google Colaboratory</a></p><p>Machine learning models have been for long coined as a “black box”, due to their complex nature. The more complex and robust models become, the more difficult it becomes to obtain meaningful variable importance.</p><p>Of course, the field of model visualization and interpretation is as active as the AI field itself. We have all seen how powerful and helpful tools such as SHAP and LIME are at visualizing and explaining machine learning models for both image and tabular data.</p><p>LIME (Local Interpretable Model-Agnostic Explanations) became popular around 2016 when the paper <em>“Why Should I Trust You?”: Explaining the Predictions of Any Classifier</em>, was published. It showed particularly interesting cases, like the one below, where a husky was classified as a wolf due to the snow in the background (that’s how they found the network was actually learning the difference between dogs and wolves).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*re6GiLh82LFO1KQDYbm1Fg.png" /></figure><p>Some of the issues of explainers such as LIME, is that they often train a new model to explain the one you created, and the explanations are “sample-wise” which can be hard to condense and make any conclusions.</p><figure><img alt="Example of sample-wise prediction from LIME" src="https://cdn-images-1.medium.com/max/364/1*4H0YoVx9T2br8Cpcwbp72Q.png" /><figcaption>Example of sample-wise prediction from LIME</figcaption></figure><p>More recently, SHAP condensed knowledge from multiple explainers, including LIME, and created their own visualization toolkit. While LIME is “model -agnostic”, which means it can be used with any classifier, SHAP has different modules for each “family of classifiers”, like tree based model, linear models, etc.</p><p>Some improvements from SHAP over many others, are the following:</p><p>1- Global explanations for the model/dataset instead of just sample-wise;</p><p>2- Ways to evaluate local feature interactions</p><p>3- Explanation/visualizations are generated based on your trained model instead of training a new one.</p><p>An example of one of my favorite SHAP visualization plots is shown below. The summary plot shows not only variable importance (from top to bottom), but also shows how the feature values (using the color spectrum from blue to red) affect the label prediction. Here, high values of alcohol influence a positive label prediction(SHAP values above zero). More about this can be found <a href="https://towardsdatascience.com/explain-your-model-with-the-shap-values-bc36aac4de3d">HERE</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*Buz3kvYoMV2qcokROrd9zw.png" /><figcaption>Shap summary plot</figcaption></figure><p>It is relatively easy to create such plot. Below we create this plot for the breast_cancer dataset.</p><pre>from sklearn.model_selection import train_test_split</pre><pre>from sklearn.datasets import load_breast_cancer</pre><pre>from sklearn.ensemble import RandomForestClassifier</pre><pre>import shap</pre><pre>import pandas as pd</pre><pre>#loading and preparing the data</pre><pre>iris = load_breast_cancer()</pre><pre>X = iris.data</pre><pre>y = iris.target</pre><pre>columns = iris.feature_names</pre><pre>X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)</pre><pre>X_train = pd.DataFrame(X_train,columns=columns)</pre><pre>X_test = pd.DataFrame(X_test,columns=columns)</pre><pre>#training model</pre><pre>clf = RandomForestClassifier(random_state=0)</pre><pre>clf.fit(X_train, y_train)</pre><pre>#explaining model</pre><pre>explainer = shap.TreeExplainer(clf)</pre><pre>shap_values = explainer.shap_values(X_test)</pre><pre>#shap_values of 1 for positive label</pre><pre>shap.summary_plot(shap_values[1], X_test)</pre><p>Which results in:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/617/1*SXeVEUjBfjI8w3I1J55ifg.png" /></figure><h3><strong>The Real Deal</strong></h3><p>Now, that works fine for 1 iteration, but everyone knows it is good practice to run multiple cross-validation iterations when developing your models to reduce the risk of a luck train/test split.</p><p>To generate a plot to summarize all cross-validation iterations I suggest you use k-fold cross-validation. This way we are able to compute shap values for each test set fold (see code below).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/95b3ad4524d19897bf95e9ebec7aecfb/href">https://medium.com/media/95b3ad4524d19897bf95e9ebec7aecfb/href</a></iframe><p>After running the code for all folds, we get a plot with the summary of feature importance for the whole experiment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*whot0iqOxDYSLiodMIsdhw.jpeg" /><figcaption>Summary plot for a single fold on the left and for all folds together on the right.</figcaption></figure><p>Despite the results not changing a lot (I guess the conclusion regarding the values would still be the same), there’s a clear difference between plots, especially in the density and spread of values. We have also to keep in mind that this is a relatively easy dataset, and for more complex problems, reporting results for multiple iterations might actually change your final conclusions.</p><p>Finally, this might not be the best way to do it, so I’m curious and open for discussing your ideas to handle it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd5075e9063a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>