<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lucas Estevam on Medium]]></title>
        <description><![CDATA[Stories by Lucas Estevam on Medium]]></description>
        <link>https://medium.com/@estevamlucas?source=rss-2f471de6ea60------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*-gRKXKrrxZoeTYyVjfKmnQ.jpeg</url>
            <title>Stories by Lucas Estevam on Medium</title>
            <link>https://medium.com/@estevamlucas?source=rss-2f471de6ea60------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 00:07:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@estevamlucas/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Introducing fklearn: Nubank’s machine learning library (Part II)]]></title>
            <link>https://medium.com/building-nubank/introducing-fklearn-nubanks-machine-learning-library-part-ii-c67176fe3cc8?source=rss-2f471de6ea60------2</link>
            <guid isPermaLink="false">https://medium.com/p/c67176fe3cc8</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[functional-programming]]></category>
            <dc:creator><![CDATA[Lucas Estevam]]></dc:creator>
            <pubDate>Wed, 04 Dec 2019 19:31:01 GMT</pubDate>
            <atom:updated>2020-08-21T21:24:28.094Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>This article has been migrated to our new blog. Find it at <a href="https://building.nubank.com.br/introducing-fklearn-nubanks-machine-learning-library-part-ii/">https://building.nubank.com.br/introducing-fklearn-nubanks-machine-learning-library-part-ii/</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Sy3H-n9O2hd101tb.png" /></figure><p><em>Read Part I of this series </em><a href="https://building.nubank.com.br/introducing-fklearn-nubanks-machine-learning-library-part-i-2/"><em>here</em></a><em>.</em></p><p>In the first part of this series, we talked about fklearn’s principles, and how it can be useful to build machine learning pipelines using pure functions. However, the biggest benefits of using fklearn come when doing analysis and validation of models, as we’ll see next.</p><h3><strong>Extensible Validation</strong></h3><p>For models that go into production, validation often requires a lot more than just finding out the value of some metric on a holdout set, or comparing a metric to a benchmark. We often want to answer questions such as:</p><ul><li>How do we expect the model to perform over time, say 6 months after it has been trained?</li><li>Would more data help improve performance?</li><li>How often should I retrain?</li><li>Is it better to use more data or only recent data?</li><li>Does it perform equally well across user segments?</li></ul><p>These may all sound like very different questions, but they can be answered by the same generalized model validation algorithm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sB6L5mXVh0coFDfeCaRPWQ.png" /><figcaption><em>Fklearn’s generalized validation algorithm: Data is first split into chunks, a training function is applied to some subset of data, predictions are made on another subset, then an evaluation function is applied to the predictions.</em></figcaption></figure><p>In fklearn this algorithm is implemented by the <em>validator</em> function:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3953884bf9a4a1245e0991c60f0bf3a6/href">https://medium.com/media/3953884bf9a4a1245e0991c60f0bf3a6/href</a></iframe><ul><li>The <em>train_fn</em> is a learner function, exactly as we’ve defined in the last blog post.</li><li>The <em>split_fn</em> controls how the data will be split into training and testing sets. They receive DataFrames and return lists of training and testing indexes.</li><li>The <em>eval_fn</em> defines which metrics will be computed on the testing sets. More on evaluation functions later.</li></ul><p>The validator returns a set of <strong>logs </strong>containing the results of all evaluations made on the testing sets. Analysis can then be done by extracting the data from these logs (fklearn provides helper functions for doing so).</p><p><strong>Simply by swapping the splitting and evaluation functions, we can simulate and evaluate many different real-life scenarios</strong>, which helps us answer those questions. Let’s look at an example, using the learner we defined in Part 1, in which we try to answer these two questions:</p><ol><li>We have two different user segments, A and B. How different will model performance be between the two segments?</li><li>We expect this model to keep running for a few months after it is trained. How much do we expect model performance to degrade 3, 6 or 9 months after training?</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d30c7f44d6876b13b5cf5d4d8eb0446f/href">https://medium.com/media/d30c7f44d6876b13b5cf5d4d8eb0446f/href</a></iframe><p>Let’s go over this example in more detail.</p><h3><strong>Splitting Functions</strong></h3><p>In the example above, after we define the learner, we define two splitting functions:</p><ol><li>A <strong>K-fold cross-validation splitting function</strong>. It splits data randomly into k separate sets and is used for performing standard cross-validation. Cross-validation is useful for getting a basic estimate of model generalization performance, within the same time period as it is trained.</li><li>A <strong>stability validation splitting function</strong>. It works by splitting data based on the <em>time_column</em> parameter. The model will be trained using data up to <em>training_time_limit</em>, and<em> </em>validated on data split monthly after then. Stability evaluation is useful for estimating how the model will generalize in time, beyond the training period.</li></ol><p>Fklearn comes pre-packaged with many splitting functions for common use cases (you can find a full list and descriptions of when they are useful <a href="https://fklearn.readthedocs.io/en/latest/api.html#splitters-fklearn-validation-splitters">here</a>). Most splitters are designed to simulate real-life situations, where usually models are trained on data from one time period but applied in the future. Simple cross-validation is frequently insufficient for real models.</p><h3><strong>Customizable Evaluation</strong></h3><p>This comes up often in model evaluation: single, global metrics often don’t tell the full story. We might want to isolate the effects of model rank ordering and model calibration, or look into performance in specific subgroups of our population. We might also be interested in the evolution of a certain metric, rather than just a point in time estimate.</p><p>For this, we need to simultaneously evaluate multiple metrics, split across dimensions (e.g. time, customer segment). Fklearn enables this by allowing us to define “evaluation trees”, combining individual evaluation functions. Looking back at our example, here’s how we defined the evaluation function:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba1e77e2f99faf46489212662d43aed0/href">https://medium.com/media/ba1e77e2f99faf46489212662d43aed0/href</a></iframe><p>This code snippet leads to the evaluation tree shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HaCJlESxsPRGW3vb" /><figcaption>Tree representation of the evaluation function defined above.</figcaption></figure><p>Once <em>final_eval_fn</em> is applied to data, the entire tree of evaluators runs, and all results are returned in the log. This means both r2 and Spearman correlation will first be computed for everyone, then separately for each user segment. These “evaluation trees” can be very powerful, allowing Data Scientists to automate recurring analysis.</p><h3>Analyzing results</h3><p>As you may have noticed, most operations in fklearn return <strong>logs</strong>. These logs concentrate valuable information, be it model parameters, dataset metadata or validation results. Fklearn can be very verbose with logging, as we’ve found ourselves regretting not saving information often.</p><p>Fklearn also provides helpful functions to extract data from these logs (they can become quite large), and it is easy to create evaluation plots using only the logs. This allows us to build generic evaluation code that receives logs from training runs and generates dashboards with model performance, speeding up the iteration process. Here’s an example of extracting data from the logs:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ea967a84a9c43cb471bdaa4311b71f9/href">https://medium.com/media/7ea967a84a9c43cb471bdaa4311b71f9/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6BykKAeCmmAsuLgdM-XCvg.png" /><figcaption>Sample stability curve plot, after extracting the data from the logs. It shows model performance (spearman correlation between prediction and target) over time, split by segment. It would also be possible to plot R2 from the same data.</figcaption></figure><h3>Functional bliss</h3><p>As a final note on validation, note that the <em>learner </em>we defined will be used again and again inside these validator calls, training several models. We get the peace of mind of knowing it is a pure function (so nothing that happens inside validation can change our model definition), and knowing that when validating our model, all the steps in our pipeline are being applied consistently to all the data folds. <strong>Ultimately, this means that our final model, which goes to production, accurately matches the models going through all these validation scenarios.</strong></p><p>The same goes for <strong>tuning </strong>or <strong>feature selection</strong>. For both, fklearn provides functions that are similar in spirit to the validator: you define how to split data, how to evaluate model performance, and re-utilize your learner function.</p><h3><strong>What’s next?</strong></h3><p>This post concludes our brief introduction to fklearn. For more examples of fklearn’s model validation capabilities and other powerful tools, check the documentation <a href="https://fklearn.readthedocs.io/en/latest/examples/regression.html#Evaluate">here</a>. We also hope this gets you excited about <a href="https://github.com/nubank/fklearn">trying fklearn yourself</a>.</p><p><strong>Interested in Data Science and the exciting products being built at Nubank? </strong><a href="https://boards.greenhouse.io/nubank"><strong>We’re hiring</strong></a><strong>!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c67176fe3cc8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/building-nubank/introducing-fklearn-nubanks-machine-learning-library-part-ii-c67176fe3cc8">Introducing fklearn: Nubank’s machine learning library (Part II)</a> was originally published in <a href="https://medium.com/building-nubank">Building Nubank</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing fklearn: Nubank’s machine learning library (Part I)]]></title>
            <link>https://medium.com/building-nubank/introducing-fklearn-nubanks-machine-learning-library-part-i-2a1c781035d0?source=rss-2f471de6ea60------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a1c781035d0</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[functional-programming]]></category>
            <dc:creator><![CDATA[Lucas Estevam]]></dc:creator>
            <pubDate>Tue, 23 Apr 2019 21:01:01 GMT</pubDate>
            <atom:updated>2020-08-21T21:27:48.249Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>This article has been migrated to our new blog. Find it at <a href="https://building.nubank.com.br/introducing-fklearn-nubanks-machine-learning-library-part-i-2/">https://building.nubank.com.br/introducing-fklearn-nubanks-machine-learning-library-part-i-2/</a></blockquote><p><em>Read Part II of this story </em><a href="https://building.nubank.com.br/introducing-fklearn-nubanks-machine-learning-library-part-ii/"><em>here</em></a><em>.</em></p><p>Nubank has just open-sourced <a href="https://github.com/nubank/fklearn">fklearn</a>, our machine learning python library!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pyfI8jM6PowBKxq0o7-Zag.png" /></figure><p>At Nubank we rely heavily on machine learning to make scalable data-driven decisions. While there are many other ML libraries out there (we use <a href="https://xgboost.readthedocs.io/en/latest/">Xgboost</a>, <a href="https://lightgbm.readthedocs.io/en/latest/">LGBM</a>, and <a href="https://scikit-learn.org/stable/">ScikitLearn</a> extensively for example), we felt the need for a higher level abstraction that would help us more easily apply these libraries to the problems we face. Fklearn effectively wraps these libraries into a format that makes their use in production more effective.</p><p>Fklearn currently powers a large set of machine learning models at Nubank, solving problems ranging from credit scoring to automated customer support chat responses. We built it with the following goals in mind:</p><ol><li>Validation should reflect real-life situations</li><li>Production models should match validated models</li><li>Models should be production-ready with few extra steps</li><li>Reproducibility and in-depth analysis of model results should be easy to achieve</li></ol><p>Early on we decided that functional programming would be a powerful ally in trying to achieve these goals.</p><h3>F is for Functional</h3><p>Here at Nubank we’re big <a href="https://medium.com/building-nubank/demystifying-functional-programming-in-a-real-company-e954a2591504">fans of functional programming</a>, and that isn’t limited to the Engineering chapter. But how does Functional programming help Data Scientists?</p><p>Machine Learning is frequently done by using object-oriented python code, and that’s the way we used to do it at Nubank as well. Back then, the process of building machine learning models and putting them into production was tiresome and often full of bugs. We’d deploy a model only to find that predictions made in production didn’t match the ones seen during validation. What’s more, validation was often impossible to reproduce, frequently being done in stateful Jupyter Notebooks.</p><p>Functional programming helps fix these issues by:</p><ul><li>Making it easy to build pipelines where the data transformations that happen during training match the models in production.</li><li>Allowing for safer iteration in interactive environments (e.g. Jupyter Notebooks), preventing mistakes caused by stateful code and making research more reproducible.</li><li>Allowing us to write very generic validation, tuning and feature selection code that works across model types and applications, making us more efficient overall.</li></ul><p>Let’s go through an example to see how functional programming does this in practice. Let’s say we’re trying to predict how much someone will spend on their credit card based on two variables: monthly income and previous bill amount. As the output of this model will be used for sensitive decision making, we’d like to make sure it is robust to outliers in the input variables, which is why we decide to:</p><ol><li>Cap monthly income to 50,000, since income is self-reported and sometimes exaggerated.</li><li>Limit the output range of the model to the [0, 20,000] interval.</li></ol><p>And then use a simple linear regression model. Here’s what the code looks like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/93405438d60eb37c8bad473cdb8fc072/href">https://medium.com/media/93405438d60eb37c8bad473cdb8fc072/href</a></iframe><p>Don’t be alarmed! We’ll go through the code step by step explaining some important fklearn concepts.</p><h3>Learner functions</h3><p>While in scikit-learn the main abstraction for a model is a class with methods <em>fit</em> and <em>transform</em>, in fklearn we use what we call a <strong><em>learner function.</em></strong><em> </em>A learner function takes in some training data (plus other parameters), learns something from it and returns three things: a <strong><em>prediction function, </em></strong><em>the</em><strong><em> transformed training data, </em></strong><em>and a </em><strong><em>log.</em> </strong>The first three lines of our example are initializing three learner functions: <em>capper, linear_regression_learner, </em>and <em>prediction_ranger.</em></p><p>To better illustrate, here’s a simplified definition of the <em>linear_regression_learner:</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/25f91a4239592f3c42e24b6de62d7584/href">https://medium.com/media/25f91a4239592f3c42e24b6de62d7584/href</a></iframe><p>Notice the use of type hints! They help make functional programming in python less awkward, along with the immensely useful <a href="https://toolz.readthedocs.io/">toolz</a> library.</p><p>As we mentioned, a <strong>learner function</strong> returns three things (a <em>function</em>, a <em>DataFrame</em>, and a <em>dictionary</em>), as described by the <strong>LearnerReturnType </strong>definition:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/47c9fb171a0ecdc892931248d80f7f30/href">https://medium.com/media/47c9fb171a0ecdc892931248d80f7f30/href</a></iframe><ul><li>The <strong>prediction function</strong> always has the same signature: it takes in a DataFrame and returns a DataFrame (we use <a href="https://pandas.pydata.org/">Pandas</a>). It should be able to take in any new DataFrame (as long as it contains the required columns) and transform it (it is equivalent to the <em>transform </em>method of a scikit-learn object). In this case, the prediction function simply creates a new column with the predictions of the linear regression model that was trained.</li><li>The <strong>transformed training data</strong> is usually just the prediction function applied to the training data. It is useful when you want predictions on your training set, or for building pipelines, as we’ll see later.</li><li>The <strong>log </strong>is a dictionary, and can include any information that is relevant for inspecting or debugging the learner (e.g. what features were used, how many samples there were in the training set, feature importance or coefficients).</li></ul><p><strong><em>Learner functions </em></strong>show some common functional programming properties:</p><ul><li>They are <a href="https://en.wikipedia.org/wiki/Pure_function">pure functions</a>, meaning they always return the same result given the same input, and they have no side-effects. In practice, this means you can call the learner as many times as you want without worrying about getting inconsistent results. This is not always the case when calling <em>fit</em> on a scikit-learn object for example, as objects may mutate.</li><li>They are <a href="https://en.wikipedia.org/wiki/Higher-order_function"><em>higher order functions</em></a><em>, </em>as they return another function (the prediction function). As the prediction function is defined within the learner itself, it can access variables in the learner function’s scope via its <a href="https://en.wikipedia.org/wiki/Closure_(computer_programming)"><em>closure</em></a>.</li><li>By having consistent signatures, learner functions (and prediction functions) are <a href="https://en.wikipedia.org/wiki/Function_composition_(computer_science)"><em>composable</em></a>. This means building entire pipelines out of them is straightforward, as we’ll see soon.</li><li>They are <a href="https://stackoverflow.com/questions/36314/what-is-currying"><em>curriable</em></a>, meaning you can initialize them in steps, passing just a few arguments at a time (this is what’s actually happening in the first three lines of our example). This will be useful when defining pipelines, and applying a single model to different datasets while getting consistent results.</li></ul><p>It may take some time to wrap your head around all this, but don’t worry, you don’t need to be an expert in functional programming to use fklearn effectively. The key is understanding that models (and other data transformations) can be defined as functions following the <em>learner</em><strong><em> </em></strong>abstraction.</p><h3>Pipelines</h3><p>Machine Learning models rarely exist on their own however. By focusing only on the model, Data Scientists tend to forget what transformations the data is going through before and after the ML part. These transformations often need to be exactly the same when training and deploying models, and Data Scientists might try to manually recreate their training pre- and post- processing steps in production, which leads to code duplication that is hard to maintain.</p><p>Learner functions are <em>composable</em>, meaning two or more learners combined can be seen as just a new, more complex learner. This means that no matter how many steps you have in your pipeline, your final model will behave just the same as a single one, and making predictions is as simple as calling the final prediction function on new data. Having all the steps in your modeling pipeline contained in a single, pure function also helps with validation and tuning, as we can pass it around to other functions without fear of side effects.</p><p>In our example, our pipeline consists of three steps: <strong>capping the income variable</strong>, <strong>running the regression</strong> then <strong>constraining the regression output</strong> to the <em>[0, 20000] </em>range. After each learner is initialized, we build the pipeline and apply it to the training set using these two lines of code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/10becb71ab0187adea02eee105928413/href">https://medium.com/media/10becb71ab0187adea02eee105928413/href</a></iframe><p>The <em>learner</em> variable now contains the pipeline resulting from composing the three learner functions, and is applied to the training data to yield the final prediction function. This function will apply all the equivalent steps in the pipeline to the test data, as the image below illustrates:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0HDv-ShpifDbpPXZt-6j_g.png" /><figcaption>Example of how data flows through a pipeline when training, and through a prediction function when predicting. The prediction function itself is returned by the pipeline; it is the composition of the three prediction functions generated by each learner when the pipeline was first called on the training data. The logs are a combination of the logs coming from all learner functions in the pipeline.</figcaption></figure><h3>What’s next?</h3><p>We’ve seen how models and data transformation steps can be written as <em>learner functions</em>, and how functional pipelines in fklearn help us guarantee that transformations done during training and validation match those done in production.</p><p>In <a href="https://medium.com/building-nubank/introducing-fklearn-nubanks-machine-learning-library-part-ii-c67176fe3cc8">Part II</a> of this blog post we talk about model validation and analysis, and the tools fklearn provides to make those steps more effective.</p><p>In the meantime, we invite you to try <a href="https://github.com/nubank/fklearn">fklearn</a> for yourself! We don’t expect fklearn to replace the current standards in ML, but we hope it starts interesting conversations about the benefits of functional programming for Machine Learning.</p><p><strong>Interested in Data Science and the exciting products being built at Nubank? </strong><a href="https://nubank.workable.com/"><strong>We’re hiring</strong></a><strong>!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a1c781035d0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/building-nubank/introducing-fklearn-nubanks-machine-learning-library-part-i-2a1c781035d0">Introducing fklearn: Nubank’s machine learning library (Part I)</a> was originally published in <a href="https://medium.com/building-nubank">Building Nubank</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>