<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tom Steenbergen on Medium]]></title>
        <description><![CDATA[Stories by Tom Steenbergen on Medium]]></description>
        <link>https://medium.com/@tom_steenbergen?source=rss-c619cbb9ce18------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*2foM2nvCGMdmQph9mcuYWQ.png</url>
            <title>Stories by Tom Steenbergen on Medium</title>
            <link>https://medium.com/@tom_steenbergen?source=rss-c619cbb9ce18------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 18:13:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tom_steenbergen/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[MLOps Principles to build Picnic’s Data Science Platform]]></title>
            <link>https://medium.com/picnic-engineering/mlops-principles-to-build-picnics-data-science-platform-851cbe2e8045?source=rss-c619cbb9ce18------2</link>
            <guid isPermaLink="false">https://medium.com/p/851cbe2e8045</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[platform]]></category>
            <dc:creator><![CDATA[Tom Steenbergen]]></dc:creator>
            <pubDate>Wed, 21 Sep 2022 06:22:14 GMT</pubDate>
            <atom:updated>2022-09-21T06:22:14.066Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Az5eVVJrh1sa3Uv4xfEoAQ.png" /></figure><p>Here at Picnic, <a href="https://blog.picnic.nl/search?q=data">we love data</a>. Over the last years, Picnic has grown into a data-driven online supermarket that is active in three countries. By leveraging data and algorithms, we have been able to support the company’s growth while maintaining high service levels. Besides numerous demand forecasting models, we have for example built machine learning models to <a href="https://blog.picnic.nl/machine-learning-and-customer-success-automated-feedback-classification-cb87ec8098fb">improve our customer service</a> and <a href="https://medium.com/picnic-engineering/the-trade-off-between-efficiency-and-being-on-time-optimizing-drop-times-using-machine-learning-d3f6fb1b0f31">increase the efficiency of our trips</a>. The number of machine learning (ML) solutions within Picnic has been steadily growing and these solutions have been bringing a lot of business value. That said, this of course doesn’t mean that we are done. How can we bring our data science game to the next level? How can we go from tens of models to hundreds of models in order to make an impact in every part of the business? In this blog we will dive into our plans to further scale data science at Picnic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FM2UlWiRwX5uSOupZT_IeQ.png" /></figure><h3>Data Science at Picnic</h3><p>We have been building and maintaining our ML systems as a central <a href="https://blog.picnic.nl/data-science-as-a-product-f383dead5aa4">Data Science team</a> with a lot of emphasis on software engineering skills. So far, this has been working great for Picnic. We skipped the Jupyter Notebook phase. From the start, we aligned our tech stack with the rest of Picnic’s Tech team and adopted the same DevOps practices. On top of that, the Data Science team is able to tackle the entire lifecycle from ideation and scoping to packaging the model and deploying it on Kubernetes. We don’t train models and throw them over a fence, but instead, we foster a strong “you train it, you own it”-mentality.</p><p>We have been able to build ML solutions that integrate well with other systems and processes within Picnic, while simultaneously standardizing some of the common components in an ML system as we go. For example, we have built an in-house model registry to simplify model storage and retrieval, and a prediction archive to standardize the logging of predictions for long-term storage and analysis.</p><p>However, in order to continue supporting the growth of the company, we also recognize we have to become better at scaling machine learning solutions in the coming few years. The number of use-cases for ML increase at a faster rate than that we can grow our team. Additionally, the more we build, the more time we spend on maintaining existing solutions. Hence, we need to increase the number of models a single data scientist can build and manage, as well as increase the number of data science practitioners in our company.</p><h3><strong>MLOps to the rescue</strong></h3><p>To further scale and accelerate data science within Picnic, we have initiated a focused effort on Machine Learning Operations (MLOps). MLOps is the set of practices with the goal of reliably building, testing, deploying and maintaining machine learning solutions. The benefit of MLOps is that it smoothens and shortens the machine learning lifecycle. For example, through automating model retraining and deployment, creating reproducible experiments, managing and tracking model artifacts, and more.</p><p>There is quite a bit of overlap between DevOps and MLOps, as many blog posts already point out. Many of these practices and processes are already adopted by Picnic’s Data Science team. Yet, there are definitely some unique aspects to machine learning solutions that are not covered yet by our current tooling and infrastructure. For example, actionable alerting on model drift and easy model deployment on GPUs.</p><p>As mentioned earlier, we also want to increase the number of data science practitioners. In order to enable more people at Picnic to build high quality ML systems, we need to solidify MLOps practices more explicitly through standardization. We also require more automation in the tooling and infrastructure used across the machine learning lifecycle. Last, but certainly not least, the complexities of the wide array of technologies used when building an ML solution should be abstracted away.</p><p>Thus, Picnic’s vision for its Data Science Platform was born. In the following sections, we will dive deeper into our vision for the platform and the principles that we are keeping in mind while building it.</p><h3><strong>Picnic’s Data Science Platform</strong></h3><p>The vision of Picnic’s Data Science Platform (DSP) is that everyone at Picnic can deliver industry-leading machine learning systems.</p><p>The users of the DSP are not just the central Data Science team. Business analysts and developers from other teams should just as easily be able to use the platform and collaborate on models in order to achieve the best solution possible.</p><p>More concretely, the DSP should provide a collection of tooling, infrastructure, and best practices to scale industry-leading ML systems at Picnic. It supports its users to reduce the time-to-market and increase the quality of their systems, while eliminating complexity.</p><p>When designing a platform that supports Picnic’s ML systems of the future, we keep in mind a set of core principles. The following eight principles align with our vision and are core to our platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*luSGYUDYVGGRishhvD93MA.png" /></figure><h4><em>Collaboration</em></h4><p>Collaboration with business and tech teams should be seamless, such that users can leverage domain knowledge and drive adoption. By promoting and facilitating collaboration, the platform supports its systems to build more accurate models and to quickly make actual business impact.</p><h4><em>Versioning</em></h4><p>All artifacts of an ML system should be versioned. Artifacts are anything that is an input or output of the system, such as data, configurations, models, etc. By baking in versioning for all artifacts, the platform enables users to fully control what is used in production.</p><h4><em>Testing</em></h4><p>Each part of an ML system should be tested. Tests should not only cover the training or inference code, but also the data and the model. By giving users the tools and practices to test properly, users can have the confidence that their changes work as intended before deployment.</p><h4><em>Reproducibility</em></h4><p>Data, models and code should be easily reproducible. By embedding reproducibility across the machine learning lifecycle, the platform enables users to translate experimental results to production performance.</p><h4><em>Continuity</em></h4><p>ML systems should be automatically and continuously updated with new data, models, and code. By automating the mundane steps, the platform allows users to focus on their craft and minimize errors caused by manual work.</p><h4><em>Scalability</em></h4><p>ML systems should scale accordingly to their load and business value. By abstracting away complexity, the platform helps users to build systems that are future-proof, without them having to be experts in software engineering or infrastructure.</p><h4><em>Monitoring</em></h4><p>Each part of an ML system should be extensively monitored. By making data, models and deployments transparent, the platform gives users actionable insights and alerts to exercise operational excellence.</p><h4><em>Governance</em></h4><p>The platform should strive for transparency and accountability of ML systems. Through best practices and sensible defaults, the platform promotes users to build only fair and ethical models.</p><h3><strong>Working towards our vision</strong></h3><p>We will tackle the building of the Data Science Platform with a product-oriented approach. We recognize that MLOps is a field that is still very much in flux. Start-ups trying to solve some new challenge within the machine learning life cycle are constantly popping up. At the same time, none of the big machine learning platform offerings is able to cater to all use cases and cover all challenges you might encounter (<a href="https://ai-infrastructure.org/ai-infrastructure-ecosystem-report-of-2022/">AI Infrastructure Alliance</a>, 2022). In other words, it is hard to find something that meets all your requirements (not just on paper, but in practice) and doesn’t become outdated within a year.</p><p>We will therefore look to iteratively expand our existing set of tooling by selecting per component the best of breed solution. Whether they are vendor solutions or built in-house. By taking a more modular approach, we aim to be independent of any platform and be flexible enough to swap out components in this rapidly changing field. This also allows us to first focus on the challenges considered most important by our users and innovate quickly.</p><p>Sounds interesting? Make sure to follow the Picnic Engineering blog and keep your eyes open for new blog posts! If you are interested in helping to build the most data-driven modern milkman, head over to <a href="https://picnic.app/careers/">https://picnic.app/careers/</a> and take a look at our vacancies!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=851cbe2e8045" width="1" height="1" alt=""><hr><p><a href="https://medium.com/picnic-engineering/mlops-principles-to-build-picnics-data-science-platform-851cbe2e8045">MLOps Principles to build Picnic’s Data Science Platform</a> was originally published in <a href="https://medium.com/picnic-engineering">Picnic Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The trade-off between efficiency and being on time: optimizing drop times using machine learning]]></title>
            <link>https://medium.com/picnic-engineering/the-trade-off-between-efficiency-and-being-on-time-optimizing-drop-times-using-machine-learning-d3f6fb1b0f31?source=rss-c619cbb9ce18------2</link>
            <guid isPermaLink="false">https://medium.com/p/d3f6fb1b0f31</guid>
            <category><![CDATA[distribution]]></category>
            <category><![CDATA[vehicle-routing-problem]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[multilayer-perceptron]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Tom Steenbergen]]></dc:creator>
            <pubDate>Tue, 10 Mar 2020 08:26:21 GMT</pubDate>
            <atom:updated>2020-03-12T07:30:11.837Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HTB9TXIzsKeC8X0rAe86-g.jpeg" /></figure><h3>Optimal drop times using machine learning</h3><h4>The trade-off between efficiency and being on time</h4><p>Picnic delivers groceries to thousands of customers every day. In order to do so both efficiently and effectively, Picnic’s distribution system needs to be as smooth as possible. One part of this system is the vehicle routing model, that determines which routes are optimal given the customers and their placed orders, amongst other factors. An important input into this model is the <em>drop time</em>: how much time we expect a delivery to take for a given customer (in the vehicle routing literature better known as the service completion time). This blog post covers why drop times are so important and how we, the Data Science team within Picnic, developed a model to better predict this amount of time.</p><h4><strong>Time is of the essence</strong></h4><p>Customers value the convenience of having their groceries delivered to their kitchen. To make the experience even more convenient, Picnic communicates a short time window on the day of the delivery in which the deliverer, or <em>runner</em>, will be at the customer’s door. This time window is only 20 minutes long and it falls within the one-hour time slot the customer chose when placing the order. In order to make sure that we can make the customer happy by delivering his/her groceries within those 20 minutes, Picnic’s distribution system needs to plan enough time so the runner can easily arrive at the customer on time.</p><p>As Picnic is growing rapidly, the number of customers that would like to have their groceries delivered by us keeps increasing. Therefore, it is key for the business to be able to deliver efficiently. In other words, there are two contrasting objectives here. On one hand, we want to have a high on-time percentage, which requires us to plan enough time for each and every customer. On the other hand, we want to increase efficiency and serve as many customers as possible within the duration of a single delivery trip made by a runner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/0*wFgzA9t4Ehly3_hB" /><figcaption><em>Gantt chart of a delivery trip</em></figcaption></figure><p>In order to make sure that we can increase efficiency while maintaining the same on-time percentage, we need to have a better estimate of how much time is ‘enough’ to deliver a given order. Visually, this means that we would like the red bars (the actual drop times) in the chart above to start right in the middle of the green bars (the planned 20-minute windows). If we plan too much time, the red bars move to the left side of the green bar. In severe cases, this would mean a runner would even have to wait before he/she can deliver as the planned time window hasn’t started yet. This is of course very inefficient and is something we would like to avoid. If we plan too little time, the red bars move to the right side of the green bars, causing each subsequent drop to be later. This could potentially cause orders to be delivered later than the planned time window, which deteriorates the customer’s experience.</p><p>In short, when our drop time estimate is rather inaccurate, we need to plan a lot more redundant time on top of our estimate as a safety buffer to maintain a high on-time percentage. When this estimate becomes more accurate, we can reduce this safety buffer, increasing the distribution system’s efficiency, without risking many late deliveries.</p><h4><strong>The road to success</strong></h4><p>We started off with some exploratory data analysis and looked for factors that affect the duration of a delivery. These factors can be roughly grouped into four areas: 1) customer, 2) order, 3) region, and 4) runner. A broad range of features was explored, from historical average drop time of a customer and the total weight of the delivery, to weather forecasts and address density. Naturally, a large and heavy order consisting of multiple bags of groceries takes a lot longer to deliver than an order containing only a couple of articles. There were also features that we would like to have, but we could not construct with the data that we have. For example, we would like to know on which floor the customer lives, as that could impact the drop time a lot. As you can imagine, delivering to a customer on the top floor of a four-story apartment building takes a lot longer than delivering to a customer with a front door on street level. For these kinds of features that we do not have, we constructed proxy variables. In this example, we used address density and the historical drop time of the customer.</p><p>Furthermore, we also had to make sure we chose a sensible loss function. Even though we have a great data warehouse that captures data from various back-end systems and third parties, sometimes the data received can still contain anomalous records. For example weird timestamps cause the recorded drop time of a delivery to be different from reality. In this case, we could luckily identify most of the anomalous entries and filter them out with some business rules. Additionally, we chose to use a loss function that handles outliers differently than the regular mean squared error loss function used for regression problems. The loss function we used was the Huber loss, a combination of the squared loss and the absolute loss. This function, like the mean squared error, penalizes large errors more severely than small errors. However, when errors exceed some threshold defined by a parameter δ, (indicated by the dotted lines in the picture below) the Huber loss uses a linear loss to avoid putting too much weight on these potentially anomalous errors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/0*rC7sI7gbli1wmjTP" /><figcaption><em>Comparison of squared, absolute, and Huber loss</em></figcaption></figure><p>Armed with an initial set of features and a sensible loss function, we started comparing various algorithms. As explained in <a href="https://blog.picnic.nl/the-art-and-science-of-data-science-80e215f92a99">this</a> other blog post our team made, we prefer simple over complex. Therefore, we looked at simple models first and compared that to the drop time calculation that was in place at that time. A simple linear regression model already turned out to be a lot more accurate than what was in use at the time. Comparing against this baseline model, we iteratively evaluated various other features and models.</p><h4><strong>Time for a test drive</strong></h4><p>After evaluating a number of options, we ended up with a multilayer perceptron (MLP). This model yielded the highest performance, and with all the well-developed deep learning frameworks that are available nowadays, it resulted in code that was hardly more complex. Compared to the current situation, the MLP model reduced the error by approximately 30% when evaluating historical test data. The data included all of Picnic’s <em>hubs</em> (site from which Picnic delivers to one or more cities) in the Netherlands. When these results translate to practice as well, it would mean that we could greatly cut down on the safety buffer.</p><p>In order to test how the new model would perform in reality, we decided to do a quick pilot in one of Picnic’s hubs. For a period of one week, we planned all deliveries using drop times calculated by the MLP model. After this week we found that efficiency, measured in the number of deliveries per trip, increased for this particular hub by roughly 20%. The on-time performance only decreased by approximately 2 percentage points. A small decrease, but using a goodness-of-fit test found to be statistically significant. The decrease can be owed to unexpected events during the trip, such as roadblocks or traffic jams. When you don’t have enough safety buffer on top of your drop time estimate, these unexpected events cause some orders to be delivered late. Overall, the test was a huge success and we made plans to roll out the model throughout the Netherlands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/995/1*zsO8KtHV_NOoOEnG0J5lwA.png" /><figcaption>Efficiency of the new model during the pilot, compared to the old drop time predictions</figcaption></figure><p>We started off with a couple of hubs and gradually rolled out to more and more cities. Instead of immediately starting with the new model’s predictions, we added a safety buffer on top of the MLP’s predictions because of the decrease in on-time performance we noticed during the pilot. We then steadily decreased this safety buffer over the weeks following the roll-out so we would increase the efficiency of the system, while keeping a close eye on the on-time performance. This set-up gave us the tools to play around with the trade-off between efficiency and on-time performance for each individual hub. Fast-forward a couple of months and we now have the new model running in production for each hub in both the Netherlands and Germany.</p><h4><strong>In for a challenge?</strong></h4><p>Are you interested in such challenges and do you want to contribute to Picnic’s growth as a data scientist? Then follow <a href="https://join.picnic.app/job/experienced-data-scientist-1497370">this</a> link and get in touch!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d3f6fb1b0f31" width="1" height="1" alt=""><hr><p><a href="https://medium.com/picnic-engineering/the-trade-off-between-efficiency-and-being-on-time-optimizing-drop-times-using-machine-learning-d3f6fb1b0f31">The trade-off between efficiency and being on time: optimizing drop times using machine learning</a> was originally published in <a href="https://medium.com/picnic-engineering">Picnic Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>