<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Chadchapol (Jemmy) Vittavutkarnvej on Medium]]></title>
        <description><![CDATA[Stories by Chadchapol (Jemmy) Vittavutkarnvej on Medium]]></description>
        <link>https://medium.com/@vittavutkarnvej.c?source=rss-7b585727fa57------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*260qrwK93ahz_AmfX4Fpcg.png</url>
            <title>Stories by Chadchapol (Jemmy) Vittavutkarnvej on Medium</title>
            <link>https://medium.com/@vittavutkarnvej.c?source=rss-7b585727fa57------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 05:54:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vittavutkarnvej.c/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[5 considerations before writing integration tests for serverless applications]]></title>
            <link>https://medium.com/i-love-my-local-farmer-engineering-blog/5-considerations-before-writing-integration-tests-for-serverless-applications-ea77ff62fd1c?source=rss-7b585727fa57------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea77ff62fd1c</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Chadchapol (Jemmy) Vittavutkarnvej]]></dc:creator>
            <pubDate>Tue, 21 Sep 2021 14:58:19 GMT</pubDate>
            <atom:updated>2021-09-21T14:58:19.327Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Early decisions to minimize your future regret</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*lqbZbnntfWnyXxoYo5NZDQ.png" /><figcaption>A small bird trying to expand the middle part of a pyramid in the cloud</figcaption></figure><p>While similar to unit tests, writing integration tests require many decisions outside of code itself. Today, I would like to share what we regret we wish we had considered before writing the tests.</p><ol><li>Everyone has different definitions of “integration testing”.</li><li>Different challenges compared to unit testing</li><li>Making trade-off decisions regarding dependencies</li><li>What usually breaks in serverless applications</li><li>Whether to emulating AWS services locally or not</li></ol><p>This post is the 3rd part of testing serverless applications series. <a href="https://medium.com/i-love-my-local-farmer-engineering-blog/testing-serverless-applications-1-unit-testing-aws-lambda-functions-in-java-80b7731ef706">Part 1</a> and <a href="https://medium.com/i-love-my-local-farmer-engineering-blog/testing-serverless-applications-2-applying-ports-and-adapters-architecture-e2cf69a41cd6">part 2</a> covered unit tests and general testing design. You don’t need to read the first two parts to understand this post.</p><h3>1. Everyone has different definitions of “integration testing”.</h3><p>Let’s take a look at a simple 3-tier architecture below. For end-to-end testing, the tests will exercise actual components from all 3 tiers (from UI to database). That’s straight-forward. What about integration testing?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/415/1*fpzslxLFMbpQdXIl_ZtAng.png" /><figcaption>Possible scope of integration testing</figcaption></figure><p>Every team I worked with always had different ideas on what the scope of integration testing constitutes. From the picture above, is integration testing number 1, 2, or 3? Let’s take a peek at Wikipedia.</p><blockquote><em>Integration testing is the phase in software testing in which </em><strong><em>individual software modules are combined and tested as a group</em></strong><em>… — Wikipedia</em></blockquote><p>Based on the definition of “modules”, all three options could be considered integration testing. At first glance, testing only the UI or API backend does not seem to be an integration test. However, our frontend engineers use the term “integration test” to describe their frontend tests which verify that their modules work together (e.g. React components, Redux store and reducers, etc.). Among frontend folks, there are different philosophies on where they should start to mock things out. The same goes for when we test just the API backend in number 2. Our API consists of several classes, we can test handlers and services together but mock the database out.</p><p>Number 3 includes only API backend and actual database. Some engineers call it API testing. And this is the type of integration tests we are going to focus in this post.</p><p>In serverless architecture, it is more common to have message queues or have asynchronous communication for scalability. The integration tests should have a clear boundary on which services to start and end. In some systems, we have multiple integration tests for different part of the systems.</p><h3>2. Different challenges compared to unit testing</h3><p>Compared to unit testing, the challenges faced in integration testing are more than just class design:</p><ol><li>They run slower due to network call (and cold start)</li><li>They are prone to flakiness. For instance, the connection between the API backend and the database may have temporary issues from time to time, or latency of each call may be different depending on network traffic.</li><li>Database tests will need data for read or list operations. We need to seed and clean them properly before running the tests.</li><li>One test can interfere or conflict with another when running in parallel. While a test for listing all items is running, another test can create a new item and fail the listing test. It is also possible that two tests modify the same item and both fail. In addition, we oftentimes (unintentionally) introduce dependencies between test cases. (e.g. running the listing test after the create test)</li><li>APIs require authentication and authorization, and so the test must have a way to obtain test credentials with right permission.</li></ol><p>These challenges mean that integration testing requires much more maintenance. It also takes a longer time to cover all possible cases (compared to unit tests). The whole test suit can take several minutes to run and slow down development cycle. When it takes longer than a minute, I notice that our CI/CD breaks more often as people tend to forget to run integration tests locally before pushing. As a result, we tried to speed things up by running the tests in parallel and discovered the dependencies among test cases.</p><p>Some people recommend having 1–2 success and failure scenarios for high-level testing. As tools and practices for testing have gained more maturity in the past few years. I find that it is usually feasible to have more than that. Our rule of thumb is to use integration tests to catch what unit tests cannot cover (or that they can, but only in a very brittle manner).</p><p>On the positive side, API specification doesn’t change often. As a result, integration tests are typically not brittle and most changes to the tests do not break the API contract. So well-writen integration tests usually last with the team for a long time. Also, the level of confidence we gain from integration tests is much higher than unit tests. It checks not only the code correctness, but also the integration points between components.</p><h3>3. Making trade-off decisions regarding dependencies</h3><p>Service-oriented (or microservice) architecture makes integration testing more complicated. Our API backend (or even UI) could make calls to multiple services. And this brings up several questions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*sUoFiUnfupoe9ZXvq6414A.png" /><figcaption>A single service can call many other dependencies</figcaption></figure><ol><li>Should we emulate these services (or use a mock API server) with some canned responses, or call the actual services?</li><li>If we call actual services, should we call their development, acceptance, or production environments.</li><li>How do we ensure that those environments are stable when we are running our tests in CI/CD pipeline?</li><li>How do we clean the persistent data created by our tests on their services? If we call their production, how can we avoid messing up the production data?</li><li>If we emulate any of these services, how can we be confident that there aren’t any bugs or differences in the emulator without checking the actual integration?</li></ol><p>This gets complex very quickly. Choosing one option over another is usually a trade-off. We have to balance the confidence that our overall system works (system verification), with the speed and reliability of our tests. We also have different decisions to make when it comes to internal services (e.g. those owned by our neighbour teams) and external services (e.g. those owned by SaaS providers).</p><p>Thus, it is beneficial for all of our teams to have a common strategy in regards to integration testing. Initially, each engineer made arbitrary decisions on each integration test, and it quickly became a nightmare to maintain. We had to remember how we handled each dependency and how to configure each of them to run locally.</p><p>When things get complex, it is easy for us to lose sight of overall goal. I usually go back to the 5 testing properties mentioned in<a href="https://medium.com/i-love-my-local-farmer-engineering-blog/testing-serverless-applications-1-unit-testing-aws-lambda-functions-in-java-80b7731ef706"><strong> </strong>part 1</a> as a criteria to weigh pros and cons. Basically, I ask how our decision will impact:</p><ul><li>The confidence that our whole application/service will work (e.g. If we call against acceptance environment, how can we be certain that it also works with the version on production. Can we trust that they will be backward-compatible?)</li><li>The speed of our test (e.g. Calling the actual service vs a mock)</li><li>The reliability of our test (e.g. How flaky will it be if we call non-production internal services?)</li><li>How brittle the tests will be (e.g. Will changes in other services cause massive rewrite? How often do they change? )</li><li>How easy it is to locate bugs if our test fails (e.g. If we call an external service, how do we know that it isn’t an issue on our side if the test fails? Should we also trigger our tests when they redeploy so we know who broke it?)</li></ul><h3>4. What usually breaks on serverless applications</h3><p>Everything covered above are general considerations for integration tests in any environment. However, in the serverless world, the connections we want to ensure are working properly are the integrations between API Gateway, Lambda, and RDS. Assuming that our unit tests have 100% coverage and pass, what else can still go wrong?</p><p>Take your time to think before scrolling down.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*fbutAF-xyYqKXeN5OxzkMw.png" /></figure><p>The answer lies in the integration points between services. They may be any of these:</p><ol><li>The configuration or API mapping of API Gateway to Lambda is incorrect</li><li>The parameters from API Gateway may be missing, different from what we expect, or have incorrect combinations</li><li>There is a missing permission in IAM roles or resource policies</li><li>The table schema in RDS might not work with the SQL query from Lambda (typo or schema update during deployment)</li></ol><p>These issues cannot be detected by unit tests. Integration testing provides highest value for this type of verification.</p><p>In serverless architectures, our services usually span across many different external services. The architecture above is very simplified. In real production application it’s common to have multiple API Gateways, , cache, storage (e.g. S3), orchestration service (e.g. Step Functions), special-purpose databases like ElasticSearch, and message queues and event streaming services. So it is more common to have bugs at the integration points.</p><p>In the article <a href="https://tyrrrz.me/blog/unit-testing-is-overrated">Unit testing is overrated</a>, Golub argues that the testing scope (unit → end-to-end) does not have a linear correlation with cost &amp; speed like we (unconsciously) assume when seeing the testing pyramid. Here’s the graph from his article.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*QOavxDDUx_ZWFTwURp9C5A.png" /><figcaption>Increasing scope doesn’t grow the cost and execution time linearly</figcaption></figure><p>I will say the same with the graph of scope vs benefits for serverless applications. The more services we introduce, the more likely we run into an edge case at one of the integration points. The value we gain from increasing the scope (more integration) is not linear with the value. I believe we gain much more than the cost we pay.</p><h3>5. Whether to emulate AWS services locally or not</h3><p>At I Love My Local Farmer we’ve decided that we don’t want to emulate AWS Services in acceptance environments.</p><p>In acceptance, we want always run the tests in an AWS account against actual services. Our CI/CD pipeline will deploy the infrastructure and the applications, and then start the integration tests. This is the only way to gain confidence that our infrastructure and configuration works as expected.</p><p>However, when developing we run integration tests not only in the acceptance environment, but also on our local machine. Since we cannot spin up AWS service locally, we are left with either (1) emulating the service or (2) deploying to an AWS account every time to test any changes. Since it is local environment, this is where each engineer has their preferred method.</p><p>Whether we emulate AWS services or not started a hot debate in our team. You may also have it in your team. So let me elaborate both options in details.</p><h3>5.1 Not using emulation</h3><p>The first option is not using emulation at all. In this situation, developers have their own AWS accounts to test against. They simply deploy their application on a development account, grab the API endpoint, and run the integration tests from their local machine against that endpoint. As they use actual services, this provides strong confidence that the configuration and permission on each service works correctly.</p><p>The argument against this method is that the time to deploy and execute the integration tests is slow.</p><p>But how slow are they? This differs by tools. Let’s dive into details:</p><p>SAM and CDK deploy changes through CloudFormation. The deployment will take minimum 30–60 seconds even if we change one line of code. This amount of time is more than enough to make me wander on Reddit and forget what I am doing.</p><p>To avoid this , developers package each function and upload the new code directly with a script (Here’s <a href="https://aws-blog.de/2021/04/cdk-lambda-deployment-takes-about-a-minute-how-about-sub-second-function-code-deployment.html">an example in Golang</a>.). This reduces time to just build time and a few seconds to up load. CDK will soon release a<a href="https://github.com/aws/aws-cdk/pull/15748/files"> hot swap deployment for code change</a> which uses similar concept.</p><p>For Serverless Framework, it also goes through CloudFormation (30–60 seconds). But it has an option to perform hot swap deploy ( --function or -f) so it will take only a few seconds for code change.</p><p>The Serverless Stack (SST) is a new tool that solves this problem in a different manner. They provide <a href="https://docs.serverless-stack.com/live-lambda-development">“Live Lambda Development” feature</a> which deploys a “shell” Lambda function that connects to our local machine. The shell function simply routes any requests to the code in our local machine. The machine will execute the code and return the result back to shell Lambda. Hence, we don’t need to redeploy any code change (as the shell function always does the same thing, routing requests to your local machine). The request may take a few seconds longer due to the hops. However, it doesn’t support Java which requires compilation (could be a great choice for NodeJS) and we are not testing the same environment that will be deployed on production, so some configurations may not be tested.</p><p>Here’s the comparison summary:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/1*LJJ8FC5YnFqJUnYRTR7kgw.png" /></figure><p>However, the test execution time can be very long for asynchronous applications. Imagine using multiple SQS queues to integrate Lambda asynchronously. The delay from network (50-250ms per roundtrip, depending on region) and actual service (for polling the messages) can add up quickly. If it also passes &gt;256KB data, it has to go through S3, which adds more roundtrips. As a result, the latency could jump up from a few seconds to over a minute when switching from local emulation to actual services.</p><h3>5.2 Using (whole or partial) emulation</h3><p>Folks from the other camp prefer to emulate the services locally. They substitute all (or some) AWS services with these tools:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*CWeLOO6yuZKNP5fJP9UWgg.png" /></figure><p>Emulation provides fast execution and no wait time for redeployment. In addition, the tests will be less flaky as there is no network call and you have full control of the emulator.</p><p>However, there are two main issues:</p><p>First, there is no guarantee that it will work in an actual AWS environment. Some emulators may force you to have another set of configuration for emulators, so we never really test our actual configuration (which is very important for serverless cases).</p><p>Second, the actual AWS services are complex. And the local emulation will always miss some features or fail on some edge case. Their features will always be lacking behind or some may never be implemented. We may get completely stuck if the tool doesn’t support (or has a bug for) the feature we require.</p><h3>Verdict on using emulation</h3><p>Whether a team is using emulation locally or not, we decided that we will never use emulation for integration tests in our CI/CD pipeline. It would defeat the main purpose of integration tests for our serverless architecture. The speed and stability gained from just running it once isn’t worth the lost confidence.</p><p>For local development, there are pro and cons. It is usually the trade-off between verification confidence and development speed. This can differ for each application. We let each team member find their own sweet spot.</p><p>For example, I prefer a partial option where I emulate API Gateway and Lambda but leave the other services on AWS. When it’s an asynchronous architecture, I prefer to go slower and run everything in an actual AWS environment for confidence. In this case, unit tests for individual Lambda functions are my best friend to avoid a lot of redeployments.</p><p>Also, it is possible that each developer in the same team uses different methods locally. Thus, integration testing should ideally be able to run on both emulated and actual environment. It should take only service endpoints to run and you can pass them through environment variables. This way, you can reuse the exact same test code for both CI/CD pipeline and local machines.</p><h3>Conclusion</h3><p>That was long! We touched many aspects of this topics so let me summarize our recommendations based on our own experience:</p><ol><li>Have a clear agreement with everyone in your team on testing scope. Everyone has different definitions. Be very clear on which dependencies to mock/emulate.</li><li>Have an overall agreement among internal teams on how each services integration tests will call each other.</li><li>Writing integration tests becomes more important for serverless applications. With more moving parts, it is very likely that a bug will be at an integration point, which cannot be unit tested.</li><li>Don’t emulate services in your CI/CD pipeline. Use the actual AWS environment for integration testing there.</li><li>There are pros and cons in using emulation for local development. The experience can differ greatly with different tools. Try different methods to find the best option for your application.</li></ol><p>In the next post, we will go back to coding and walk through how we code our integration testing for our serverless Java project.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea77ff62fd1c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/i-love-my-local-farmer-engineering-blog/5-considerations-before-writing-integration-tests-for-serverless-applications-ea77ff62fd1c">5 considerations before writing integration tests for serverless applications</a> was originally published in <a href="https://medium.com/i-love-my-local-farmer-engineering-blog">My Local Farmer Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Testing serverless applications #2: Applying ports and adapters architecture]]></title>
            <link>https://medium.com/i-love-my-local-farmer-engineering-blog/testing-serverless-applications-2-applying-ports-and-adapters-architecture-e2cf69a41cd6?source=rss-7b585727fa57------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2cf69a41cd6</guid>
            <category><![CDATA[unit-testing]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Chadchapol (Jemmy) Vittavutkarnvej]]></dc:creator>
            <pubDate>Tue, 31 Aug 2021 12:11:20 GMT</pubDate>
            <atom:updated>2021-08-31T12:11:20.924Z</atom:updated>
            <content:encoded><![CDATA[<p>Making your tests less brittle to lower maintenance effort</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*AXexGRbIIj4L8xW-pFHQmQ.png" /><figcaption>Pondering how to stop a bird from breaking brittle tests</figcaption></figure><p><strong><em>Disclaimer</em></strong><br><em>I Love My Local Farmer is a fictional company inspired by customer interactions with AWS Solutions Architects. Any stories told in this blog are not related to a specific customer. Similarities with any real companies, people, or situations are purely coincidental. Stories in this blog represent the views of the authors and are not endorsed by AWS.</em></p><p><em>This blog is based on the </em><a href="https://medium.com/i-love-my-local-farmer-engineering-blog/testing-serverless-applications-1-unit-testing-aws-lambda-functions-in-java-80b7731ef706"><em>previous post</em></a><em>. Please check if you haven’t read it before.</em></p><p>In the previous post, we talked about the preferred properties of tests and how to design a Lambda function that is testable. We refactored the handler function of our lambda and extracted certain functionality into the SlotService and DbUtil classes, making it easier to inject mocks for unit testing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/874/1*lZ2Oj4voo9ZKJgF53WQI9w.png" /><figcaption>Recap of refactoring steps in the previous post</figcaption></figure><p>However, there are still some improvements we can make so that the unit tests are less brittle. We will start by looking at ways to prevent any leaky abstraction in our design, by looking at our SlotService tests. Then we apply Ports and Adapters architecture (aka. Hexagonal architecture) to solve the issue. Finally, we discuss when to switch from unit testing to integration testing.</p><h3>Leaky abstraction in SlotService</h3><p>Ideally, we want the SlotService to contain only domain logics (aka. business logics). The domain logics are the unique part of of your application. In our case, it’s related to delivery slot management. The other parts, such as database connection or external API calls, are auxiliary code.</p><p>We want our unit tests for SlotService to focus on the correctness of domain logics, not underlying implementation details. However, the class is using Connection object created by DbUtil. This means it knows the underlying implementation details of SQL. If we were to test this method, we must mock all methods used in JDBC library.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b91ed07d829fb2b92db965f65e618574/href">https://medium.com/media/b91ed07d829fb2b92db965f65e618574/href</a></iframe><p>Can we do better? One option is to apply <a href="https://aws.amazon.com/blogs/compute/developing-evolutionary-architecture-with-aws-lambda/">Ports and Adapters architecture</a> (aka. Hexagonal architecture). The core idea of this architecture is to separate “what” and “how” when you interact with external components.</p><p>For example, I want to retrieve Slot data from the database (“what”). I do not care if the underlying database is done through SQL (whether it’s JDBC or ORM) or even NoSQL. The SlotService will care only about domain logic and know nothing about the underlying database.</p><p>Thus, I create an interface SlotRepository with retrieve() method as the “what”. For the implementation (the “how”), I create another class JDBCSlotRepository and move the snippets above to the retrieve() method. JDBCSlotRepository can reuse the DbUtil that has already existed.</p><p>The term “Port” is the “what” and the term “Adapter” is the “how”. By decoupling these twos, we can swap adapter to fit the same port. Let’s say that we want to migrate from AWS RDS to AWS DynamoDB, we could create a DynamoDbSlotRepository that implements interface. There should be no change on the SlotService and its tests which contains only the domain logic. Here’s a diagram of this design.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*lKlGjgBKitlna96RWMHRWA.png" /><figcaption>Introduce SlotRepository interface to decouple the “what” from the “how”</figcaption></figure><p>From caller side, the handler itself is actually another “adapter”. It transforms the AWS Lambda events into a format that is understandable by domain logic (by SlotService). If we decide to move this code to a container, we can create a new “adapter” that receive the call from a load balancer, transform inputs, and pass it to the SlotService.</p><p>From testing perspective, the main benefit is reduced brittleness. We can now pass a mock with SlotRepository.retrieve(). The tests can just call the function to get a mocked response and continue. An underlying infrastructure changes (e.g. switching away from JDBC) will not affect SlotService and its unit tests as it only contains domain logics.</p><p>SlotServiceTest can focus on the domain logics and the handler test will focus on a returned result that complies AWS Lambda contract with API gateway (e.g. status code). This increases the easiness to locate bugs. The tests for each individual class will focus on specific responsibility.</p><p>This sounds great! But what do we pay for this flexibility?</p><p>This style of architecture introduces more classes and some boilerplate code. The example SlotService uses only one external components, but it is common for a class to interact with 2-3 external components. This results in more interface classes, more overloaded constructors, and more boilerplate code to set up unit tests.</p><p>More components also mean more cognitive overhead. It indirectly impacts developers’ productivity. Your teammates also have to learn and adapt to this new pattern. And the team must agree on design and standard naming convention.</p><p>Still, if your domain logic is complex and requires a lot of test cases. It is a fair price to pay for the cost of maintaining brittle tests.</p><h3>Where to draw the line and stop unit testing</h3><p>I would like to draw your attention to these lines in the original getSlots() method.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a3913783c43a32449abc55086148c968/href">https://medium.com/media/a3913783c43a32449abc55086148c968/href</a></iframe><p>No matter how we design our classes, there will be a class that contains this low-level interaction with external components. To test this part of code, we can create a mocked Connection object and pass it via mocked DbUtil. The mock connection will expect subsequent calls.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e3f7bf514691e88431b9e4ab1b600a7b/href">https://medium.com/media/e3f7bf514691e88431b9e4ab1b600a7b/href</a></iframe><p>Are you comfortable with this test? I am not.</p><p>We want to test that the getSlots() returns right result, not calling JDBC library correctly. The returned result is actually a fake one generated by the library itself. The amount of “system verification” we gained from this is also very low. This test is basically mirroring implementation, which makes it brittle to implementation details change. In the example above, adding a field in WHERE clause will requires two changes in the test. The first one is updating the the expectedQuery variable. The second one is adding another setObject method.</p><p>We had a discussion in the team and decided that this is where we draw the line for unit tests. Mirroring the implementation too heavily doesn’t guarantee us more confidence that the code is working. Instead, integration test will verify a correct query execution and the returned result is in an expected format.</p><p>The example here is for JDBC but you will run into a similar situation in the repository layer. This may be in the form of external API or AWS SDK calls (for DynamoDb, SQS, or EventBridge, etc.). Sometimes, it makes sense to mock and verify that the parameters to those external calls are correct (especially if there is a complex logic in building those parameters). Sometimes it does not. Ask yourself which properties are we improving for the tests? Which are we reducing? Does the gain worth the cost?</p><p>Usually, you will be increasing the confidence level of your tests(“system verification), at the expense of speed, brittleness, and reliable. For unit testing, the most common property you trade is brittleness. Personally, I consider using integration tests if I have to mirror the implementation with 3+ mock method calls. Each system has different optimal points and each team has different preference.</p><h3>Conclusion</h3><p>In the previous post, we refactored the Lambda handler to make it more testable. However, SlotService still had some of the database implementation details (JDBC). This made the unit tests of SlotService brittle as they assume the implementation details. Thus, we applied Ports and Adapter architecture to take the implementation details out.</p><p>We also discussed when to stop writing unit tests. While 100% branch coverage with unit testing is nice, it might not worth the cost of adding very brittle tests which do not add much value. We write tests to verify that our system works, not to raise the number. However, there is no test that cover the JDBC calls and it is a critical part of our application. Thus, we will cover this part with integration testing in the next post.</p><h3>We love to hear from you</h3><p>The most exciting time for a writer is hearing the feedback from readers. Do you like, dislike, or having any other topic to suggest? Please do let us know in the comment or in <a href="https://twitter.com/ilmlfarmereng">our Twitter</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2cf69a41cd6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/i-love-my-local-farmer-engineering-blog/testing-serverless-applications-2-applying-ports-and-adapters-architecture-e2cf69a41cd6">Testing serverless applications #2: Applying ports and adapters architecture</a> was originally published in <a href="https://medium.com/i-love-my-local-farmer-engineering-blog">My Local Farmer Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Testing serverless applications #1: Unit testing AWS Lambda functions in Java]]></title>
            <link>https://medium.com/i-love-my-local-farmer-engineering-blog/testing-serverless-applications-1-unit-testing-aws-lambda-functions-in-java-80b7731ef706?source=rss-7b585727fa57------2</link>
            <guid isPermaLink="false">https://medium.com/p/80b7731ef706</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Chadchapol (Jemmy) Vittavutkarnvej]]></dc:creator>
            <pubDate>Tue, 10 Aug 2021 16:42:39 GMT</pubDate>
            <atom:updated>2021-08-10T16:42:39.612Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JzO0cvW9sCiHVIEUQshzJg.png" /><figcaption>Testing Lambda functions in cloud with love</figcaption></figure><p>Here is my confession: I never like writing tests. It’s boring, tedious, and depressing. I always want to ship a new shiny feature.</p><p><strong><em>Disclaimer</em></strong><br><em>I Love My Local Farmer is a fictional company inspired by customer interactions with AWS Solutions Architects. Any stories told in this blog are not related to a specific customer. Similarities with any real companies, people, or situations are purely coincidental. Stories in this blog represent the views of the authors and are not endorsed by AWS.</em></p><p>However, I always write tests for production-grade applications. I often block a pull request without adequate tests and poke my teammate to add tests for each bug fixed. Why? Because we simply cannot sustainably build new features without tests. Writing production code without accompanied tests is like building a house without solid foundation. Sooner or later, the house will fall apart.</p><p>In the previous service we built, we compromised testing in order to ship new features quicker. After a few months, we were overwhelmed with issues in production and hot fixes. We caught regression bugs very late in the cycle (usually through manual testing or from customer complaints). At one point, we had to freeze building a new feature in order to write more automated tests.</p><p>To make things worse, the code that’s written without testing in mind was difficult to write tests for later. Refactoring the code was like walking in a landmine, a simple change request makes my palm sweaty as it might introduce more regression bugs. It was terrible.</p><p>Perhaps you have experience (or are experiencing) a similar situation, that’s why you are reading this blog.</p><p>Here’s the outline of this post.</p><ol><li>Properties of good tests</li><li>Uniqueness of testing serverless applications</li><li>Writing testable Lambda handler</li><li>Mocking database object</li></ol><p>In this post, we will start by exploring the properties of good tests, why not all tests are created equal. And how these are different in the serverless context. These two parts provide a good foundation for you to form a testing strategy for your serverless application.</p><p>Then we move on to the main topic of writing unit tests in a Lambda function. We will present some unique challenges that don’t exist in traditional applications. Then we show how to restructure our code to be more testable.<br> <br>The examples in this post are written in Java with JUnit and Mockito. But the concepts are applicable to other programming languages.</p><h3>Properties of good tests</h3><p>We can divide tests broadly in to 3 levels:</p><ol><li>Unit tests</li><li>Integration tests</li><li>End-to-End tests</li></ol><p>The definition of these terms are blurry and debatable. For this blog series, unit tests are limited to public method level and do not call any external components. The integration tests include external components that we cannot control (e.g. calling an external services). The end-to-end tests emulate real user interactions to the application, and use every actual external components.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*OB9LyOj_HJhEU6WYG_eBSw.png" /><figcaption>The Testing Pyramid, Mike Chon 2004</figcaption></figure><p>Before going further, let’s take a step back to examine what makes some tests better than others. Here are 5 properties we prefer to have in our tests.</p><ol><li><strong>System verification</strong>. This is the whole reason for writing tests. We want to make sure the application is working (not just that we’re seeing the tests pass). If the tests do not cover all test cases, we can’t trust that our existing features still works as they were. If we can’t trust the tests, we have to test manually, defeating the whole purpose of writing automated tests.</li><li><strong>Fast. </strong>The slower the tests, the less likely we run it before pushing the code to CI tools, the more often we block each other with failed build. While we usually think of testing as a separate activity from coding, we actually do testing all the time. (Printing an output is also a form of testing.) Fast tests not only speed up the CI steps, they also speed up the development process.</li><li><strong>Reliable. </strong>Some of us may experience tests being “flaky”. Running flaky tests is like throwing a dice. They often pass, but not always. There are many possible root causes (e.g. tests rely on wait time, share their states etc.). The flakiness appears more often in the higher level tests (End-to-end tests and Integration tests). Avoiding this issue takes careful consideration and experience.</li><li><strong>Not brittle. </strong>Our tests are brittle when most existing tests fail after adding a small change. We have to make changes in several places to get them to work. One possible cause is leaky abstraction. Brittle tests are not harmful but expensive to maintain.</li><li><strong>Easy to locate bug</strong>. When a test fails, we should be able to know in which part of the code that bug lives. In unit testing, the ideal scenario is to have only one test fail for a bug, instead of 90% of the tests.</li></ol><p>Notice that each level of testing (unit, integration, end-to-end) has different <strong>tendencies</strong> of the 5 properties above. End-to-end testing is a great way to guarantee that everything works (System verification). But it’s slow, prone to flakiness (if badly-written), usually brittle, and very hard to pinpoint where things went wrong.</p><p>On the other end of the spectrum, unit testing is fast, usually reliable, and makes it easy to locate bugs. If written well, it is not too brittle and easy to maintain. However, we need to make sure that we cover all test cases. And even if we do, we cannot be sure that there is no bug when this unit of code interact with other parts. Many costly production issues actually come from configuration rather than code.</p><p>I said “<strong>tendency”</strong>, as it is not always true in all cases. We can write very robust end-to-end tests as well as a very brittle unit tests. That’s why we write this series. Some tips and know-hows can drastically improve these properties with little effort.</p><h3>Uniqueness of testing serverless applications</h3><p>There are some unique characteristics in serverless applications that affect our testing strategy:</p><ol><li><strong>Smaller code units.</strong> The code in our Lambda functions is usually small. Overall business logic can be split into multiple Lambdas and orchestrated by other AWS services (e.g. Step Functions, EventBridge)</li><li><strong>More integration points.</strong> A Lambda function usually relies a lot on external services (e.g. API Gateway, database, queues and message buses, another API, etc.). Any failure usually comes from those integration points (e.g. Missing IAM permission, incorrect resource policy, incorrect parameters, mismatch format etc.)</li><li><strong>Asynchronous side-effects.</strong> Asynchronous calls are more common in serverless architecture. The supporting services are usually asynchronous (e.g. SNS, Writing to DynamoDB that uses Stream, etc. ), increasing the chance of flaky tests.</li><li><strong>Longer deployment time.</strong> Deploying a Lambda function to the cloud takes more time. We can emulate the function locally, but we won’t be able to verify that it works correctly within the actual context.</li></ol><p>Due to the first three characteristics, many serverless practitioners favor integration testing (or End-to-End testing) over unit testing¹. Still, unit tests are not out of the picture. Integration tests that actually call real services are unavoidably slower. It will take very long time to cover all test cases. In additional, simulating failure scenarios are difficult. One does not simply switch AWS services off every time the test runs.</p><p>The last characteristic, <strong>longer deployment time</strong>, makes unit tests important for a development workflow. Given that deployment time usually takes 20+ seconds to several minutes, it is crucial to get quick feedback from unit tests to verify that the few lines of code I just added didn’t break anything. So I only need to deploy when I have successfully passed all unit tests.</p><blockquote>Here is another confession. I used to test code change via deployment without writing unit tests first. I would end up either (1) getting distracted by Reddit or Twitter while waiting in that 20+ seconds. (2) running out of patience and opening an editor in the AWS Console to edit the code directly. The results were not very productive.</blockquote><p>Of course, we can use some tools to reduce deployment time or even emulate Lambda infrastructure. Nevertheless, we still need a robust test suite that is fast and repeatable to check that future changes do not break anything.</p><h3>Writing testable Lambda handlers</h3><p>We are going to use the Delivery service API from previous posts² as an example. You should be able to follow this post without reading the prior posts. We start with a naive implementation of Lambda handler and see why it cannot be easily tested. Then we will propose a refactoring that allow us to inject a mock object into our tests.</p><p>In Lambda, we usually initialize reusable resources (e.g. Database connections) up-front for performance reason³. In this code, we have a GetSlots.handleRequest() as handler. It retrieves slot data from a database based on given parameters. To interact with the database, the method uses a SlotService object, which is created in the constructor.</p><p>Lambda will call the constructor when it is first initialized. Then the execution context of this function will be “kept warm”. This allows us to reuse the database connection inside the SlotService object in subsequent requests.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1ad726f744056b27c8cd2024438e89ca/href">https://medium.com/media/1ad726f744056b27c8cd2024438e89ca/href</a></iframe><p>When we started writing unit tests, we found that it is impossible to unit test this handler without side effects. If we call the handler directly, a new SlotService() will be called inside the constructor. And the SlotService will try to call an RDS database. That isn’t what we expect unit tests to do.</p><p>Thus, we add another constructor for testing. This one takes a SlotService as a parameter.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/48ec5f5cfa8ce6d1e528ddf52b09e916/href">https://medium.com/media/48ec5f5cfa8ce6d1e528ddf52b09e916/href</a></iframe><p>With this overloaded constructor, we can pass a mocked SlotService in our test code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/542b99bef9a53c1438f4fbf1410b879e/href">https://medium.com/media/542b99bef9a53c1438f4fbf1410b879e/href</a></iframe><p>In each test case, we use Mockito to intercept the call and return a canned response. This allows us to test the code in this handler without any side effects (e.g an external call to an RDS database).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/235fe7545418033faa44296ff7bab630/href">https://medium.com/media/235fe7545418033faa44296ff7bab630/href</a></iframe><p>With this setup, we can develop the handler without having to deploy anything to the cloud. When there is a change, we can rerun the unit tests to confirm that we didn’t break any existing functionality. Once we get all the tests green, we can then deploy and run integration tests.</p><h3>What did we achieve with this design?</h3><p>Let’s revisit the mentioned properties one by one and see how does this design perform.</p><p><strong>1. System verification. </strong>Injecting mocked service improve this property. We can now simulate edge cases easily. Here’s a test case when a given parameter (farm id) does not exist:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1299cbe581b8e6d9ee64e5f2fd0d23c2/href">https://medium.com/media/1299cbe581b8e6d9ee64e5f2fd0d23c2/href</a></iframe><p>Similarly, we can write test for other edge cases like required parameters, invalid id, etc. (<a href="https://github.com/aws-samples/i-love-my-local-farmer/blob/de70a50b891aea5710879f03ac89d08e41cfb468/DeliveryApi/ApiHandlers/tst/com/ilmlf/delivery/api/handlers/GetSlotsTest.java#L99">See the whole test class</a>).</p><p><strong>2. Fast. </strong>Not calling external services makes the unit tests faster to run.</p><p><strong>3. Reliable. </strong>Not calling external services makes the unit tests reliable. It will consistently pass or fail regardless of external environment.</p><p><strong>4. Not brittle. </strong>we test at the handler level and only need to create the response to API Gateway. The test will last as long as we don’t change the API contract or switch out API Gateway.</p><p><strong>5. Easy to locate bug</strong>. By having each case handled independently, we make it easy to locate bugs if any of the cases fail.</p><h3>Mocking database connection to test SlotService</h3><p>We started with the first attempt to unit test the handler. But the handler connects to Amazon RDS and we do not want to actually call any real database operations.</p><p>Instead of putting all the code inside the handler, we extract the business logic out into the SlotService class. The test for the handler injected a mocked SlotService where we can control the behavior.</p><p>However, we don’t have any tests for SlotService yet. The handler tests ensure that we return the correct status code and results based on the API contract. But the business logic for the database is inside SlotService. Also, the class needs to get data from Amazon RDS. So we need to mock the connection to write a proper test.</p><p>We extracted the code that is related to our database to a class DbUtil . This class has a createConnectionViaIamAuth() method that returns a Connection object back to the caller. Again, we can use a mocked DbUtil to inject a fake connection object that doesn’t really connect to RDS or simulate connection failure. The SlotService also has an overloaded constructor for testing purpose.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/775f01a6e4360665c3a57a2e2114858f/href">https://medium.com/media/775f01a6e4360665c3a57a2e2114858f/href</a></iframe><h3>Conclusion</h3><p>Automated testing is crucial for an application to live long. Not all tests are created equal and there are some properties we prefer to have in our tests. Low-quality tests can be as good as having no tests, or even liability for future changes. For serverless applications, integration tests have more important role than it was. The amounts of each type depend on your application. The bottomline is that we want to maximize system verification while not lowering any of its quality.</p><p>We demonstrated how to write good unit tests for AWS Lambda function in Java. The key idea is having a right abstraction so that you can inject mock object and test each class in isolation.</p><p>In the next post, we will visit the idea of Ports and Adapters architecture, use it in testing Lambda functions, and check how it affects the test properties. Also, we will discuss about when to draw the line to stop writing unit tests, and move to the integration tests.</p><h3>We love to hear from you</h3><p>The most exciting time for writer is hearing the feedback from readers. Do you like, dislike, or having any other topic to suggest? Please do let us know in the comment or in <a href="https://twitter.com/ilmlfarmereng">our Twitter</a>.</p><h3>Reference</h3><p>[1] Yubl’s road to Serverless <a href="https://medium.com/hackernoon/yubls-road-to-serverless-part-2-testing-and-ci-cd-72b2e583fe64">https://medium.com/hackernoon/yubls-road-to-serverless-part-2-testing-and-ci-cd-72b2e583fe64</a> <br>[2] The Strawberry’s Journey: From Your Local Farm to Your Table <a href="https://medium.com/i-love-my-local-farmer-engineering-blog/the-strawberrys-journey-from-your-local-farm-to-your-table-70e80b5a4f51">https://medium.com/i-love-my-local-farmer-engineering-blog/the-strawberrys-journey-from-your-local-farm-to-your-table-70e80b5a4f51</a><br>[3] Optimizing your Java Lambda Cold Starts and Initializations <a href="https://medium.com/i-love-my-local-farmer-engineering-blog/optimizing-your-java-lambda-cold-starts-and-initializations-5ca24de2c078">https://medium.com/i-love-my-local-farmer-engineering-blog/optimizing-your-java-lambda-cold-starts-and-initializations-5ca24de2c078</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80b7731ef706" width="1" height="1" alt=""><hr><p><a href="https://medium.com/i-love-my-local-farmer-engineering-blog/testing-serverless-applications-1-unit-testing-aws-lambda-functions-in-java-80b7731ef706">Testing serverless applications #1: Unit testing AWS Lambda functions in Java</a> was originally published in <a href="https://medium.com/i-love-my-local-farmer-engineering-blog">My Local Farmer Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Strawberry’s Journey: From Your Local Farm to Your Table]]></title>
            <link>https://medium.com/i-love-my-local-farmer-engineering-blog/the-strawberrys-journey-from-your-local-farm-to-your-table-70e80b5a4f51?source=rss-7b585727fa57------2</link>
            <guid isPermaLink="false">https://medium.com/p/70e80b5a4f51</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[requirements]]></category>
            <dc:creator><![CDATA[Chadchapol (Jemmy) Vittavutkarnvej]]></dc:creator>
            <pubDate>Tue, 11 May 2021 14:32:36 GMT</pubDate>
            <atom:updated>2021-05-11T14:32:36.446Z</atom:updated>
            <content:encoded><![CDATA[<h3>What to expect from this series?</h3><p>This series explains how we build a new service to enable our e-commerce platform to support home delivery. It will walk through all steps we take from scoping business requirements, architecting, and implementation.<br> <br> In this first post, we will start with scoping business requirements. We will elaborate a standard setup of delivery business operation and unique challenges we face in fresh product delivery. Then we will highlight trade-off decisions between project scope and delivery timeline. Like most technical design, there is no right and wrong. Everything is about trade-off so we would explain in details on why we make such decisions. This will be the foundation for the next blog post where we design API and architecture to meet these requirements.</p><p><strong><em>Disclaimer</em></strong><br><em>I Love My Local Farmer is a fictional company inspired by customer interactions with AWS Solutions Architects. Any stories told in this blog are not related to a specific customer. Similarities with any real companies, people, or situations are purely coincidental. Stories in this blog represent the views of the authors and are not endorsed by AWS.</em></p><h3>How COVID affected our business</h3><p>It all started with the first lockdown in France…<br> <br> I Love My Local Farmer’s mission is to give a direct access to local fresh vegetables and fruits to everyone. Our customers make orders to local farms through our e-commerce platform and went to pick it up at a designated time.</p><p>During the COVID period, people were more reluctant to go out to pick up. In some countries, it even became impossible to go out due to lockdown restrictions. We saw a big drop in the number of orders. Our revenue decreased quickly while the fixed operation expense were almost the same. As a result, the senior leadership team called for an emergency meeting. <br> <br> The outcome of the meeting was the decision to expedite adding a ‘home delivery’ capability into our platform. This enables customers to choose a preferred delivery time slot and we would deliver to their addresses during the selected time slot. This option would cost an extra fee. We would launch this service only in some regions in France to test the water, as soon as possible. Then we would roll it out to other regions and countries.<br> <br> This home delivery project wasn’t inspired purely by the pandemic. In fact, we had already reviewed the business plan multiple times and talked to a few delivery companies. However, the project required agreements between multiple stakeholders and therefore progressed very slowly. Once the decision was made, priorities were aligned, and we all focused on this project to rollout as soon as possible.<br> <br> We defined the scope of our MVP (Minimum Viable Product) like this: customers would use almost exactly the same interface as normal pick-up orders. Except that they would see the “home delivery possible” tag on the farms that opt-in to the delivery service model. (See wireframe below).</p><figure><img alt="A wireframe highlighting that the new delivery option is available for some farms" src="https://cdn-images-1.medium.com/max/983/1*9pOecTYB4NTLPMYBeEZ7_g.png" /></figure><p>When they go to the checkout page, they would get possible delivery time slots to choose from.</p><figure><img alt="A wireframe shows an additional option for home delivery in the checkout page" src="https://cdn-images-1.medium.com/max/1004/1*pBDxcEGvxcRzwNn38zB06A.png" /></figure><p>The change looks simple from customer perspective, but it requires sweeping changes on our end, from our business operations to our application itself.</p><h3>Challenge of fresh product delivery</h3><p>Logistics is a competitive business. Companies compete through price, using economies of scale and all kinds of optimization techniques. After all, who will pay additional 10 EUR for delivery? <br> <br>Therefore, our first thought was to completely outsource delivery process to a 3PL (3rd Party Logistics) company. This option minimizes cost but delivery time is around 2–5 days, depending on locations. It would have been the best option if we deliver non-food products. However, our value proposition is having fresh products directly from local farms. Ideally, the products should arrive within the same day. Some products must be kept in fridge during transportation. So this option was not feasible.<br> <br> The other option was to rent spaces in existing warehouses. We could pick products from local farms in an area and temporarily stock them in a warehouse nearby. Each day we could assemble orders at the warehouses and deliver to customers from there. This streamlines delivery process and minimize cost. Again, the drawback of this model is that it would take longer than 24 hours to arrive at customers’ houses. We also had to find warehouses with right locations that could store fresh products. <br> <br> This brings up another concern that we are not sure how customers will respond to the home delivery option. If there isn’t enough demand, the scale we operate will not be sufficient to justify even the lease cost. <br> <br> As a result, we decided to deliver directly from farms to customers. We would manage major parts of delivery process ourselves as a trial. Though we would hire drivers through contractors in logistics company to minimize capital investment. Based on what we learn, we would reiterate and tune our operating model once we see that this option could work.</p><h3>Simplifying business requirements</h3><p>We had to trade off complexity for fast launch. These are the requirements established by the business team:</p><ol><li><strong>We do not support a single order from multiple farms, even if they are close to each other.</strong> This decision is based on the fact that customers are already familiar with ordering and picking up from a single farm. If we allow a single order from multiple farms, we will have to (1) rewrite our e-commerce platform (2) have drivers pick products up from multiple farms and combine them into a single order before delivery. We find this too complicated for our first try.</li><li><strong>Customers will pick a time slot from established delivery time slots, and must be home to receive their products. </strong>We want to have orders from a single farm delivered in least numbers of drive. Thus, customers will have to choose a time slot from established choices. The slot will be either in the morning (8:00–12:00) or afternoon (13:00–17:00). Some farms with low volume might have only one slot per week. This is not very convenient for customers as they might have to order up to one week in advance, but it allows us to optimize to acceptable delivery cost.</li><li><strong>For home deliveries, we will expect a minimum order value. </strong>This is in order to compensate for delivery expenses.</li><li><strong>Farmers have to opt-in to this delivery option.</strong> We do not onboard every farm with this option by default. For home delivery orders, farmers need to pack items in standardized boxes/crates provided by us (more details on this later). They also get charged at higher fee than normal pickup order.</li><li><strong>For the MVP, the process will be controlled by human schedulers. </strong>We will eventually switch to an automated scheduler to optimize the delivery automatically based on the knowledge acquired during the MVP period.</li></ol><p>These scoped down requirements are not permanent. It will be only for the trial period and we will reiterate based on what we learn. The crucial factor is whether we could optimize the cost to the level that we could at least break-even with the delivery fee we charge customers and farmers. Using what we learn from the launch in the first region, we might decide to abandon this idea, or expand to other regions.</p><h3>The strawberry’s journey</h3><p>Now that we have understood the challenges and our business requirements. Let’s embark on a journey of our favorite fruits, strawberry. We will start from placing an order to arriving at a customer’s home. This journey will influence how we design our service API and architecture later.<br> <br>When a user clicks on “Find nearby farm” button to locate local farms. We use user’s address to get the list of farms in close proximity. Each farm that opts in to our delivery option will be linked to a list of post codes that is within delivery range. If the user’s post code is in the list, they will see a bright tag of “home delivery available”.</p><figure><img alt="A diagram visualize the steps for customers to place a home delivery order" src="https://cdn-images-1.medium.com/max/1024/1*q6O74CC0412GYqlU2SZu0A.png" /></figure><p>In the first step, the user clicks on a farm, adds strawberries and other products to user’s cart. Then the user goes to the checkout page in step 2, where delivery time slots are presented.</p><p>The slot data is owned by new delivery service, so the existing e-commerce platform has to call delivery service API to get available slots for that farm. The user selects the preferred slot and goes to the payment page.</p><figure><img alt="A diagram visualize the steps for farmers from receiving an order, packing item, to handling products over to drivers" src="https://cdn-images-1.medium.com/max/960/1*BJvMmQCc7dEtr5OYQwy5KQ.png" /></figure><p>Once the user has successfully paid, the order will be propagated to delivery service. We notify our farmers 2 days in advance so they have time to prepare the order. They also receive the printable labels for all orders. They pack each order in a separate bag/box and put an order label on it. This is something they are familiar with as it’s the same when customers come to pick up items themselves.</p><p>The additional step for the farmer is to put the bags in the standard crates that we gave them. Then they put a “crate label” associated with orders. The standardized crates allow drivers to stack products without crushing them. The crate labels have sorting numbers for the houses that they will be delivered to.</p><p>Our scheduler assigns delivery slots to farms in the way that all orders from a single farm are usually delivered on the same date. This is to optimize delivery cost and prevent overbooking. Each time slot has a maximum number of orders it could fulfill. Assuming that drivers could take 15 minutes per order, they could deliver approximately 16 orders in half a day (4 hours x 4 orders per hour). (Note that the number is different for each area)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ndLf6-UCL2DFfSZvGvy99w.png" /></figure><p>Based on historical data, the scheduler can estimate that the farm will have approximately 50 orders per week. Thus, the scheduler allocates 1 driver for all of Tuesday (~2x16 orders) and another driver on Thursday (~16 orders)</p><p>If there aren’t sufficient orders (e.g. only 10 our of expected 32), schedulers might route the van to a nearby farm for additional orders.</p><p>As June is coming, the scheduler knows from historical data that the farms selling strawberries will get more orders as they are in season. The scheduler would adjust the number of slots and make sure that we have enough contracted drivers. <br> <br>Now that the strawberries are safe in the van, the driver will go to each address planned by schedulers, unloading crates. With this process, the strawberries will arrive within 8 hours after leaving the farm.</p><h3>What’s next?</h3><p>Now that we have covered the business requirements, we are going to dive deeper into technical details in the next blog post. We will create API specification and architect our backend to connect with existing e-commerce system to make our strawberries journey as smooth as possible.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70e80b5a4f51" width="1" height="1" alt=""><hr><p><a href="https://medium.com/i-love-my-local-farmer-engineering-blog/the-strawberrys-journey-from-your-local-farm-to-your-table-70e80b5a4f51">The Strawberry’s Journey: From Your Local Farm to Your Table</a> was originally published in <a href="https://medium.com/i-love-my-local-farmer-engineering-blog">My Local Farmer Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>