<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[The Hotels.com Technology Blog - Medium]]></title>
        <description><![CDATA[Public blog for the Hotels.com Technology team. - Medium]]></description>
        <link>https://medium.com/hotels-com-technology?source=rss----29989b59e37d---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>The Hotels.com Technology Blog - Medium</title>
            <link>https://medium.com/hotels-com-technology?source=rss----29989b59e37d---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 04:00:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/hotels-com-technology" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[We have moved…]]></title>
            <link>https://medium.com/hotels-com-technology/we-have-moved-c0bcc91ee1f?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/c0bcc91ee1f</guid>
            <dc:creator><![CDATA[Hotels.com Engineering]]></dc:creator>
            <pubDate>Thu, 20 Jun 2019 10:10:58 GMT</pubDate>
            <atom:updated>2022-03-07T17:54:28.911Z</atom:updated>
            <content:encoded><![CDATA[<p>…but we’re taking you with us in a bigger place!</p><p>We at Hotels.com Tech, will keep posting the coolest tech stuff alongside many people coming from the several brands of the group on the new <a href="https://medium.com/expedia-group-tech">Expedia Group techblog</a>.</p><h4>Follow us to get the latest updates, and see you there!</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0bcc91ee1f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/we-have-moved-c0bcc91ee1f">We have moved…</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Resilience at Hotels.com (Part 1 — Kube-Monkey)]]></title>
            <link>https://medium.com/hotels-com-technology/resilience-at-hotels-com-part-1-kube-monkey-c54e6f95f2c4?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/c54e6f95f2c4</guid>
            <category><![CDATA[resilience]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Nikos Katirtzis]]></dc:creator>
            <pubDate>Mon, 13 May 2019 13:01:01 GMT</pubDate>
            <atom:updated>2019-05-14T08:47:02.325Z</atom:updated>
            <content:encoded><![CDATA[<h3>Resilience at Hotels.com (Part 1 — Kube-Monkey)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rvVSsDT0_a-5yrhh2egE5Q.png" /><figcaption>On the right side of the picture you can see a monkey… and kube… keep reading.</figcaption></figure><p>When we wrote our first <a href="https://medium.com/hotels-com-technology/i-dont-know-about-resilience-testing-and-so-can-you-b3c59d80012d">blog post</a> that’s related to resilience engineering we didn’t think about starting a series of posts in that area, mainly due to the following facts:</p><ul><li>We didn’t have enough exposure to resilience testing and resilience engineering (we don’t have even now).</li><li>We were still at early stages in terms of our cloud migration from our data centres to AWS with Kubernetes (now we are not).</li></ul><p>Since then lots of things have happened, many services are currently running in AWS in production while we are also collaborating with the other Expedia Group brands in the area of resilience engineering considering that we’re all tackling similar issues.</p><p>That said, from now on you’ll hear more about what we do at Hotels.com in that area!</p><p>This blogpost focuses on <a href="https://github.com/asobti/kube-monkey">Kube-Monkey</a>, an implementation of <a href="https://github.com/Netflix/chaosmonkey">Netflix’s Chaos Monkey</a> tool for Kubernetes clusters which we started using a few months ago.</p><h4>Why we needed Kube-Monkey</h4><p>During the last year many teams have migrated services from our data centre to our Kubernetes platform in AWS. Where applications previously ran on fixed hosts for their lifetime, there are two new types of change we must be prepared for:</p><ol><li>Kubernetes dynamically managing the lifecycle of our applications.</li><li>The EC2 servers that underlie our platform are ephemeral and may fail or be replaced at any time.</li></ol><p>Development teams should make sure that their app is a <strong>good citizen</strong> and can handle these changes. This means an app should:</p><ol><li>Provide a good quality of service quickly as new instances may be created at any time.</li><li>Be resilient to changes in its dependencies that come as a result of the flexibility of the new platform.</li></ol><h4>The cost of unknown unknowns</h4><p>For some cases it is possible to prepare for our applications failing ahead of time. At the simplest level we do this with exception handling and circuit breaking, and these can be verified with unit tests. We can also simulate more complex failures with integration tests that inject failures at the tcp level, for example with <a href="https://github.com/Shopify/toxiproxy">ToxiProxy</a> (you can read more about how we do it in our <a href="https://medium.com/hotels-com-technology/i-dont-know-about-resilience-testing-and-so-can-you-b3c59d80012d">blog post</a>).</p><p>This is great for cases that we know about already or suspect might happen. But with distributed systems there may be cascading failures, or infrastructure issues outside the scope of our testing pipeline. How can we be resilient to these unknown unknowns?</p><p>The easiest way is to assert that our services should not fail, wait for an issue to happen in production… no?</p><p>Perhaps a better approach is to bring forward these kinds of errors in a controlled way so we can fix them the soonest possible.</p><h4>Exposing issues using Kube-Monkey</h4><p>At Hotels.com we have deployed Kube-Monkey into our staging and production Kubernetes clusters to help expose these sorts of problems.</p><p>At the start of every week day, Kube-Monkey creates a schedule of Kubernetes deployments whose pods it will terminate throughout the day. The daily schedule for production is sent to an internal DL on which our developers can subscribe.</p><p>At the scheduled times, Kube-Monkey selects a number of pods from that deployment and terminates them. This is based on labels and is currently <strong>opt-in</strong>. Kubernetes will then recreate the missing pods, and as this process happens we can understand what happens to this app and its dependencies when it is running at reduced capacity.</p><p>Many apps are using this in our staging environment, with a few of them trialing it in prod. Running pod terminations against these services in production has exposed a few issues — both with regard to being a good citizen, and resilience to external factors.</p><p><em>Response times were slower for new instances even after they were marked as able to take traffic and having been through a warmup routine.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xHdZ1dYoGPP6maPkT6B-vw.png" /></figure><p><em>Traffic was observed to be unevenly spread across the nodes of the services cluster due to persistent connections.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oL6QPVub6y9ir1Yi0V50XA.png" /><figcaption><em>Traffic evenly spread across the nodes of the services cluster (expected).</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dcaO39ZDi_xXx9I5FJu4iA.png" /><figcaption>Traffic <em>unevenly spread across the nodes of the services cluster (reality).</em></figcaption></figure><p>Another issue we found was that the grace time after pod shutdown that allows for connection draining was less than the default Java DNS cache TTL of 30 seconds. This meant that requests were still sent to pods that had been terminated.</p><p>None of these things are easily identifiable without running in a real Kubernetes cluster, but do the tests really need to be run in production? Why not staging? While we can strive to make the environments as similar as possible, even with shadow traffic it is not possible to completely replicate prod, and almost certainly not cost effective as we would need to replicate the entire infrastructure. Provided we have a way to cancel them, controlled experiments in production are the best way to verify our resilience capabilities.</p><h4>How do our apps opt in</h4><p>Kube-Monkey is being trialed in production with the intention of it being compulsory for all new apps. We’ve tried to keep the opt-in mechanism as simple as possible and so all apps can just set a single property in their Helm Charts to enable Kube-Monkey. But they also have the option to control the attack by specifying the time between attacks, or the number/percentage of pods to be attacked.</p><h4>Next Steps</h4><p>Kube-Monkey is limited to only one sort of attack — terminating a fixed number or percentage of a deployment’s pods. In order to help teams expose a wider range of issues we are planning trials for other similar tools, including <a href="https://www.gremlin.com/">Gremlin</a>, <a href="https://istio.io/docs/tasks/traffic-management/fault-injection/">Istio’s failure injection capabilities</a>, and an internal solution from another Expedia Group brand.</p><p>But the tooling is only part of the story. To maximise its value we need to measure against expected and actual values so that we can track how our resilience improves over time. This means defining an application’s steady state in given situations and is closely related to service level indicators/objectives/agreement.</p><p>That’s Kube-Monkey in a nutshell, and it’s only a small part of resilience engineering. Our journey has just begun!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*kjL6GtIhZ7zsGNjsN7K3tg.png" /></figure><p><em>Note</em>: This post was a joint effort of <a href="https://medium.com/u/735671929442">Andy Stanton</a>, <a href="https://medium.com/u/6a71cc885fbc">Daniel Albuquerque</a>, and myself.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c54e6f95f2c4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/resilience-at-hotels-com-part-1-kube-monkey-c54e6f95f2c4">Resilience at Hotels.com (Part 1 — Kube-Monkey)</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[‘Responding to Change’ over ‘following a Plan’ — my learnings from Agile]]></title>
            <link>https://medium.com/hotels-com-technology/responding-to-change-over-following-a-plan-my-learnings-from-agile-578bc67e8640?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/578bc67e8640</guid>
            <category><![CDATA[acceptance]]></category>
            <category><![CDATA[agile]]></category>
            <category><![CDATA[accepting-change]]></category>
            <category><![CDATA[change]]></category>
            <category><![CDATA[responding-to-change]]></category>
            <dc:creator><![CDATA[Tanya Saxena]]></dc:creator>
            <pubDate>Tue, 07 May 2019 13:01:01 GMT</pubDate>
            <atom:updated>2019-05-07T13:01:01.513Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong><em>‘Responding to Change’ over ‘following a Plan’ — my learnings from Agile</em></strong></h3><p>I joined Hotels.com a few months ago, after a career break of 9 months. This bought about a lot of positive changes in my life, while all the beautiful memories, experiences and values form the past remained intact. From being a QA Engineer with 8 years experience I started my new journey as a Scrum Master. In addition, I moved from London to Rome, with little job security or financial stability. But thankfully my Agile background made the transition easier.</p><p>Change is inevitable. Political, environmental, psychological, physical or the ones related to our careers, diets, practices, beliefs, the list is endless. It is not always easy to deal with these changes. Sometimes change can be very scary because it takes you into something unknown, takes something away you. It may have been what made you comfortable. You may have loved the way things were. Even if you know the change is needed, you may grieve what was left behind.</p><p>The reaction varies from excitement to fear, over enthusiasm to confusion, numbness to anger, resentment or a confusing mixture of all these emotions. And that’s cause something somewhere goes off track, off plan. These changes are usually beyond our control. However, we always have the ability to control how we respond.</p><p>Our attitude to change determines how we embrace it. That’s where the Agile philosophy comes in. An approach that helps you react differently to the way people normally react. It takes effort. And practice. But it makes the adaptation easier.</p><p>I would like to share how the Agile philosophy could be inspirational, and I hope you find it helpful in your own transition.</p><h3><strong>Reactions to Change</strong></h3><h3><strong>Unfamiliarity</strong></h3><p>More often than not, change can be disorienting. Everything is going as per plan, you are in your comfortable, familiar space and suddenly, you are required to you embrace something unexplored. You don’t know what to feel, which means you don’t know how to respond. You close yourself off and start ignoring your thoughts, avoiding to come to terms with the new situation.</p><p>Agile is <strong>open to change</strong> throughout the development process. It is based on the assumption that circumstances change. It moulds our mind-set to embrace change as a natural and expected part of development. It shifts the focus from preventing change, to managing expectations, accessing risks, measuring impact and tracking changes.</p><p><strong>This helps you to</strong>:<br>- fully accept and identify with your changed circumstances as like it or not; you cannot control it</p><p>- fight your internal resistance to change by having an open mind</p><p>- manage your feelings by shaping your response in a positive manner</p><p>- examine opportunities that could be explored</p><p>- move forward with the knowledge and value that you have gained so far</p><h3><strong>Uncertainty</strong></h3><p>Adjusting to changes is not always easy. Fear comes in many different forms. There are a variety of things that we are afraid of. But the one we would all agree on experiencing, is the fear of failure. It is the emotional, cognitive reaction to the outcome you predict for failing to achieve a goal. It makes you weak, reluctant to think creatively. Brings about a lot of negative thoughts and makes you feel blocked.</p><p><strong>Fail Fast, Fail Cheap, Fail Often</strong> — the agile approach to accept uncertainty, and to test your ideas, problems, and solutions as quickly, cheaply, and often as possible — with the understanding that many of these ideas might fail. However, without testing new ideas, you’ll either invest too much (time, money, effort) into the wrong things, or you’ll never bring those challenging and inventive ideas to market. So, your focus should be on failing fast, failing cheap, and failing often.</p><p><strong>This helps you to:<br></strong>- accept that not all ideas are good but that shouldn’t restrict your exploration process</p><p>- understand that failures and false starts are totally acceptable</p><p>- start honest discussions by giving dynamic and convincing reasons</p><p>- fail an idea earlier in the process by making the best use of available resources and opportunities</p><p>- pursue only those ideas that deliver the greatest value</p><h3><strong>Confusion</strong></h3><p>You have accepted the change in it’s full glory but now your head is exploding trying to figure out how to address the million thoughts going through your head. Where do you even begin. How do you comprehend the situation in its complexity. Everything looks fuzzy.</p><p>A <strong>product roadmap </strong>communicates how a product is likely to evolve across several major releases. It captures requirements at a high level by describing the journey you want to take your product on, split over the next few months. You distinguish the things that are a little bit fuzzy from the things that you’ve already committed to for the short term.</p><p><strong>This gives you:</strong><br>- a high level plan that only sketches out the major stages of the upcoming journey</p><p>- a high level, visual representation of your goals</p><p>- a format to track your progress</p><p>- flexibility to incorporate enforceable changes</p><p>- the ability to visualise the big picture</p><h3><strong>Acceptance and moving on</strong></h3><p>Change is a slow process. Even after you think that you have crossed the initial hurdles, you could still find yourself stranded on unknown grounds.</p><p>Agile advocates <strong>breaking down features</strong> into small pieces known as “user stories.” These stories are then scored relatively using”story points” based on their complexity. Once the team begins development and the project is tracked, data is available to measure the project velocity, therefore enabling more precise forecasting of requirements and timelines. IT supports small, continuous, incremental, and iterative releases.</p><p><strong>It teaches you to:</strong><br>- accept that you are still adapting to the new reality that faces you</p><p>- break down your bigger goals into smaller, easily attainable tasks</p><p>- put your energies on the present things</p><p>- try to prioritise completing tasks that are in your control</p><p>- take note of your progress and your bottlenecks</p><p>All in all, when you spend time to Reflect on a situation with the right attitude, you learn how to Respond more thoughtfully!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=578bc67e8640" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/responding-to-change-over-following-a-plan-my-learnings-from-agile-578bc67e8640">‘Responding to Change’ over ‘following a Plan’ — my learnings from Agile</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Guardian — answering questions from other teams efficiently]]></title>
            <link>https://medium.com/hotels-com-technology/guardian-answering-questions-from-other-teams-efficiently-62401c97221c?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/62401c97221c</guid>
            <category><![CDATA[team-collaboration]]></category>
            <category><![CDATA[team-communication]]></category>
            <category><![CDATA[focus]]></category>
            <category><![CDATA[agile]]></category>
            <category><![CDATA[teamwork]]></category>
            <dc:creator><![CDATA[Adriana Santos]]></dc:creator>
            <pubDate>Thu, 25 Apr 2019 13:01:01 GMT</pubDate>
            <atom:updated>2019-04-26T07:19:52.983Z</atom:updated>
            <content:encoded><![CDATA[<h3>Guardian — answering questions from other teams efficiently</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AqIZvzZ2BHULcQty-peqCA.jpeg" /></figure><p>Do you have a Slack channel or emails where colleagues outside your team ask questions? Sometimes everybody is busy with their work and whoever asks the question will struggle to get an answer. We had this problem, but not anymore.</p><p>Every week, we have a person in the team (developer, QA, product manager, tech lead, it doesn’t matter) who is the <em>guardian</em>. Besides their main responsibilities, the guardian is also responsible for answering questions from other teams. The guardian doesn’t necessarily know the answer, but is responsible for finding the right person.</p><p><strong>Who is the next guardian?</strong></p><p>We have a list of all team members on a wiki page which we consult on Mondays to find the next guardian. If that person is on holiday/unavailable, then the next person from the list will take the role.</p><p><strong>How do people from other teams identify the guardian?</strong></p><p>We mention the guardian in our public Slack channel’s topic. However, people don’t need to know this. They can still ask a question by referring @here or by sending an email to our mailing list and then the guardian will answer.</p><p><strong>What improvements have we seen since we have the guardian?</strong></p><ul><li>The team is less distracted, since there is a single person responsible for looking at questions from other teams.</li><li>We are quicker at answering questions from other teams.</li><li>There is more cross-team collaboration.</li></ul><p><strong>I want to try it. What do I need to do?</strong></p><ul><li>Create a page with the list of people in your team and an empty column for adding dates.</li><li>Agree with the team that one person is responsible for addressing questions on your team’s communication channels (e.g. email, Slack channels) for a week.</li><li>Every Monday morning, the guardian from the previous week will ensure the next guardian is selected and identified in the Slack channel’s topic. The previous guardian also needs to remember to update the page with the dates when they were the guardian.</li></ul><p>This process worked well for us, but you might need to adjust it to your needs.</p><p>Give it a try, if you face the problem of answering questions from other teams efficiently. You will all work more closely, help each other and be less distracted.</p><p>Photo by <a href="https://unsplash.com/photos/yn8aHOdNLZo?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Ilkka Kärkkäinen</a> on <a href="https://unsplash.com/?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=62401c97221c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/guardian-answering-questions-from-other-teams-efficiently-62401c97221c">Guardian — answering questions from other teams efficiently</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Service invocation optimization techniques]]></title>
            <link>https://medium.com/hotels-com-technology/service-invocation-optimization-techniques-84155809649d?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/84155809649d</guid>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[architecture]]></category>
            <dc:creator><![CDATA[Dániel Imre]]></dc:creator>
            <pubDate>Mon, 15 Apr 2019 13:01:01 GMT</pubDate>
            <atom:updated>2019-04-15T13:01:01.246Z</atom:updated>
            <content:encoded><![CDATA[<p>Development teams working in a microservice architecture face a tricky problem: improving performance but also integrating with more and more services. Let’s explore some solutions for optimizing service invocations at the backend side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*dWY3eA6NFTPMSHYjfv6M5Q.png" /><figcaption>An extremely complicated architecture</figcaption></figure><h3>Background</h3><p>Few years back we were working only with few service calls per page request and those were shallow invocation chains as well. Now, we might need to invoke 10+ services per a single page request and some of those have really deep call chains with varying response times. Page performance matters more than ever yet it is getting harder and harder to maintain a consistent response time with that many dependencies.</p><p>Also, while most services perform well under the 95th percentile (p95), some of them struggle to keep an acceptable response time above p99. The effect of these spikes multiply with the number of calls so it will affect much lower percentiles at aggregation points.</p><h3>What can we do?</h3><p>In general caching can help. However, with a cold cache or expired/evicted entry we will face the original problem. Also, several requests can hit this missing entry at once and will end up with multiple downstream calls for the same element.</p><p>Blocking is also a problem we should address with async/reactive execution flows. I/O should be non-blocking and we should defer execution where we can.</p><p>We should also prefer shared remote caches to enable scaling out but that also introduces another layer of flaky remote call.</p><p>Let’s address these problems one by one with different techniques.</p><p>The sequence-diagrams use the following notation:<br>–&gt; means a synchronous call<br>-x-&gt; means an asynchronous/reactive call</p><h3>Async service invocation with double timeout</h3><p>Caching can happen at multiple service layers. These caches usually need warm-up. Also, the response times heavily depend on whether the values are coming from cache or not. Sometimes, response times are just unacceptable (1+ seconds) yet we would like to have them both cached for next time and return them if they arrive in time.</p><p>For such situations we can introduce a caching downstream service wrapper with primary and secondary timeouts. The primary timeout would be the maximum time we wait for the response no matter if it is coming from cache or not. The secondary timeout is the maximum time we wait for the downstream service to respond with the value we can then put into cache (asynchronously).</p><p>2 timeouts:</p><ol><li>first lower timeout (T1) in ServiceAdapter</li><li>second higher timeout (T2) in Service</li></ol><h3>Request for uncached value with fast service call</h3><p>In this case the the downstream response time is less than T1 so we can get the value with the first call.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*_3WyTSOhK1v2RBZaBodAuQ.png" /></figure><h3>Request for uncached value with slow service call</h3><p>If the service response time is between T1 and T2 then initially we will get an empty value but the response will be (most probably) cached for next invocations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/789/1*TEdGMoX1rfgF8g1KCR6TjQ.png" /></figure><h3>Request for uncached value with too slow service call</h3><p>If the response time for the given request is beyond T2 then it will return an empty result and won’t be cached ever. To avoid always timing out at T1 for such cases we can cache an empty result if T2 is reached but we should be careful with this as it can happen for different reasons (network congestion, garbage collection spike). Though, when the empty result is evicted from cache, it will be retried later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/1*resLO6xy9328MUja__YHvQ.png" /></figure><p>Benefit of this solution is that it enables cache warmup for slow calls without significantly affecting upstream response times.<br>Drawback is that is can be only used if a temporary empty result is acceptable because it is either optional or there’s a fallback.</p><h3>Async reloading cache</h3><p>The plain and simple caches are synchronous (thus blocking) and have a single TTL (time to live) defined. They evict elements when their TTL is reached or the cache is full. The problem with this solution is that it causes response time spike when it encounters a cache miss even if the element was previously cached.</p><p>While on-heap caches are usually very limited with the cached elements, we are usually better with off-heap caches anyway which can have significantly more room so cache eviction caused by size limit is neglectable there.</p><p>To mitigate the eviction because of expiry we can define a cache with both TTL and TTR (time to refresh). TTL should be significantly larger than TTR so elements are barely evicted. Some cache solutions will support this out-of-the-box (e.g. Guava or Caffeine) but for remote caches (Redis, ElastiCache) we can implement this easily by storing a timestamp next to the cached value. So when an element is retrieved from the cache this timestamp is checked against the TTR and if it needs refresh then a request is sent to downstream service and the cached value is updated asynchronously.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/443/1*bgiYcNrt1lfE9mBaZgDxHQ.png" /></figure><p>If the result is optional we can also make the caching wrapper return an empty result if it encountered a cache miss. This will remove the initial spike as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*kpGB4kVSYsIHgYiFR6aNkA.png" /></figure><p>The benefits are clear here:</p><ul><li>less or no spikes (actually for remote cache calls we might still encounter small spikes)</li><li>added resiliency as there will be a cached value for some time even if the downstream service is down</li></ul><p>Just like for the previous pattern the drawback might be that this can be used only with limitation if the result is mandatory. Also, because of asynchronous execution tracing the downstream service requests will be tricky for the value-refresh calls.</p><h3>Deduplicating request collapsing</h3><p>The previous caching mechanisms solved some aspects of the complex problem we are trying to solve but we can do more. As a good citizen we might also care about the downstream service (and also save some resources on our side) and fire as few requests towards it as we can. By deduplicating similar parallel requests we can decrease the number of calls which will definitely help with resources both in our application (threads, memory, CPU) and on the network (bandwidth).</p><h3>Request collapsing with release when finished</h3><p>We can choose to collapse ongoing requests and release them once the collapsed call finishes. First request to a context will fire a downstream service call. While the response is pending every subsequent calls to same context will be put on hold and will get notified once the result is ready. This can be done semi-asynchronously with a promise but it is even better to be done in a reactive-way eliminating blocking at all levels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*v5aeChJWmKTYZhTwrCvBgA.png" /></figure><h3>Request collapsing with release on TTL expiration</h3><p>We can also store on-going requests for a while after they have been initiated (few seconds TTL). This will act like a short-term request-promise cache potentially with only few hundred elements to mitigate call spikes under higher data cache layers. The important difference compared to usual data caching solution is that you will have the cached promise of the data even before the data is actually available.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*aOOfUjYwWI7Fj4Jv6o22QA.png" /></figure><p>A benefit of this solution is that it will optimize the resource usage of downstream calls.</p><p>One can also notice that the above pattern speeds up parallel calls as they can get a result from a former request sooner. For example if a request needs 100 ms to complete then a blocking data caching solution would need to wait 100 ms to get the data even if there’s already another request for that data. With this solution if there’s an on-going request started 60 ms before then a subsequent request will only need to wait 40 ms for the result.</p><p>It also comes handy when your logic can be split to parallel subtasks. e.g. resolving placeholders in a template where you will need an entity for multiple placeholders but you don’t know in advance which placeholder will require it and when as the template can be arbitrary. You could do some pre-processing phase to determine this but that would complicate things and would work only for your current task.</p><p>With such a request collapser your subtasks won’t need to know about each other. Each can call the service (through the collapser) to fetch the very same data in parallel. Still, the service won’t be stressed as the request collapser will protect it by deduplicating similar overlapping requests to a single one. This can lead to much simpler code design and architecture while also having the effect of this optimization shared for all higher level requests.</p><p>This is not a silver bullet though. Real effect can only be observed with high frequency calls with lots of redundancy. If there’s not enough parallel calls we might only add extra overhead.</p><h3>Fan-out request collapsing</h3><p>Some services support bulk requests: getting results for more contexts at once. While this sounds great it introduces another problem: if the contexts are varying and overlapping by different upstream calls than the downstream service will get multiple requests for the same context unnecessarily.</p><p>To mitigate this we can introduce another layer of abstraction which acts like a middle-man and exposes a single context API upstream while sending bulk requests (multiple contexts at once) downstream and once the result is ready it delivers the correct result to each upstream caller. Might sound complicated but lets see the diagram.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*sWQewQhL1Xjx7y_nE9liQA.png" /></figure><p>The benefits should be clear: we optimize the calls by always batching the optimal number of contexts in one.</p><p>Please note that this solution won’t deduplicate calls on its own. The previous request collapser can be used for this.</p><p>The drawback is that the number of in-flight requests most probably won’t match the batch size. There will be always some partial batches. For these we have to introduce some timeout: the maximum time to wait for the next batch to be full. This can add some latency to partial calls but we can at least control how much we are willing to wait.</p><h3>The pieces together</h3><p>Now, that we have solution for most of the problematic parts, let’s see how these look like when combined.</p><p>So this is a sequence diagram for an asynchronous reloading cache over a request collapser over a fan-out request collapser with batch size of 2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6FIdu0aFIP0wheFCRCFCOg.png" /></figure><p>As we can see they nicely work together to tackle the problem. We could also add the double timeout on top of this if still needed.</p><p>For a given case with the above technique we could see 75% reduction in call rates against the downstream service while upstream response times improved by 400% and the same stability (no or minimal spikes) could be achieved at p99 from previous p95. The result of course heavily depends on your use-case and call patterns.</p><h3>Implementation</h3><p>Implementing the above patterns might not be trivial. Async flows are challenging on their own. If you also would like to make them reactive that’s another level of complexity. The good thing is that these can be made generic and can serve your architecture in the future without additional investment. Reactive libraries like RxJava and Reactor provide convenient building blocks for these patterns (e.g. buffer, timeout, cache, Subject/Processor).</p><p>Also, be sure to have some internal metrics in place to be able to monitor these patterns in practice. This is also important when you need to tweak some of their parameters. e.g. how does adjusting maximum wait time affect overall completion time in a fan-out request collapser.</p><h3>Next steps?</h3><p>While we were talking about application level here, some of these patterns can be moved to system integration level as well: e.g. a request collapsing proxy. Whether we should build something like that or not is a question as it introduces additional maintenance complexity and network overhead, and might be against other operational and architectural decisions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84155809649d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/service-invocation-optimization-techniques-84155809649d">Service invocation optimization techniques</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hotels.com speakers at conferences]]></title>
            <link>https://medium.com/hotels-com-technology/hotels-com-speakers-at-conferences-43661e8b81d3?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/43661e8b81d3</guid>
            <category><![CDATA[conference]]></category>
            <category><![CDATA[events]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[speakers]]></category>
            <category><![CDATA[talks]]></category>
            <dc:creator><![CDATA[Hotels.com Engineering]]></dc:creator>
            <pubDate>Mon, 01 Apr 2019 13:01:01 GMT</pubDate>
            <atom:updated>2019-04-02T12:44:58.154Z</atom:updated>
            <content:encoded><![CDATA[<p>So far in 2019, many conferences across Europe accepted presentations from our colleagues at Hotels.com Technology. We are so proud of everyone who has had their call for papers successfully accepted! It is a great opportunity for our technologists to share their learnings with the vibrant tech community around Europe.</p><p>Some of those speakers have also contributed blog posts to this very publication! Have you read our blogs and thought <em>“I want to know more?”</em> Or <em>“I want to meet the people behind the tech?”</em> Now is your chance….</p><h3>Here is a full rundown of who from the Hotels.com Tech team is speaking, where they are speaking as well as the topics they will touch.</h3><p><strong>Speaker:</strong> <a href="https://medium.com/u/ade7e9aa5a25">Jonathan Rigby</a></p><p><strong>Conference:</strong> <a href="https://london2019.theleaddeveloper.com/">The Lead Dev</a>, London 27th-28th June</p><p><strong>Talk Title: <em>“How long is a piece of string: the key to solving the conundrum of software estimation.”</em></strong></p><p><strong>What will you learn:</strong> Whilst we can use a wide variety of software estimation techniques to help us with this difficult area of software development, the real key to success is building trust with our product teams. I’ve not spoken at a conference before, so I am excited about this opportunity after having spoken internally many times.</p><p><strong>Speaker:</strong> <a href="https://medium.com/u/5466ed0958f8">Gayathri Thiyagarajan</a></p><p><strong>Conference:</strong> <a href="https://skillsmatter.com/conferences/11982-con-london-2019-the-conference-on-microservices-ddd-and-software-architecture#overview">MuCon</a>, London, 29th-31st May</p><p><strong>Talk Title:</strong> <strong><em>“Adopting Domain Driven Design at Scale”</em></strong></p><p><strong>What will you learn:</strong> Domain Driven Design is much talked about these days particularly with the rise of Microservices, CQRS, Event Based architectures. This talk is about the story of a large scale DDD implementation that went wrong and steps to turn it into a success.</p><p><strong>Where you might have seen Gayathri speaking before:</strong></p><p>· Devoxx 2016 — Harnessing Domain Driven Design in Distributed Systems</p><p>· Devoxx 2017 — Distributing Memory in Mircoservices environment</p><p>· DDD Xchange 2018 — Model thinking as a way of working</p><p><strong>Speaker:</strong> <a href="https://medium.com/u/a008b2a3f60e">Jack Shirazi</a></p><p><strong>Conference:</strong> Devoxx London, 8th — 10th May</p><p><strong>Talk Title:</strong> <strong><em>“Quickly Analysing A Heap Memory Leak” and a Byte-Size talk “The 10 Common Concurrency Models”.</em></strong></p><p><strong>What will you learn:</strong> In this Tools-in-action talk, the Memory Leak analysis will show developers the set of tools to use and the procedure to follow to quickly identify what is causing a memory leak. The Concurrency Models talk will explain why there are all these high-level concurrency models and will help developers understand their options in choosing a model.</p><p><strong>Where you might have seen Jack speaking before:</strong></p><p>My previous recent talks include “Become A Guru: How To Solve Java Memory Leaks In Under 10 Minutes” and “Don’t Make it a Race: The Four Common Concurrency Control Patterns”, both given at JaxLondon 2018 and Java2Days 2018; “How to analyze the most common performance and memory problems in Java” at Java2Days 2017; and “10,000 Java performance tips over 15 years — what did I learn?” at Devoxx 2017. That last talk has had 25 000 views <a href="https://www.youtube.com/watch?v=OYpTn0nWKR4">https://www.youtube.com/watch?v=OYpTn0nWKR4</a></p><p><strong>Speakers:</strong> <a href="https://medium.com/u/631fdd92affd">Elliot West</a> and <a href="https://linkedin.com/in/pradeepbhadani/">Pradeep Bhadani</a></p><p><strong>Conferences:</strong> <a href="https://conferences.oreilly.com/strata/strata-eu/public/schedule/detail/74154">Strat Data Conference</a>, London, 29th April — 2nd May and DataWorks, Washington, 20th-23rd May</p><p><strong>Talk Title:</strong> <strong><em>“Herding Elephants: Seamless Data access in a multi-cluster clouds”</em></strong></p><p><strong>What you will learn:</strong> Advantages/Disadvantages of multi-data lake solutions. Enabling data sharing in decentralized data platforms. The virtual data lake: Providing a unified view of data platforms in large organizations.</p><p><strong>Speakers:</strong> <a href="https://medium.com/u/731f4fc531de">Annarita De Biase</a> and <a href="https://medium.com/u/72285fc691da">Miro Barsocchi</a></p><p><strong>Conference:</strong> <a href="https://events.codemotion.com/conferences/amsterdam/2019/wp-content/themes/event/detail-talk.php?detail=11728">Codemotion</a>, Amsterdam, 3rd April</p><p><strong>Talk Title<em>: “The story of an unconventional testing solution”</em></strong></p><p><strong>What will you learn:</strong> We developed our own test framework and want to share our journey through our experience in developing and using HEAT test framework, an open source project to test REST APIs, not as monkeys.</p><p><strong>Speakers:</strong> <a href="https://medium.com/u/731f4fc531de">Annarita De Biase</a> and <a href="https://medium.com/u/72285fc691da">Miro Barsocchi</a></p><p><strong>Conference:</strong> <a href="https://events.codemotion.com/conferences/rome/2019/wp-content/themes/event/detail-talk.php?detail=11731">Codemotion</a>, Rome, 23th March</p><p><strong>Talk title:</strong> <strong><em>“How to improve your daily tech work… WITHOUT WORKING”</em></strong></p><p><strong>What will you learn:</strong> Do you really think that you can improve your daily tech work only by studying and developing 24/7? Without sleeping, without meeting boys and girls and without having a real ‘not purely nerdish’ life?</p><p>Let’s try to think out of the box, the key could be different!</p><p>We will climb mountains and make blankets with crochet, we will play piano and fight Federer on a tennis court (ok.. maybe not this…). We will act in comedies and many other things! We’ll show you how to use those kinds of experiences in your tech daily work.. to be tech-cool!!!</p><p><strong>Speakers:</strong> <a href="https://medium.com/u/73f38766a45c">Angela Distratis</a> and <a href="https://medium.com/u/1882bf3cf377">Luca Pelosi</a></p><p><strong>Conference:</strong> <a href="https://events.codemotion.com/conferences/rome/2019/wp-content/themes/event/detail-talk.php?detail=11813">Codemotion</a>, Rome, 22nd March</p><p><strong>Talk Title:</strong> “<strong><em>Karate”</em></strong></p><p><strong>What will you learn:</strong> GraphQL is the alternative to REST in Web API design. Are the existing test frameworks ready for this revolution? This is where Karate comes in handy: an open source test framework, highly expressive, which allows to write tests quickly and in a simple way, reducing the code up to 80%. In this talk we’ll explore the main features of Karate with practical examples, comparing it with the most used testing solution available today.</p><p><strong>Speakers:</strong> <a href="https://medium.com/u/631fdd92affd">Elliot West</a> and <a href="https://medium.com/u/4b87ad35c782">Jay Green-Stevens</a></p><p><strong>Conference:</strong> <a href="https://dataworkssummit.com/barcelona-2019/session/mutant-tests-too-the-sql/">Data Works Summit</a>, Barcelona, March 18th-21st</p><p><strong>Talk Title:</strong> “<strong><em>Mutant Tests Too: The SQL”</em></strong></p><p><strong>What will you learn:</strong> The big data platforms of many organisations are underpinned by a technology that is soon to celebrate its 45th birthday: SQL. This industry stalwart is applied in a multitude of critical points in business data flows; the results that these processes generate may significantly influence business and financial decision making. However, the SQL ecosystem has been overlooked and ignored by more recent innovations in the field of software engineering best practices such as fine grained automated testing and code quality metrics. This exposes organisations to poor application maintainability, high bug rates, and ultimately corporate risk.</p><p>We present the work we’ve been doing at Hotels.com to address these issues by bringing some advanced software engineering practices and open source tools to the realm of Apache Hive SQL. We first define the relevance of such approaches and demonstrate how automated testing can be applied to Hive SQL using HiveRunner, a JUnit based testing framework. We next consider how best to structure Hive queries to yield meaningful test scenarios that are maintainable and performant. Finally, we demonstrate how test coverage reports can highlight areas of risk in SQL codebases and weaknesses in the testing process. We do this using Mutant Swarm, an open source mutation testing tool for SQL languages developed by Hotels.com that can deliver insights similar to those produced by Java focused tools such as Jacoco and PIT.</p><p><strong>Speakers: </strong><a href="https://medium.com/u/f9ed76fbd13a">Neeraj Bhadani</a></p><p><strong>Conferences:</strong> Tech(K)now Day, London, 9th March</p><p><strong>Talk Title:</strong> “<strong><em>Data Analysis and Machine Learning using Apache Spark”</em></strong></p><p><strong>What you will learn:</strong> Apache Spark is General purpose computing engine which has in-memory computing capabilities. It can be used for variety of workloads like Batch processing, Iterative problems, stream processing etc. It is designed to be highly scalable and provides various APIs like Scala, Python, R, Java and SQL. It can be easily integrated with other BIG Data tools as well. In This workshop we will cover following topics: Spark CORE (RDDs), Spark SQL (DataFrames) and Spark ML.</p><p>If you attend one of these conferences make sure you check out these talks and we would love to hear your thoughts! Please comment below 😊</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mi0HoElDaoUnTUJaXciRNQ.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43661e8b81d3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/hotels-com-speakers-at-conferences-43661e8b81d3">Hotels.com speakers at conferences</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When life gives you bash, make leMonad]]></title>
            <link>https://medium.com/hotels-com-technology/when-life-gives-you-bash-make-lemonad-b7b2993563ff?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/b7b2993563ff</guid>
            <category><![CDATA[functional-programming]]></category>
            <category><![CDATA[bash]]></category>
            <category><![CDATA[lemonade]]></category>
            <dc:creator><![CDATA[Daniel David Kovacs]]></dc:creator>
            <pubDate>Thu, 28 Mar 2019 14:01:01 GMT</pubDate>
            <atom:updated>2019-03-28T14:01:00.968Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How fake functional programming turns scripts more readable</em></p><h3>Why Bash?</h3><p>Why not bash? I cannot vouch for the practicality of this choice. Not even in the context of dockerized tools, where its portability is guaranteed by packaging it with a hand picked runtime that promises compatibility. Or exactly because of this. A node.js, or a python script would prove equally trivial to execute. Also both provide fairly high level abstractions, that are easy to deal with. Yet I chose bash; maybe it’s because it is idiomatic to compose short and sweet single purpose bits, that do one thing well; or maybe it’s because of my attraction to the weird an unusual.</p><p>Either way, instead of dwelling on the why, consider this a given and instead let’s discuss how to put lipstick on a pig!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YtNVBsoviw-aY5zNAk1ScQ.png" /><figcaption>artist’s rendering of <a href="https://tenor.com/view/miss-piggy-ms-piggy-gif-7619915">https://tenor.com/view/miss-piggy-ms-piggy-gif-7619915</a></figcaption></figure><h3>Dummy Example</h3><p>Let’s start with a simple dummy example, and iterate on it. Let’s write a script, that</p><ol><li>checks out a git repository</li><li>renders mustache templates in certain directories</li><li>commits and pushes the changes to a different branch</li></ol><p>What’s the use of it? Nothing, but it has enough moving parts to demonstrate the point. It has steps that don’t trivially succeed, and each of the steps strictly depend on the prevous one.</p><h3>What’s Wrong with Spagetti?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U9CYACwDc-ziveEIHRFhNw.png" /><figcaption>artist’s rendering of <a href="https://giphy.com/gifs/buster-keaton-spaghetti-XylffH52LeiA0">https://giphy.com/gifs/buster-keaton-spaghetti-XylffH52LeiA0</a></figcaption></figure><p>Nothing, if it’s short and concise. Except that spagetti is not supposed to be short; seriously, don’t break it in half , it’ll just slide into the pot! But I digress.</p><pre>repo=$1 <br>path=$2</pre><pre>git clone $repo</pre><pre>for path in $(find &quot;$repo/$path&quot; -type f -name &quot;*.mustache&quot;); do <br>  cat $path | mo &gt; ${path%.mustache} <br>done</pre><pre>git add --all <br>git commit -m &quot;@noissue applied mustache template&quot; <br>git push</pre><p>But your code won’t be this short either, because steps sometimes fail. And between them you’ll need to riddle the code with error checks and guards.</p><p>Let’s add some error checking and turn it into the promised spagetti monster:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aX3MsDxp6fBm8-8mH-nLGw.png" /><figcaption>artist’s rendering of <a href="https://imgur.com/gallery/dfsYUIB">https://imgur.com/gallery/dfsYUIB</a></figcaption></figure><pre>repo=$1 <br>path=$2 <br>branch=$3 <br>git clone $repo</pre><p><em>What if git clone fails?</em></p><pre>if [ $? -ne 0 ]; then <br>  echo &quot;cannot clone repo&quot; &gt; /dev/stderr <br>  exit 1 <br>fi <br>paths=$(find $path -type f -name &quot;*.mustache&quot;)</pre><p><em>What if there are no mustache templates?</em></p><pre>if [ -z &quot;$paths&quot; ]; then <br>  echo &quot;No templates in $path&quot; &gt; /dev/stderr <br>  exit 1 <br>fi<br>for path in $paths; do <br>  cat $path | mo &gt; ${path%.mustache} <br>done</pre><p><em>Any of the git steps may fail, let’s chain them:</em></p><pre>git add --all &amp;&amp; \ <br>  git commit -m &quot;@noissue applied template&quot; &amp;&amp; \ <br>  git push origin $branch</pre><p><em>Did it work?</em></p><pre>if [ $? -ne 0 ]; then <br>  echo &quot;cannot push changes&quot; &gt; /dev/stderr <br>  exit 1 <br>fi <br>echo DONE</pre><p>That’s quite the mouthful of error checking. Discovering dependencies among the steps also adds to the already significant cognitive load. Let’s tackle the two issues separately</p><ul><li>first orchestration</li><li>then error handling</li></ul><h3>Pipedreams</h3><p>The previous example chains some git commands with &amp;&amp;, putting short circuiting to use. If only the dependent steps could pass partial results for further processing. Enter the almighty pipe!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IhdrGf_YLRfrXPVuts3GqA.png" /><figcaption>artist’s rendering of <a href="https://gfycat.com/freethoughtfulamericancrayfish">https://gfycat.com/freethoughtfulamericancrayfish</a></figcaption></figure><p>I hear you, pipes are for asynchronous processing. Yes they are indeed. But it’s also true, that blocking on read can turn it into a synchronisation point; especially if we make sure to only pass exactly one or zero elements through.</p><p>Given the right plumbing we can make the most readablestest code ever, which bears cunning resemblence to human speach. Not really, but it’s fine:</p><pre>main() { <br>  repo=$1 <br>  path=$2 <br>  branch=$3</pre><pre>git-clone $repo | <br>  find-templates $path | <br>  apply-templates | <br>  git-commit-push $branch | <br>  echo DONE <br>}</pre><p>Functionality is trivial now. Not the inner workings, but unless it needs to be tweaked, there should be no need to understand implementation details. To make this work, git-clone needs to pass the named of the cloned directory to find-templates, which in turn needs to pass mustache template file names to apply-templates, which passes the number of new files to git-commit-push. These are details that can be dealt with separately. Result is a collection of compact bash function, each of which can be written independently in a truly bashful spirit.</p><p>Each pipe spawns a new child process; by so we minimise the side effects of previous steps, starting with a fresh copy of the original parent shell. This means, that even if we changed the directory in one step, it has no effect on the next. Neither are newly added environment variables. The exception is the stuff that gets persisted to the disk. This helps composition more, than it hinders.</p><p>I say hinders, because, for example, you cannot rely on exit codes for error propagation between steps. $? gives you the error code in the current process, the one from the parent is lost. Adding an extra if [ $? -eq 0 ]; guard has no effect, regardless of exit codes, the chain executes all the way to the end.</p><p>What we need is two channels, one to transmit success responses, one for error signals. Pipe already operates on /dev/stdout, it is trivial to use that for success signals: just echo the result. Analogously /dev/stderr could be used for error signals, which could enable fine grained error handling. In the simple case examined here, a lack of success signal is treated as error signal, and /dev/stderr is reserved for logging. It is possible to do this, as long as pipes are synchronisation points.</p><h3>leMonad: from Pipes to Flatmap</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U5rH-avmUNvWx6ow-VjWrg.png" /><figcaption>artist’s rendering of <a href="https://gameraboy1.tumblr.com/post/174606722966/legend-of-the-superheroes-1979-the-challenge">https://gameraboy1.tumblr.com/post/174606722966/legend-of-the-superheroes-1979-the-challenge</a></figcaption></figure><p>Let’s deal with errors! We want the pipe to express mapping on the success channel. We also want to apply different logic to the error channel, for example log an error, and propagate the error signal. This sounds exactly like a chain of Optional#flatmap-s in Java. Or a watered down maybe monad… a leMonad. Procrastinating on the implementation details further, let’s see an iteration on the original example, while retaining the script’s intent obvious:</p><pre>main() { <br>  repo=$1 <br>  path=$2 <br>  branch=$3</pre><pre>git-clone $repo | <br>  maybe find-templates $path | <br>  maybe apply-templates | <br>  maybe git-commit-push $branch | <br>  maybe echo DONE <br>}</pre><p>We need maybe to invoke a mapping function, if the previous stage succeeded, or skip it, and propagate the error signal further.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fkcmSMaKijdMqSplM7XbtA.png" /><figcaption>artist’s rendering of <a href="https://weheartit.com/entry/46607769">https://weheartit.com/entry/46607769</a></figcaption></figure><p>Lambdas in bash? Yes, why not. Loose (as in almost non-existent) typing of bash makes it possible, of course sans the safeguards preventing you to shoot yourself in the foot. Here’s simple implementation:</p><pre>maybe() { <br>  read line if [ -z &quot;$line&quot; ]; <br>    then echo &quot;✘ $1&quot; &gt; /dev/stderr <br>  else <br>    echo &quot;$line&quot; | $@ <br>  fi <br>}</pre><p>read blocks, so if nothing is sent down the pipe until the parent (ie the previous stage in the chain) terminates, $line remains empty. In which case we take to the error channel, and log an error as the simplest possible measure; error is also propagated implicitly, as nothing is written to /dev/stdout. Like ✘ apply-templates.</p><p>If something does come through the pipe, it is forwarded to squiggly-wiggly-snail-thing. Yup, that’s the lambda. Actually it’s all the parameters passed to maybe. $1 is invoked, because, well, it’s at the first place; the rest becomes the parameter list. In the example, $1 becomes find-templates, apply-templates, git-commit-push, and echo.</p><h3>Mr. T, foo, bar, and baz</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P8OV-0IbEsbkiN6sUw1HwQ.png" /><figcaption>artist’s rendering of <a href="https://tenor.com/view/eye-roll-mr-t-gif-3528716">https://tenor.com/view/eye-roll-mr-t-gif-3528716</a></figcaption></figure><p>Let’s add some logging. We reserved /dev/stderr for that, so all we need to do, is redirect /dev/stdout, while not breaking the pipes. Mr T to the rescue:</p><pre>log() { <br>  tee &gt;( while read line; do echo &quot;[$1] $line&quot;; done &gt; /dev/stderr ) }</pre><p>read everything from the pipe, copy to /dev/stderr with a prefix passed as parameter, and keep moving it through the pipe. For synchronization and to avoid duplication of the final output, we need to terminate tee with</p><pre>end() { <br>  tee &gt; /dev/null <br>}</pre><p>Thus the final version of main takes the form of</p><pre>main() { <br>  repo=$1 <br>  path=$2 <br>  branch=$3 git-clone $repo | log &quot;clone&quot; | <br>  maybe find-templates $path | log &quot;find&quot; | <br>  maybe apply-templates | log &quot;apply&quot; | <br>  maybe git-commit-push $branch | log &quot;commit&quot; | <br>  maybe echo DONE | log &quot;done&quot; | <br>  end <br>}</pre><p>Pipe is for asynchronous processing, it spawns a new subprocess. This is why in general the idomatic way of dealing with pipes on the consumer end is to read in a while loop. Thus the subprocess enters the loop the moment pipe is opened, and is kept alive via the loop as long as the pipe is open.</p><p>This is not what is done here: instead we synchronize the subprocess via read, which blocks until either something is written to the pipe, or it is closed. If you fail to synchronize, steps will no longer wait for input from the previous one.</p><p>If you pass multiple items through the pipe, like the result of a find, only the first result will be processed, the rest will be ignored. Hence the restriction to pipe exactly zero or one item.</p><h3>Conclusion</h3><p>I think this style produces bash scripts that are weird and certainly up to aquired taste to get used to. Yet if backed by a more robust maybe, it might prove to be a handy tool in your toolbox. Maybe not the daily toolbox, but for the one full of ikea wrenches and non-standard plugs, tucked away in the garden shed.</p><p><em>Originally published at </em><a href="http://kicsikrumpli.github.io/today-i-learned/til/bash/monad/2019/02/21/lemonad.html"><em>kicsikrumpli.github.io</em></a><em> on February 21, 2019.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7b2993563ff" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/when-life-gives-you-bash-make-lemonad-b7b2993563ff">When life gives you bash, make leMonad</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Amazon DocumentDB Review]]></title>
            <link>https://medium.com/hotels-com-technology/amazon-documentdb-review-4754e28ac5d2?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/4754e28ac5d2</guid>
            <category><![CDATA[documentdb]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[amazon]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Gianluca Della Corte]]></dc:creator>
            <pubDate>Thu, 14 Mar 2019 14:01:01 GMT</pubDate>
            <atom:updated>2019-03-14T14:01:01.009Z</atom:updated>
            <content:encoded><![CDATA[<p>On January 9th Amazon announced a new database service called <strong>Amazon DocumentDB</strong> that they described as a <em>“</em><strong><em>fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads”.</em></strong></p><h3>Is Amazon DocumentDB a real MongoDB?</h3><p>While offering a MongoDB-compatible API, DocumentDB is not running MongoDB software, but <strong><em>“Amazon DocumentDB emulates the responses that a client expects from a MongoDB server by implementing the Apache 2.0 open source MongoDB 3.6 API”</em></strong> on top of an undisclosed storage engine. From some information, it looks like it is built on top of the Aurora storage subsystem that is also used by both Aurora MySQL and Aurora PostgreSQL. In fact the following features/limitations are common to both DocumentDB and Aurora:</p><ul><li>both replicate six copies of data across three AWS Availability Zones</li><li>both have cluster size limit of 64 TB</li><li>both do not allow null characters (‘\0’ ) in strings</li><li>identifiers are limited to 63 letters for both</li><li>both persist a write-ahead log when writing</li><li>both don’t need to write full buffer page syncs</li></ul><h3>High Availability</h3><p>Amazon DocumentDB is designed for<strong> 99.99% availability</strong> and replicates six copies of your data across three AWS Availability Zones (AZs). DocumentDB availability goal is lower when you have less instances or when it is deployed in less than 3 AZs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/489/1*UV6QxvkGgy7krCWFqI6eGw.png" /><figcaption>Fig. 1: DocumentDB availability</figcaption></figure><p>An Amazon DocumentDB cluster consists of two components:</p><ul><li><strong>Cluster volume</strong>: cluster has exactly one cluster volume, which can store up to 64 TB of data.</li><li><strong>Instances</strong>: provide the processing power for the database, writing data to, and reading data from, the cluster storage volume. An Amazon DocumentDB cluster can have 0–16 instances:<br> - <strong>Primary instance: </strong>supports read and write operations and performs all data modifications to the cluster volume. Each Amazon DocumentDB cluster has one primary instance.<br> - <strong>Replica instance: </strong>supports only read operations. An Amazon DocumentDB cluster can have up to 15 replicas in addition to the primary instance.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jynBIt0MBjUhohhfr0ZaZQ.png" /><figcaption>Fig. 2: Deployment scenario</figcaption></figure><p>If the primary instance fails, an Amazon DocumentDB replica is promoted to the primary instance. There is a brief interruption during which read and write requests made to the primary instance fail with an exception. Amazon estimates this interruption is less than 120 seconds.<br>You can customise the order in which replicas are promoted to the primary instance after a failure by assigning each replica a priority, note that it is strongly suggested that replicas should be of the same instance class as the primary. It is also really important to create at least one or more Amazon DocumentDB replicas in two or more different Availability Zones, in this way your datastore can survive a zone failure.</p><h3>Scalability &amp; Replication</h3><p>By placing replica instances in separate Availability Zones, it is possible to scale reads and increase cluster availability.</p><p><strong>Compute and storage scale independently.</strong> It is possible to scale reads by deploying additional replicas. Scalability and storage are scalable up-to 64TB. DocumentDB automatically adds 10GB whenever it reaches capacity.</p><p>DocumentDB is also able to automatically fail over to a read replica in the event of a failure–typically in less than 30 seconds. Currently Amazon DocumentDB doesn’t support any kind of multi-region setup.</p><p>Amazon DocumentDB does not rely on replicating data to multiple instances to achieve durability, data is durable whether it contains a single instance or 15 instances.<br>All writes are processed by the primary instance that executes a durable write to the cluster volume. It then replicates the state of that write (not the data) to each active replica. Writes to an Amazon DocumentDB cluster are atomic within a single document.</p><h3>Consistency</h3><p><strong>Reads from Amazon DocumentDB replicas are eventually consistent</strong> with minimal replica lag (AWS says usually less than 100 milliseconds) after the primary instance writes the data:</p><ul><li>reads from an Amazon DocumentDB cluster’s primary instance have read-after-write consistency</li><li>reads from a read replica have eventual consistency</li></ul><p>It is possible to modify the <strong>read consistency level</strong> by specifying the read preference for the request or connection (it supports all <a href="https://docs.mongodb.com/v3.6/core/read-preference/#read-preference-modes">MongoDB read preferences</a>):</p><ul><li>primary: reads are always routed to the primary instance</li><li>primaryPreferred: routes reads to the primary instance under normal operation, in case of failover a replica is used</li><li>secondary: reads are only routed to a replica, never the primary instance</li><li>secondaryPreferred: reads are routed to a read replica when one or more replicas are active. If there are no active replica instances in a cluster, the read request is routed to the primary instance</li><li>nearest: read preference routes reads based solely on the measured latency between the client and all instances in the Amazon DocumentDB cluster</li></ul><h3>Operations</h3><p>It is possible to create an AWS DocumentDB cluster using CloudFormation stack (as described <a href="https://docs.aws.amazon.com/documentdb/latest/developerguide/quick-start-with-cloud-formation.html">here</a>).</p><p>Amazon DocumentDB is a fully managed solution that provides the following features:</p><ul><li>auto scaling storage (up to 64 TB in 10GB increments)</li><li>simple compute resource scaling (resources allocated to an instance can be modified by changing instance class)</li><li>built-in monitoring, fault detection, and failover</li><li>daily snapshots</li></ul><h3>AWS DocumentDB vs AWS ElasticSearch</h3><p>DocumentDB and ElasticSearch have a lot of features in common, in fact you could even use ElasticSearch as a primary datastore. Some of the features they have in common are:</p><ul><li>document oriented store</li><li>schema-free</li><li>distributed data storage</li><li>high-availability</li><li>replication</li></ul><p>However, they come from 2 different database families and are made for different purposes. DocumentDB is a <strong><em>document store</em></strong> while ElasticSearch is a <strong><em>search</em></strong> <strong><em>engine.</em></strong></p><p>Here are some key differences between the two:</p><ol><li>Indexing — ElasticSearch uses Apache Lucene for indexing while MongoDB indexes are based on traditional B+ Tree. Real-time indexing and searching power of ElasticSearch comes from Lucene, which allows creation of indexes on every field of a document by default. In MongoDB, we have to define the index, which improves query performance, but affects write operations.</li><li>Writing — ElasticSearch is slower on adding new data. In ElasticSearch indexing semantics are defined on client side. Indexing cannot be optimised as well as with DocumentDB.</li></ol><p>In practice, ElasticSearch is often used together with NoSQL and SQL databases. A datastore is used as persistent storage and source of truth, and ElasticSearch is used for doing complex search queries.</p><p>Another key consideration while evaluating DocumentDB vs ElasticSearch is the effort/complexity associated with an ElasticSearch domains definition, sizing and maintenance. It is not so straightforward to do it (in fact it is hard to correctly size storage, shards and instance size). AWS provides some good <a href="https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/sizing-domains.html">guidelines</a>, but it is more complex than working with DocumentDB which doesn’t require these considerations.</p><h3>Hotels.com Architecture team’s advice</h3><p>Currently in Hotels.com we use many different datastores/search engines, so it is good to summarise our advice on when Amazon DocumentDB is or is not a good option.</p><p>Amazon DocumentDB is a good solution when you need to store unstructured data that doesn’t require too many indexes or complex search features. <br>A good benefit is that you don’t need to care too much about queries upfront. This is particularly useful when you are not the owner/producer of the data you are storing, so you don’t need to adapt your schema to a possible new data structure (like you must do with a SQL database like Amazon Aurora) and you can query data also using new fields (thing that you cannot easily do using another NoSQL solution like Amazon DynamoDB, where your data schema is based on your queries).</p><p>It is also a good solution when you don’t need rich indexing capabilities and complex/fast search support (ranked results, full text search with partial matching without using regex, complex geospatial queries with inclusion/exclusion). For these kind of scenarios Amazon ElasticSearch is a better choice.</p><p>Currently Amazon DocumentDB has two big drawbacks:</p><ul><li>no multi-region support</li><li>just provisioned mode (not available in serverless mode)</li></ul><h3>References</h3><ul><li><a href="https://aws.amazon.com/blogs/aws/new-amazon-documentdb-with-mongodb-compatibility-fast-scalable-and-highly-available/">https://aws.amazon.com/blogs/aws/new-amazon-documentdb-with-mongodb-compatibility-fast-scalable-and-highly-available/</a></li><li><a href="https://aws.amazon.com/documentdb/">https://aws.amazon.com/documentdb/</a></li><li><a href="https://aws.amazon.com/documentdb/pricing/">https://aws.amazon.com/documentdb/pricing/</a></li><li><a href="https://www.infoq.com/news/2019/01/aws-documentdb-mongodb">https://www.infoq.com/news/2019/01/aws-documentdb-mongodb</a></li><li><a href="https://aws.amazon.com/blogs/database/introducing-the-aurora-storage-engine/">https://aws.amazon.com/blogs/database/introducing-the-aurora-storage-engine/</a></li><li><a href="https://docs.aws.amazon.com/documentdb/latest/developerguide/quick-start-with-cloud-formation.html">https://docs.aws.amazon.com/documentdb/latest/developerguide/quick-start-with-cloud-formation.html</a></li><li><a href="https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/what-is-amazon-elasticsearch-service.html">https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/what-is-amazon-elasticsearch-service.html</a></li><li><a href="https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/sizing-domains.html">https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/sizing-domains.html</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4754e28ac5d2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/amazon-documentdb-review-4754e28ac5d2">Amazon DocumentDB Review</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scala Central at Expedia Group]]></title>
            <link>https://medium.com/hotels-com-technology/scala-central-at-expedia-group-3a8002c6bb60?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/3a8002c6bb60</guid>
            <category><![CDATA[london]]></category>
            <category><![CDATA[functional-programming]]></category>
            <category><![CDATA[scala-central]]></category>
            <category><![CDATA[scala]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Ryan Burke]]></dc:creator>
            <pubDate>Thu, 07 Mar 2019 14:01:01 GMT</pubDate>
            <atom:updated>2019-03-07T14:01:00.980Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*qOtywhu7rDDRkJkoKU6hpg.jpeg" /></figure><p>On Wednesday 30th January we hosted the <a href="https://www.meetup.com/Scala-Central/">Scala Central meet-up</a> at the Expedia Group offices in Angel, London.</p><p>Scala Central has been running for nearly 3 years now and is an incredible source of Scala knowledge and talent. Hotels.com have several teams writing applications in Scala so it was a perfect match for us to host.</p><p>The evening began with drinks and networking before the main talks began.</p><h4>Greg Dorrell – Effect Types: What? Why? Which one?</h4><p>Greg is a former Hotels.com employee and has since moved to another Expedia Group company, HomeAway. To summarise his talk, Scala Future is side effect producing as when it is created it immediately performs the operation within it. <a href="https://typelevel.org/cats-effect/datatypes/io.html">Cats IO</a>, <a href="https://github.com/scalaz/scalaz-zio">ScalaZ IO</a> and <a href="https://monix.io/docs/3x/eval/task.html">Monix Task</a> are all alternatives which provide similar functionality but lazily evaluate the operation. This means the act of execution can be separated from the definition.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FAEScmarqa80%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DAEScmarqa80&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FAEScmarqa80%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/11e3b6ad520b1285c3d0b480f09ebcd6/href">https://medium.com/media/11e3b6ad520b1285c3d0b480f09ebcd6/href</a></iframe><h4>Leszek Sliwko – Drone Delivery Queue</h4><p>Leszek followed with his talk on his experience of release/build processes and how this was set up in his team. His talk was not only relatable to Scala Developers, but could also be applicable for those working with other technologies too.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F8ARNNArtwW0%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D8ARNNArtwW0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F8ARNNArtwW0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/51c99fbf87ed4b68d6d26efa9faff29f/href">https://medium.com/media/51c99fbf87ed4b68d6d26efa9faff29f/href</a></iframe><p>Our annual Scala meetup was really excellent and we love supporting the Scala community here at <a href="https://hotels.com/">Hotels.com</a> .</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3a8002c6bb60" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/scala-central-at-expedia-group-3a8002c6bb60">Scala Central at Expedia Group</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Three things I learnt being a scrum master]]></title>
            <link>https://medium.com/hotels-com-technology/three-things-i-learnt-being-a-scrum-master-7af04752b764?source=rss----29989b59e37d---4</link>
            <guid isPermaLink="false">https://medium.com/p/7af04752b764</guid>
            <category><![CDATA[agile]]></category>
            <category><![CDATA[scrum]]></category>
            <category><![CDATA[scrum-master]]></category>
            <category><![CDATA[kanban]]></category>
            <category><![CDATA[agile-coach]]></category>
            <dc:creator><![CDATA[Giuseppe Sorrentino]]></dc:creator>
            <pubDate>Thu, 21 Feb 2019 14:01:01 GMT</pubDate>
            <atom:updated>2019-02-21T14:01:01.087Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X6V1XYO90ZtqoSPssk132A.png" /></figure><h3>Introduction</h3><p>I am very happy to have had the opportunity to work in the Agile world for almost 4 years, that have been fantastic and challenging.</p><p>Being a Scrum master is an invaluable experience and makes you understand and reflect a lot about company processes and software development in general.</p><p>It is very hard to discover and address disfunctionalities in teams’ processes. In fact, disfunctionalities are often sneaky. Metrics and surveys can help you but you need to develop an insight to recognize them and this helps you improve a lot as person and professional.</p><p>I decided to share with you three thoughts I noted down in these years.</p><h3>1. Training is not enough, make it real by being assertive (when necessary)</h3><p>In these four years I did tons of training. Prepared tons of presentations on the various agile practices and artifacts: Kanban, Scrum, backlog and backlog refinement, pair programming are only examples.</p><p>One thing I learnt is that while training on agile is valuable, practice is more valuable. The capacity toward making practices real in day to day life is fundamental in the scrum master profession. In order to do that there are two different and antithetic approaches:</p><ul><li>wait that a practice emerges in the team</li><li>be assertive and effectively contribute by pushing for the application toward some beneficial practices.</li></ul><p>Being able to find the right balance between these two approaches is a fundamental key in a scrum master role. In a perfect world the Scrum master would always choose the first approach. But in the real world, this is not always feasible. For example, there could be situations where it is not possible to wait until the team becomes mature enough to adopt a practice. On these occasions, in my honest opinion, is when the Scrum master needs to be assertive.</p><h3>2. If you want to go with Kanban, start with Scrum</h3><p><em>I am assuming you are familiar with the </em><a href="https://en.wikipedia.org/wiki/Tuckman%27s_stages_of_group_development"><em>Tuckman’s stages of group development</em></a><em> here.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/0*ccD6_dJIKjUSO4my" /><figcaption>The <a href="https://en.wikipedia.org/wiki/Tuckman%27s_stages_of_group_development"><em>Tuckman’s stages of group development</em></a></figcaption></figure><p>It is harder to start directly with Kanban than starting with Scrum and transitioning to Kanban. In fact, Kanban requires much more discipline from the team than scrum. Pulling stories at the right time, limiting the amount of work-in-progress items, are very challenging tasks, even for a very small group of people. This makes Kanban more functional in the teams that are in the norming or performing phase or however not at their beginning. While scrum being more prescriptive, is perfect for a team in the forming and storming phase.</p><p>It is a good idea to start with Scrum and transition smoothly to Kanban when you feel the team is ready, or rather when the team is entering in the norming/performing phase. There are many indicators a team is transitioning toward the norming/performing phase:</p><ul><li>stability in practices adopted</li><li>stability in team composition</li><li>continuous success of sprints</li><li>self-organization in main scrum ceremonies</li><li>stability in velocity and throughput.</li></ul><h3>3. Scrum application outside the software world often is not clear</h3><p>While scrum is supposed to be an universal framework, in the sense it should be applicable outside of software world, this application is not always immediately clear.</p><p>In <a href="http://hotels.com/">Hotels.com</a> we give training on Agile to very different functions and we encountered difficulties in recognizing a way to apply scrum to certain realities outside of technology. For example there is not so much literature on how backlog items should be documented. Neither is clear how to manage realities where we have mostly personal work rather than team work.</p><h3>Conclusion</h3><p>I had four challenging years as Scrum master and this opportunity make me grow as person as well as IT professional. During these years I had the opportunity to reflect on some aspect of the Scrum master practices.</p><p>Particularly I discovered that the Scrum master need to be assertive and effectively contribute by pushing for the application toward some beneficial practices when necessary. The natural emergence of all the team practices is simply a Scrum myth.</p><p>I, furthermore, think that Starting directly with Kanban for a new team can be counterproductive. My suggestion here is to evaluate Scrum as bootstrap for Kanban.</p><p>The last point: the fact that Scrum universality (its application outside of IT projects) is not crystal clear. Under this point of view a great community effort to make Scrum more accessible is needed.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7af04752b764" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hotels-com-technology/three-things-i-learnt-being-a-scrum-master-7af04752b764">Three things I learnt being a scrum master</a> was originally published in <a href="https://medium.com/hotels-com-technology">The Hotels.com Technology Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>