<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[FourScouts - Medium]]></title>
        <description><![CDATA[FourScouts Blog - Medium]]></description>
        <link>https://medium.com/fourscouts?source=rss----ad181d2289c3---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>FourScouts - Medium</title>
            <link>https://medium.com/fourscouts?source=rss----ad181d2289c3---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 01:06:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/fourscouts" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Leaving the mob will get you in trouble]]></title>
            <link>https://medium.com/fourscouts/leaving-the-mob-will-get-you-in-trouble-258f5402f58f?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/258f5402f58f</guid>
            <category><![CDATA[mob-programming]]></category>
            <category><![CDATA[teamwork]]></category>
            <dc:creator><![CDATA[Arno van Rossum]]></dc:creator>
            <pubDate>Thu, 01 Oct 2020 13:22:00 GMT</pubDate>
            <atom:updated>2020-10-01T13:21:59.965Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*GIe_wk81ODeL2XYub0gc8g.jpeg" /></figure><p>After an awesome workshop by <a href="https://twitter.com/WoodyZuill">Woody Zuill</a> on mob programming, we wound up repeating old patterns. This is a short story about implementing mob programming.</p><p>We were already familiar with pair programming and consider this the minimum way of programming, 2 minds always beat 1 mind. 4 minds beating 2 minds makes even more sense, so we thought a workshop on mob programming could help us. We were right, Woody showed us what it is like to build the right thing in a short amount of time and thus feeling part of a highly efficient team.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZXSFqpGjkszdSB7o.jpg" /></figure><h3>Mob programming rules</h3><p>The workshop had different setups each time, with different rules. We started with 1 driver (the guy with the keyboard), 1 navigator (tells the driver what to do), the rest observers (only allowed to think), where only the navigator may speak.</p><p>As an outspoken, opinionated, and loud person, this was unquestionably the hardest setup for me. Only observing and not being able to blurt out everything forces you to watch your team members figure the problems out and even get to the same or better conclusions. This thought me I should listen more and trust team members to fix the problem.</p><p>The next setup was 1 driver, 1 navigator, the rest observers, the navigator allows observers to speak if he chooses to. This brought a new vibe. Having observers help the navigator on request changes the game when being stuck. In the previous setup, you are just stuck for that round and telling the driver to google, this format made more sense.</p><p>However, the last setup was the best, 1 driver and the rest navigators. Navigators are having a continuous discussion on solving a specific problem, whereas not being bothered by implementation details like typing. The best part is you share the cognitive load of the problem and having a team consensus on the direction of the solution. Having experienced the previous steps is useful, as they teach you to be more silent and only speak when you have something to say.</p><h3>Our setup</h3><p>A few days after the workshop, we changed our office, moving around tables, chairs, screens, etc to do mob programming. Tables ready, tv ready, 4 keyboards, 4 mouses, 1 machine, we are ready!</p><figure><img alt="Our setup" src="https://cdn-images-1.medium.com/proxy/1*CNb3ou9igp1r8iebjoyZ4Q.jpeg" /></figure><p>And so we did, we worked on the product and mob programming worked! Better quality code, better communication, shared understanding, and shared solutions.</p><h3>Slippery Slope</h3><p>The plan was to onboard customers in our software within 3 weeks. This didn’t mind us as we do continuous deployment and are always production-ready. We kept working in the same manner and all seemed fine, but after a conversation with the customer, the storm dawned upon us, the domain implementation is incorrect.</p><p>And so, a deadline has arrived …..</p><figure><img alt="Deadline is coming" src="https://cdn-images-1.medium.com/proxy/1*kPd-qP8zlnPuFrBWPf8W1w.png" /></figure><p>For the first week we kept working with mob programming, but the stress and time constraints tore us apart. The first cracks in our mob programming sessions became apparent.</p><figure><img alt="Slippery slope" src="https://cdn-images-1.medium.com/proxy/1*7y-G52fqkFSh8xezspUk8Q.jpeg" /></figure><p>Because the number of problems seem to grow, we neglected to align the team on 1 problem at a time, and this moved us to look at our own screens more often.</p><p>When you look at your screen, the team discussions automatically falls short, the whiteboard became uninteresting, timeboxing stopped, no more swapping drivers, and lead to stopping mob programming.</p><p>And because of the time pressure, you cut corners, and when nobody corrects; Code quality decreased, test coverage decreased, technical debt increased, integrating code with peers went horrible, and the overall system became a mess. Sadly, you get away with this unacceptable behavior.</p><p>Before the deadline, my colleagues worked hard and told me the system worked as they did a last test of the system at 1:30 AM.</p><h3>Judgment day</h3><p>I had to present the product and walk the customer through the process. In the morning before the demo, I tried everything myself, and…. Ah shit! The software barely worked….<br>By avoiding most of the problems and putting parts of the product more to a conversational setting, there was enough to get away with. The demo ended up being embarrassing and not something to be proud of.</p><p>If we stuck to the plan, kept on mob programming, it would have been high-quality workable product and something proper to show for. Perhaps we would also deliver more, as broken functionality is <strong>no</strong> functionality.<br>Now we needed to dig into broken functionality, pay off technical debt, and improve code quality.</p><p>So if the times get tough, keep the team conversation going, do mob programming and solve 1 problem at the time!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=258f5402f58f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/leaving-the-mob-will-get-you-in-trouble-258f5402f58f">Leaving the mob will get you in trouble</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The bearing of a child takes nine months, no matter how many women are assigned]]></title>
            <link>https://medium.com/fourscouts/the-bearing-of-a-child-takes-nine-months-no-matter-how-many-women-are-assigned-2d31a71b9ab5?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/2d31a71b9ab5</guid>
            <category><![CDATA[team]]></category>
            <category><![CDATA[teamwork]]></category>
            <category><![CDATA[colaboration]]></category>
            <dc:creator><![CDATA[Arno van Rossum]]></dc:creator>
            <pubDate>Thu, 17 Sep 2020 10:39:29 GMT</pubDate>
            <atom:updated>2020-09-17T10:39:29.383Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vXwYWc9U5_pYlJgQV-Myqw.jpeg" /></figure><p>I quote the title from the book Mythical Man-Month written by Fred Brooks and originated from Theodore von Kármán in the unique form “Everyone knows it takes a woman nine months to have a baby. But you Americans think if you get nine women pregnant, you can have a baby in a month.” Fred Brooks made this a very popular statement in the software industry, with good reasons. The idea behind it is that adding manpower to a late software project makes it later, also called brooks’s law.</p><p>This is a story about where I have seen this happen and what the result became to be.</p><h3>First some context</h3><p>I started working at a company as a consultant, once started I figured this to be the most political company I have seen so far. They wanted to create a product where some stakeholders had political gain to make this product fail. They did this by holding many people accountable, but not themselves. They made sure people commit to pointless deadlines on unnecessary functionality, and it was rather impossible to adhere to the agile values. They discarded any discussion on these subjects.</p><h3>Build time!</h3><p>While the backlog was filling with non-negotiable stories, the developers started working. Slowly but surely we delivered functionality, and the product took shape and became more and more useful. But soon we realized the frontend flow had problems with the overall concept of how the product should work. So the team worked on an alternative design which the stakeholders approved.</p><h3>A new frontend</h3><p>The original idea was to work with the same team and slowly push new functionality in the new frontend. This worked for a little while, but for stakeholders, this was not good enough, implementation must be faster as the imaginary deadline was closing in. As the deadline or functionality was non-negotiable it made sense to rack up the number of developers in the team. So they make the assumption that frontend is the bottleneck and that we need a lot of help, so management attracted 3 extra frontend developers.</p><p>The team of 3 suddenly became a new team of 6.</p><h3>What changed?</h3><p>This is where brooks law comes into play “adding manpower to a late software project makes it later”</p><h3>Ramp-up time</h3><p>Brook states it takes time for people to become productive in a new team, so-called ramp-up time. This extra worker needs time to get to know their colleagues, get acquainted with the domain, need to learn the code base, what the team culture is, what we expect of him, etc.</p><p>So when the company added 3 new frontenders, the other developers in the team were no longer working, but facilitating them to be productive. So we gained 3 developers but also lost 3 developers with a sudden priority shift to frontend, instead of deliverable functionality.</p><h3>Communication overhead increases as the number of people increases.</h3><p>One of the constant complaints was not knowing what the other developers within the team were doing. When you add 3 new people to an existing team of 3, the communication lines went from 6 to 15.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SVnM13ASRg2kDJYG.png" /></figure><p>This leads to a loss of time in keeping in sync with all team members.</p><p>We could have solved this with pairing or even mob programming, however, because the team duplicated this culture was no longer in place.</p><h3>Adding more people to a highly divisible task decreases the overall task duration.</h3><p>The problem, however, most of the stories were not divisible, so it ended up with more communication on being in sync with whom touches what piece of code.</p><h3>Anything else?</h3><p>Because they doubled the team, a culture shift happened and led to discussions the team already been through. For example, on Trunk based development, code reviews, continuous deployment and the definition of done.</p><p>Because they doubled the team, a culture shift happened and led to past discussions on Trunk based development, code reviews, continuous deployment and the definition of done.</p><h3>And then everything broke</h3><p>Because the team discipline was mostly frontend, it made sense to some to split up tasks for front and backend work. Which shifted away focus from delivering functionality, to finish their frontend task.<br>It ended up with frontenders creating a beautifully “working” application based on fake APIs, and they were happy, it looked great!<br>However, when the backenders try to implement the APIs based on frontend work, it became apparent that the fake APIs did not fit the existing domain model at all. We ended up with an application that looked great but had no working functionality and therefore a useless application.</p><p>If you want to grow a team, then make sure you do it for the right reasons. Brooks was right. Making a deadline isn’t one of them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d31a71b9ab5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/the-bearing-of-a-child-takes-nine-months-no-matter-how-many-women-are-assigned-2d31a71b9ab5">The bearing of a child takes nine months, no matter how many women are assigned</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reclaiming ownership in the development team]]></title>
            <link>https://medium.com/fourscouts/reclaiming-ownership-in-the-development-team-e74ec19c73e6?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/e74ec19c73e6</guid>
            <category><![CDATA[trunk-based-development]]></category>
            <category><![CDATA[release-management]]></category>
            <category><![CDATA[digital-transformation]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[team]]></category>
            <dc:creator><![CDATA[Henrique Miranda]]></dc:creator>
            <pubDate>Mon, 31 Aug 2020 07:32:02 GMT</pubDate>
            <atom:updated>2020-08-31T07:18:41.775Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Puv0mgYE_C_-Rip1vbqFeA.jpeg" /></figure><p>My previous post (<a href="https://blog.fourscouts.nl/how-to-fix-it-being-the-bottleneck">How to fix it being the bottleneck</a>) explained how we started a small transformation of a development team using Trunk based development. The post triggered some people around me to explain more about how they can apply that on their own team.</p><p><strong>A bit of my history with software development</strong></p><p>I remember my first days as a developer building a website/application in a team with 6, with no sort of version control system, just changing the files directly into the production server via Dreamweaver. There were no tests or any quality control. What a time :D</p><p>Because of that, we introduced a lot of problems which disrupted the daily work of many people that depended on that software to do their work.</p><p>Our manager grew worried about all that, his job was on the line and he did what a lot of us would do. At least considered doing it.</p><ol><li>Hired a tester to make sure this problem would not happen again.</li><li>All the code needed approval by Quality Assurance before going live.</li></ol><p>That is a logical decision for that kind of problem. The wrong one, though.</p><p><strong>Why might that be the wrong approach?</strong></p><p>It is very hard to foresee the meaningful impact of this change. As our environment has changed, our behavior changed slowly which also shaped our mindset.</p><p>The positive side of it. Fewer bugs were reaching production. That is a big win that nobody can deny.</p><p>The negative side of it? Glad you asked.</p><p>Even though fewer bugs were reaching production, more bugs were reaching the QA. Now we also had a list of bugs (some refer now to a backlog of bugs). Our tester was becoming the bottleneck to bring code to production.</p><p>Now think for a second. How would you solve this additional problem? The way we solved, we hired another tester. Because that makes sense!</p><p>With a second tester, the “bug backlog” never got cleared, and the input never slowed down. Very much the opposite. As an addition, we started having endless discussions about how many QA people we need compared to the numbers of developers.</p><p><strong>Another way of solving this problem?</strong></p><p>Fast forward a bit, I moved to a new employer that had the development process more mature and based on “best practices”.</p><p>Those guys used Git (a version control system) with GitFlow (is a branching model for Git,) and did code review in every commit. When the team explained to me all the benefits of this approach, they sold me. The way to Valhalla.</p><p>But then, I started noticing several other problems that started irritating me.</p><ol><li>For very unimportant changes that I was 100% sure that could not affect production, I still needed review and approval from someone. I had to bother people to review often. What resulted in three outcomes.</li></ol><ul><li>People were accepting my changes without looking at them, so they could go back to do what they were doing.</li><li>I was packing a lot of changes in one request.</li><li>It was still not preventing bugs</li></ul><ol><li>Another problem I saw was merge conflicts. The developers were spending a lot of time fixing those conflicts. I am sure you know a few “solutions” for this problem. You might think:</li></ol><ul><li>Did you try committing in smaller batches?</li><li>Did you try organizing work so developers would not touch the same part of the system?</li></ul><p><em>We haven’t touched the biggest problem that hides in all of those solutions. The delay to bring any solution to production. But that will be another post.</em></p><p><strong>How you should solve the problem</strong></p><p>Trunk Based Development is a simple concept. The project has only one branch. All the developers commit to that single branch.</p><p>I started doing TBD when I was working on my startup, and my Co-founder was constantly asked me to test his functionalities. I no longer wanted to be checking his code, and we did not have money for a tester. I suggested TBD for him and we gave it a shot.</p><p>The first two weeks were horrible. We did not have a magnificent set of automated tests and many bugs were going through.</p><p>But then the magic happened, our behavior changed.</p><ul><li>We read our code more often and try to test it more before committing.</li><li>We got more interested in automated testing.</li><li>We checked the production environment ourselves after every change</li><li>We got more interested in different ways to monitor the application</li></ul><p><strong>Excuses I hear for not doing Trunk Based Development</strong></p><ul><li>We don’t have automated tests</li><li>We don’t have good coverage of tests</li><li>Our developers are not in the level to do that</li><li>Our team is not ready for that</li><li>No four-eyes principle/code reviews</li></ul><p>Trunk Based Development will force you to improve in those areas, it is the environmental change that your team needs.</p><p><strong>How do you start?</strong></p><ol><li>Keep only your master/trunk/main (whatever you called it) branch</li><li>Commit/Push smaller changes.</li><li>Commit/Push often</li></ol><p>It’s that simple</p><p><strong>Last thoughts</strong></p><p>With the QA, code-reviews and pull request to protect bugs to reach production, we created a much bigger issue. We shifted the responsibility and accountability of the developer that was building the code to someone else.</p><p>We made it harder to experiment with functionalities and shifted the conversation to less important things that only made the system more rigid.</p><p>If you are a developer invite your team to try (over 2 weeks please :)), take back the control, and own your craft.</p><p>If you are a manager, give them the chance to own their own responsibilities.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e74ec19c73e6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/reclaiming-ownership-in-the-development-team-e74ec19c73e6">Reclaiming ownership in the development team</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to fix IT being the bottleneck?]]></title>
            <link>https://medium.com/fourscouts/how-to-fix-it-being-the-bottleneck-8a882e6d1d63?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/8a882e6d1d63</guid>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[mindset]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[trunk-based-development]]></category>
            <category><![CDATA[digital-transformation]]></category>
            <dc:creator><![CDATA[Henrique Miranda]]></dc:creator>
            <pubDate>Wed, 15 Jul 2020 12:24:33 GMT</pubDate>
            <atom:updated>2020-07-15T12:24:33.793Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IE1Okvo9CTcNBlO2" /><figcaption>Photo by <a href="https://unsplash.com/@amseaman?utm_source=medium&amp;utm_medium=referral">Andrew Seaman</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>A bit more than 4 years ago I decided to kill my startup on which <a href="https://medium.com/startup-foundation-stories/only-if-i-had-listened-fe1a5764a3c7">I wrote a postmortem </a>(I think it is part of the grieving process, a lot of founders do that). At that time I remember my emotions were not so positive. I was feeling ashamed, small, and hopeless.</p><p>After a few months, I decided to pursue the consultant path and help other companies to not make the same mistakes. Or as I like to think, to set them up for success.</p><p>Since then, I had the opportunity to work with companies in different markets and of different sizes, but mostly, companies that were in need to change and change fast.</p><p>This is my attempt to show the positive side of it, the huge learnings that would have taken me many years to get by working for companies. If you are trying to create your company or help your team, I hope these will resonate with you.</p><p>In this post (and more to come), I will try to explain what happened to one of our customers, what we saw there, and how we tackled it. Symptoms are:</p><h3>1. Development team never spoke to the customer</h3><p>When a feature needed to be implemented, the request always came from the CTO or sales. The project manager responsible for the product never spoke to the customer before, she was just proxying work to be done without understanding the reasons behind.</p><p>When the team had to fix a production issue, they had to talk to customer support, they were not allowed to speak with the customer directly.</p><blockquote>When asked to the team lead why they never spoke to the customer the answer was: We have a process here that makes it more efficient this way</blockquote><h3>2. Nothing gets delivered</h3><p>Priority 1 bugs took the team a full day with 4 developers to find the problem and fix it and another half day to merge code back into one of the master branches. And it could only be done by one person.</p><blockquote>We got called by their CTO saying that IT does not deliver and they are the bottleneck of the organization.</blockquote><h3>3. Fear of speaking up</h3><p>A normal standup would be the support team telling the development team how they cannot deliver any quality work and everything they do is wrong.</p><blockquote>The manager said that they don’t trust some people to do the work in front of everyone.</blockquote><h3>4. No time for improvements</h3><p>The team was constantly firefighting production issues or implementing the greatest next feature that would solve all the problems.</p><blockquote>The team would say, we have no time to improve anything, because the business wants features, features, features</blockquote><blockquote>We have no time for tests</blockquote><h3>5. Complex process</h3><p>255 branches in version control. That is a true story, not only that they had weird naming for those branches like master-v2, master-lastest, master, master-dev, development, and a bunch of feature branches.</p><blockquote>The question that triggers us is the most is. WHY? How come they don’t see this is way too complicated? Why don’t they delete all those unused branches?</blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FhSR5CiiVqO1whCtirA%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FhSR5CiiVqO1whCtirA%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FhSR5CiiVqO1whCtirA%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="327" frameborder="0" scrolling="no"><a href="https://medium.com/media/16bf44a4a4a520a57b5835225639fdf1/href">https://medium.com/media/16bf44a4a4a520a57b5835225639fdf1/href</a></iframe><h3>How did we start?</h3><p>For this specific customer, they had a clear vision and a direction and they just needed a bit of help to create an environment that would enable them to continue growing.</p><p>This is one of the problems we love to solve. You can tackle this from a lot of different angles and for this one we decided to start with so-called, <strong><em>IT being the bottleneck</em></strong> problem.</p><blockquote>Change the environment around people, it will change the way they act and consequently will change the way how they think.</blockquote><h4>How did we do that?</h4><h4><strong>1. Trunk-based development</strong></h4><p>We gave a workshop about Trunk-based development, unit tests, continuous integration/delivery/deployment.</p><p>We asked the team to pick one of the 255 branches. All the other branches would be deleted forever. We also said that everyone had to commit all the time to the same branch, no more future branches.</p><blockquote>Their reaction? <br>You are insane, this is the stupidest thing we ever heard</blockquote><h4>2. Continuous deployment to Production</h4><p>The next step was easy. We automated every commit to production 😙</p><blockquote>Reaction?<br>You cannot do that, what if it breaks? we don’t have test coverage? how is the project manager approve a functionality? They were in total disbelief</blockquote><h3>And… that was enough…</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FgghyE3bu9B9GhG6dfw%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FgghyE3bu9B9GhG6dfw%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FgghyE3bu9B9GhG6dfw%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="241" frameborder="0" scrolling="no"><a href="https://medium.com/media/b5a77a4186a945dc40cd6141f28f5896/href">https://medium.com/media/b5a77a4186a945dc40cd6141f28f5896/href</a></iframe><p>It triggered their behavior to change, which is required to change the way how they work.</p><p>What we observed was, developers were now very careful to push changes afraid of breaking production. They become more curious to learn how to write tests.</p><p>They barely had merge conflicts because we deleted all those useless and confusing branches. And when they did have conflicts, they learned more and more that you should deliver continuously in small batches</p><p>Their cognitive load became lower, they no longer needed to remember all those branches strategy they once agreed upon 🙌. Which created more space for them to improve their own way of working, have more slack, talk to customers, possibilities were endless 😝.</p><blockquote>After 3 weeks, they asked themselves. Why did we use branches in the first place?</blockquote><p>And the “business”, how did they react? That will be another post 😄. IT can be the bottleneck, but mostly is just a symptom of a bigger problem.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a882e6d1d63" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/how-to-fix-it-being-the-bottleneck-8a882e6d1d63">How to fix IT being the bottleneck?</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Replaying projections with Axon and Spring Boot]]></title>
            <link>https://medium.com/fourscouts/replaying-projections-with-axon-and-spring-boot-d5d34bebefb6?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/d5d34bebefb6</guid>
            <category><![CDATA[cqrs]]></category>
            <category><![CDATA[replay]]></category>
            <category><![CDATA[axon-framework]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Rolf Suurd]]></dc:creator>
            <pubDate>Fri, 13 Dec 2019 13:03:53 GMT</pubDate>
            <atom:updated>2019-12-13T13:03:53.452Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pnh22nBDDTDCYpxgfE9giQ.jpeg" /><figcaption>Rewinding tape so we can play it again</figcaption></figure><h3>Introduction</h3><p>Although replaying events is already documented in the <a href="https://docs.axoniq.io/reference-guide/configuring-infrastructure-components/event-processing/event-processors#replaying-events">official documentation</a>, I was struggling to have it integrated into my Spring Boot application. In this article, I would like to share with you how I trigger and keep track of a replay of my projections with Axon Framework’s Spring Boot auto configuration.</p><h3>Background</h3><p>When doing event sourcing with Axon Framework, many developers opt to project these events to the database, depending on their use case.</p><p>For example, an OrderPlaced event can add an entry to the customer’s order history, so when a customer logs into his account, they have an overview of their complete order history.</p><p>When implementing this using <a href="https://spring.io/projects/spring-data-jpa">spring-data-jpa</a> it almost looks trivial:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pa0GKxcpkKb5Rmm--7jnrQ.png" /><figcaption>Typical projections with Axon and Spring data</figcaption></figure><p>This was good enough for a long time. But after some time you find that multiple customers are requesting to also be able to see the order status in their order history. Luckily, you’ve been using OrderStatusChanged events already, so all you have to do is add a status field to your OrderHistoryItem entity and add an @EventHandler to update that with the changed status:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FeYsIqlqMQFqPUeRo96dIQ.png" /><figcaption>A new event handler to update our read model</figcaption></figure><p>And presto!✨All new status changes will update their associated history items accordingly.</p><p>But wait, what about all <em>previous </em>OrderHistoryItem entries? They have no order status! What to do? This is why replaying a projection sometimes is needed: to replay all (or part) of the events so the the projections can be made up-to-date.</p><h3>Rewind selecta</h3><p>For most use cases, you will want to replay all events from the beginning of the event stream. For that, we can reset the EventProcessor. But, we also need to reset our projections: we will be adding history items again and we don’t want customers to end up with double entries. For this, Axon offers the @ResetHandler:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/411/1*U951Bx95u3YW35khWWoMVQ.png" /><figcaption>All we need to do is delete everything in the repository 🗑️</figcaption></figure><p>Now our projections will have a clean slate when we reset them. 🛀</p><h3>Play it again, Sam</h3><p>To perform a replay (and reset), we will need to obtain the TrackingEventProcessor for our projections. These are available in the EventProcessingConfiguration, which, in our case, is available as a Spring managed bean. In order to obtain the processor, we will need to know it’s name. You can either specify it by annotating your projections with @ProcessingGroup, or use the default: the package of the projections. For example, this could be com.shop.projections.</p><p>Let’s create a @Component that is capable of replaying:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xbXRE1AfLZexPRXQbo6UOw.png" /><figcaption>Shut down before resetting — then start!</figcaption></figure><p>You can inject this component and trigger the replay either at startup of your application (for example by listening for ApplicationReady), or in some sort of controller allowing you to trigger it using a REST-call.</p><h3>Keeping track</h3><p>Considering you may potentially be replaying millions of events, doing a replay can take quite some time. It would be nice to be able to keep track of the replay so we know when it is done, for example in order to check if all is well. <br>The TrackingEventProcessor uses tokens to keep track of what events to process. A replay uses a special kind of token called a ReplayToken, which we can use to determine our position in the replay. These tokens can be obtained using the <a href="https://axoniq.io/apidocs/4.1/org/axonframework/eventhandling/tokenstore/TokenStore.html">TokenStore</a>, using the same event processor name as above. An added complication here is that we need to do it for multiple segments, so we will end up adding multiple replay tokens together and calculating the progress using that.</p><p>The ReplayToken contains a reference to the token when the reset was triggered, so we can use that to compare against. If no ReplayToken can be found, the replay is completed. A token can be queried for its position. Note that this is an OptionalLong, further complicating things: it might be impossible for the token to determine its position. In that case, we also assume no replay is currently underway.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cNjqL8vlSvpsZDIubn6hyA.png" /><figcaption>Not very pretty, is it?</figcaption></figure><p>We return an optional object containing the accumulated current position of all segments, with the accumulated position of where to reset took place (where applicable). For example, it may indicate we are at position 130 out of 304, resulting in our rebuild being 42% complete! 📈</p><p>We can create a REST-call to return these results, so we can periodically poll our system to see how far along we are.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3VAPkvtT_APYPj8bl6jbrA.gif" /><figcaption>Demo!</figcaption></figure><h3>ReplayStatus</h3><p>In the example project, we use axon’s ReplayStatus in order to determine whether or not the event is being handled as part of a replay. This can come in handy sometimes if for example your handlers send emails or push notifications. You probably do not want to do stuff like that during a replay. In our case, we use the ReplayStatus to introduce a little artificial delay in order to demo the progression a bit easier. 😴</p><p>Alternatively, you could annotate the handler with DisallowReplay to completely skip it.</p><h3>In conclusion</h3><p>I ended up interacting with my ReplayController using <a href="https://api.slack.com/interactivity/slash-commands">Slack slash commands</a>. That way I can trigger a replay and know how far along it is without actually building an admin interface for my application. I might explain how i did that in a future article.🔮</p><p>For now, this is how I managed to trigger Axon replays and keeping track of them in my Spring Boot application. I have created an example project, which is <a href="https://github.com/fourscouts/blog/tree/master/replays">available on GitHub</a>.</p><p>I hope you found this article useful for you and your projects. If it is, please let me know in the comments, and clap those hands! 👏👏👏</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d5d34bebefb6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/replaying-projections-with-axon-and-spring-boot-d5d34bebefb6">Replaying projections with Axon and Spring Boot</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A practical example to Axon’s DeadlineManager]]></title>
            <link>https://medium.com/fourscouts/a-practical-example-to-axons-deadlinemanager-98aaa20ddc67?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/98aaa20ddc67</guid>
            <category><![CDATA[cqrs]]></category>
            <category><![CDATA[scheduling]]></category>
            <category><![CDATA[axon-framework]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[axon]]></category>
            <dc:creator><![CDATA[Rolf Suurd]]></dc:creator>
            <pubDate>Mon, 18 Mar 2019 09:31:41 GMT</pubDate>
            <atom:updated>2019-03-18T09:31:41.176Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*WFNZ6XDAG6V9qKvRrgzudg.jpeg" /></figure><p>Since <a href="https://axoniq.io/docs/axon-framework-3.3">version 3.3</a>, axon introduced a new component for scheduling: the <a href="https://docs.axoniq.io/reference-guide/v/3.3/part-ii-domain-logic/sagas#keeping-track-of-deadlines">DeadlineManager</a>. Traditionally, scheduling events in Axon is done using the EventScheduler, applying an event after a certain amount of time. Should you just want to schedule a deadline and decide at that point whether or not you need to publish an event, now you can. In this blog post, we’ll explore a simple use case where we hold a reservation for a certain amount of time, and cancel it if the time expires before the reservation is confirmed.</p><h3>It’s showtime</h3><p>On April 23, 25 and 26, Drake will give a concert at the Ziggo Dome in Amsterdam. The Ziggo Dome has a capacity of 17,000 people. We can model these shows into an aggregate and start selling some tickets! To start with, a Show will have a name, time and available tickets. Oh, and of course a showId as @AggregateIdentifier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/316/1*RWqs2XE89Ath-lHTMqfq4A.png" /><figcaption>Start of our Aggregate.</figcaption></figure><p>We’ll create this Show aggregate using the PlanShow command and ShowPlanned event, containing the fields listed above.</p><h3>Making reservations</h3><p>In order to buy and subsequently pay for tickets, users should be able to reserve them for a limited amount time so they have the chance to get through the payment process. Here is where we will use Axon’s DeadlineManager to keep track of this reservation. We will schedule the deadline as soon as the users makes the reservation: at the @CommandHandler for ReserveTickets:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hy62Q3iABISI5ebJaacNQw.png" /><figcaption>Start the clock, but only if we have the tickets available! ⏲️</figcaption></figure><p>Here, we check if we have enough tickets available, and if that is the case we start the clock for 15 minutes, and apply the TicketsReserved event, with the generated reservationDeadlineId as event MetaData, so we can keep track of it. We can use this later to cancel the deadline. In the @EventHandler, we will decrease the available tickets and keep track of the reservation, in a Map&lt;String, List&lt;Reservation&gt;&gt; reservations field:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M2CSQDlin6KIUx6NFzVcRA.png" /><figcaption>Keep track of the reservation for future use.</figcaption></figure><p>A Reservation contains the reservationId, amount of tickets reserved and the associated deadlineId. We put it in a map so we can refer to it later when either the reservation expires or gets confirmed.</p><h3>Time’s up</h3><p>After 15 minutes, the reservation expires and the tickets should be released back in the pool of available tickets. The @DeadlineHandler will receive the payload as argument. In our case, that’s the reservationId. The ReservationExpired event is applied which in turn uses the stored Reservation to release the reserved tickets back in the pool:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/1*xsJTwLwl6ILNHWdWXLcOsg.png" /><figcaption>Reservation expired, tickets are for sale again.</figcaption></figure><h3>Confirming the reservation</h3><p>When a user pays for the tickets, the reservation gets confirmed. When this happens, the deadline is canceled and the tickets are definitely gone. We can now write other @EventHandlers to email the tickets to the person that made the reservation. 🎫</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*iUfPQrmI_WUy44mi05Q7Dg.png" /><figcaption>Reservation confirmed!</figcaption></figure><h3>Conclusion</h3><p>The DeadlineManager is a relatively new addition to the Axon framework and a nice alternative to the classic EventScheduler. An advantage in my eyes is that compared to the EventScheduler which always applies an event, the DeadlineManager does not, leaving you to decide what events to publish when a deadline is reached. Since the history of events is your single source of truth, it is a good thing to be in control.👍</p><p>Please check out the full code example in our <a href="https://github.com/fourscouts/blog/tree/master/deadline-manager">GitHub repository</a>! It also features a simple in-memory configuration for Spring Boot with some demo commands. If you can wait 15 minutes, you’ll see the DeadlineManager in action! You can also refer to the JUnit test for more examples.</p><p>Hopefully, you’ve enjoyed this post about the Axon DeadlineManager. If you did, do not hesitate to clap! 👏👏👏</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98aaa20ddc67" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/a-practical-example-to-axons-deadlinemanager-98aaa20ddc67">A practical example to Axon’s DeadlineManager</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Spring Boot OAuth2 with Discord as provider pt. II]]></title>
            <link>https://medium.com/fourscouts/spring-boot-oauth2-with-discord-as-provider-pt-ii-284a1d758865?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/284a1d758865</guid>
            <category><![CDATA[discord]]></category>
            <category><![CDATA[oauth2]]></category>
            <category><![CDATA[spring]]></category>
            <category><![CDATA[spring-security]]></category>
            <category><![CDATA[api]]></category>
            <dc:creator><![CDATA[Rolf Suurd]]></dc:creator>
            <pubDate>Fri, 30 Nov 2018 09:03:02 GMT</pubDate>
            <atom:updated>2018-11-30T09:03:02.222Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*iqBlDmYKKYNb-9aE9lZ3Aw.gif" /></figure><p>Some time ago, we <a href="https://medium.com/fourscouts/spring-boot-oauth2-with-discord-as-provider-fbe81ba2a721">published a blog</a> about logging in to your application using Discord as the provider. To refresh your memory, this was not possible with the default implementation of Spring Security’s OAuth2 library because it did not send a User-Agent HTTP header, which Discord requires. We ended up writing custom implementations for OAuth2AccessTokenResponseClient and OAuth2UserService, where we would find ourselves copying the original class and adding an extra line of code that would set the User-Agent header. While this was not a great solution, it worked! 🚀</p><h4>What has changed?</h4><p>With the release of <a href="https://docs.spring.io/spring-security/site/docs/5.1.0.RELEASE/reference/htmlsingle/#new">Spring Security 5.1</a>, it is now possible to customize the authorize and token requests, allowing us to add the User-Agent header without the need of copying and adjusting the original classes. Instead, we can just set custom Converters on both the OAuth2AccessTokenResponseClient and OAuth2UserService. Those converters can then augment the requests by adding a User-Agent header.</p><h4>Getting started</h4><p>The prerequisites have not changed. Please refer to our <a href="https://medium.com/fourscouts/spring-boot-oauth2-with-discord-as-provider-fbe81ba2a721">previous post on this subject</a> on how to obtain a Client ID and Client Secret. Once you have those, you can configure your application to use those credentials. Here is the application.yml:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/92d6559e74093c8d643a628070ca30ad/href">https://medium.com/media/92d6559e74093c8d643a628070ca30ad/href</a></iframe><blockquote>Note the client-id and client-secret parameters. You will need to provide the values you have got from registering your application at discord. If you don’t want to put this information in your application.yml, you can pass them as environment variables: SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_DISCORD_CLIENT_IDand SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_DISCORD_CLIENT_SECRET.</blockquote><h4>The converters</h4><p>Spring security already comes with default converters that create standard representations of an <a href="https://tools.ietf.org/html/rfc6749#section-4.1.3">OAuth 2.0 Access Token Request</a>. So, all we need to do is extend those converters and add a User-Agent header to those requests. First, let’s write a utility class that adds the header to a request:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/def484d49843e94cfeec0ecf09b679a2/href">https://medium.com/media/def484d49843e94cfeec0ecf09b679a2/href</a></iframe><p>Note that this utility creates a new RequestEntity. This is required because the headers of the original request are <em>immutable</em>, so we need to construct a new instance with our adjusted headers.</p><p>Then, we can create custom converters that call this new utility. We’ve chosen to extend these classes directly in our configuration:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/444227bdb64e43a9d9e45609c7ccb7e5/href">https://medium.com/media/444227bdb64e43a9d9e45609c7ccb7e5/href</a></iframe><p>Now each request made by the TokenResponseClient and OAuth2UserService will have the User-Agent header added.</p><p>This is all that we need in order to use Spring Security OAuth2 with Discord! We have updated <a href="https://github.com/fourscouts/blog/tree/master/oauth2-discord">our demo application</a> to use this new version of Spring Security, so you can check that out. 🎉</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=284a1d758865" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/spring-boot-oauth2-with-discord-as-provider-pt-ii-284a1d758865">Spring Boot OAuth2 with Discord as provider pt. II</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Text classification: algorithm vs neural networks]]></title>
            <link>https://medium.com/fourscouts/text-classification-algorithm-vs-neural-networks-2ea26fd92108?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/2ea26fd92108</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[naive-bayes]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Rolf Suurd]]></dc:creator>
            <pubDate>Fri, 16 Nov 2018 20:33:25 GMT</pubDate>
            <atom:updated>2018-11-16T20:33:25.439Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bhFifratH9DjKqMBTeQG5A.gif" /><figcaption>Neural network black magic hard at work</figcaption></figure><p>Text classification is the task of assigning predefined categories to free text. Documents, tweets, instagram feeds, email, basically anything that contains text. This has many interesting applications in the real world. One example<br>that you might be enjoying on a day to day basis is spam detection for your email. Or your news feed that is organized by topic. An example that might be more interesting to companies is sentiment analysis. <em>How do my customers feel about my company?</em></p><p>Doing text classification with a neural network sounds pretty cool, but <br>can be pretty daunting. After all, neural networks are like black boxes,<br>or even black magic: you don’t really know what is happening, or how the<br>result was achieved. Before you even get to use the black box, you need to build it. What do you put in it? How many hidden layers? What kind of heuristics do we put in? What are the outputs? All these things are pretty complicated, especially if you’re just starting with Natural Language Processing (<em>NLP</em>), like I am 🤖.</p><p>Then, you need to train the model. How do you know if the model is properly trained? Or if the data you feed it is correct? After all, you want to work with text, but the neural networks are usually huge matrices of magic numbers. So arcane and complicated!</p><p>This is why sometimes you want to use a good old algorithm to do these sorts of things. Especially naive ones, because we can understand them. A classic algorithm to use is <a href="https://en.wikipedia.org/wiki/Naive_Bayes_classifier"><strong>Multinomial Naive Bayes</strong></a>. This algorithm is naive because it assumes independence of features. This means that it does not care about the position of a word within a sentence, nor does it care about possible relations between the words.</p><p>For example, let’s take: “<em>Cats are good pets, for they are clean and are not noisy”</em>. In naive bayes, there is no relation between the words <em>Cats</em>, <em>pets</em>, <em>clean</em> and <em>noisy</em>. In NLP, this collection of words is called a<a href="https://en.wikipedia.org/wiki/Bag-of-words_model"> bag of words</a>. While this approach may seem naive, remember that we are not trying to understand the meaning of a sentence, but instead we are merely trying to find a proper classification.</p><h4>Training the algorithm</h4><p>In order to write an algorithm that classifies arbitrary text, we will need to create a data structure that has a bag of words per classification. We can then take text and see how that fits each of those bags of words.</p><p>We will need to provide some training data in order to build a bag of words for each classification. Let’s take a sentence from a fictional set of training data, and let’s give it the classification “<em>amazing</em>”.</p><p>Let’s break down the steps required to build a bag of words for “<em>amazing</em>”:</p><ul><li>Break down each sentence to a set of single words (<a href="https://nlp.stanford.edu/IR-book/html/htmledition/tokenization-1.html">tokenization</a>).</li><li>Filter out <a href="https://en.wikipedia.org/wiki/Stop_words">stop words</a></li><li>Normalize the words by <a href="https://en.wikipedia.org/wiki/Word_stem">stemming</a> them</li><li>Convert to lower case to have case insensitive matching</li></ul><p>The sentence we will work with is:</p><blockquote>He told us a very exciting adventure story</blockquote><p>First up, tokenizing. Tokenizing this sentence yields “<em>He</em>”, “<em>told</em>”, “<em>us</em>”, “<em>a</em>”, “<em>very</em>”, “<em>exciting</em>”, “<em>adventure</em>” and “<em>story</em>”. The result can vary depending on what tokenizer we use, since there are a ton of <a href="https://text-processing.com/demo/tokenize/">tokenizers</a> out there, but generally speaking each word is taken individually.</p><p>Next, we filter out stop words, or very frequently used words. There is no definitive list of stop words, and it also depends on what goal we are processing text for, but for this article we will use the list defined here: <a href="http://xpo6.com/list-of-english-stop-words/">http://xpo6.com/list-of-english-stop-words</a>. After removing the stop words from our example sentence, we are left with: “<em>told</em>”, “<em>exciting</em>” “<em>adventure</em>”, “<em>story</em>”.</p><p>Finally, we should stem the words. This is needed so that the algorithm recognizes variations of the same word. For example, the stem of <em>exciting</em> is <em>excit</em>. This way, when we encounter words such as <em>exciting</em>, <em>excitement</em>, <em>excite</em>, etc. we can consider them a match. The resulting bag of words for the “<em>amazing</em>” category results in: “<em>told</em>”, “<em>excit</em>”, “<em>adventur</em>”, “<em>stori</em>”. Just like with tokenizers, there are many different <a href="https://text-processing.com/demo/stem/">stemmers</a> out there, including different stemmers for different languages. A famous one is the <a href="http://snowball.tartarus.org/algorithms/porter/stemmer.html">Porter stemmer</a>.</p><p>Then, we can move on to the next sentence and do the same thing for it’s classification. Create a new bag of words if it’s a new classification we haven’t trained yet, or if we have encountered the classification before, add new words to the bag until we are done.</p><h4>Using the algorithm</h4><p>Now that we have a bunch of classifications that each have a bags of words, we can leverage that to classify new sentences. We will need to prepare the sentence in the same way as we did the training data: Tokenize the sentence, filter out the stop words, and stem the words. Then, go through each word and check for each classification if it’s bag of words contains the word. If it does, you can increase the score for that classification. The classification that ultimately has the highest score is most likely to be the correct one. 👌</p><p>In pseudocode, this could look something like so:</p><pre>def calcalateScore(sentence, classification) {<br>  score = 0;</pre><pre>// go through each word in the sentence<br>  tokenize(sentence).each(word =&gt; {</pre><pre>// skip stop words<br>    if (!stopWord(word)) {</pre><pre>if (bagsOfWords[classification] contains stem(word)) {<br>        score += 1;<br>      }<br>    }<br>  }</pre><pre>return score;<br>}</pre><p>However, simply counting occurrences of words within individual bags might not be the best approach, since some words might occur in all bags, while other words might only occur in one or two, depending on the size of your training data. This means that categories with very large bags of words are almost guaranteed to get a high score, which may not always be a relevant match.</p><p>One way to solve this is to account for <em>commonality</em> of each word: words that we have seen many times in our training data yield less of a score then words we’ve only seen once or twice. So, instead of incrementing the score by 1 for each match, we can do a weighted increment:</p><pre>score += 1 / numberOfOccurences(word);</pre><p>This way, we should get better results. 😊</p><h3>Conclusion</h3><p>Using a machine learning algorithm without deep learning may feel a bit naive to do text classification, but one of the main advantages i feel it has over neural networks is that it’s a whole lot easier to explain as to why the algorithm came to a certain classification. And in the end, if results need to be interpreted by humans (such as the user!), being able to reason and explain as to why a certain classification was derived is a lot better then to say:</p><blockquote>Because the neural network said so.<em>📖</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ea26fd92108" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/text-classification-algorithm-vs-neural-networks-2ea26fd92108">Text classification: algorithm vs neural networks</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Writing tests in hindsight is like writing no tests at all]]></title>
            <link>https://medium.com/fourscouts/writing-tests-in-hindsight-is-like-writing-no-tests-at-all-b3e98824ed9e?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/b3e98824ed9e</guid>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[legacy-code]]></category>
            <dc:creator><![CDATA[Silvester van der Bijl]]></dc:creator>
            <pubDate>Tue, 16 Oct 2018 09:52:04 GMT</pubDate>
            <atom:updated>2018-10-16T20:12:29.918Z</atom:updated>
            <content:encoded><![CDATA[<p>We frequently encounter teams that have to deal with complex and fragile codebases. Almost all of these teams are fully aware that the quality and robustness of the code is one of their main problems but they’re struggling to find the right answer.</p><p>One approach I’ve seen teams attempt (and have actually tried myself) is to attempt to write tests for the existing codebase. While valiant, this approach will most likely get you nowhere. Here’s why:</p><ul><li>Test Driven Development and unit testing is not just about validating behaviour, inputs and outputs but it forces you to think about the design of your classes, functions and methods. Writing tests in hindsight reverses this principle. You design tests around your existing code. You’re trying to reproduce the thought patterns of the multiple developers that came before you.</li><li>Legacy codebases tend to be relatively large. A team can spend months writing tests for many parts of the system only to discover the overall code coverage has gone up from 0,01% to 10%. Would you refactor without fear on a system with just 10% code coverage?</li><li>Best case scenario you’ve successfully managed to test all paths into some arcane piece of code. Are you sure this code is being used in production? Has it ever caused problems? Will the functionality change anytime soon?</li><li>This process is time consuming. You’re probably better off spending precious development capacity into delivering new (properly tested) features to your users.</li></ul><h3>How can you increase quality</h3><p>Writings tests and applying Test Driven Development is a good start! Just apply common sense and don’t go overboard with writing tests for your existing codebase. Where your code branches off into untested code perhaps apply an interface you can test. It’s probably also a good idea to add a test that validates the behaviour you’re expecting from the new code’s perspective but don’t drill down.</p><p>If you can’t make heads or tails from existing code then don’t try. Instead rebuild the <em>feature </em>the code is supporting using the help from domain experts, customers, users, etc. Make sure to apply TDD this time though ;-)</p><h3>Summary</h3><p>Proper testing is essential for keeping quality of a system high. High quality is the only way to be able to move fast but don’t assume adding tests for existing code will reap the same benefits.</p><p>Want to know more about dealing with legacy codebases? Send me a message on LinkedIn or just ask in the comments!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3e98824ed9e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/writing-tests-in-hindsight-is-like-writing-no-tests-at-all-b3e98824ed9e">Writing tests in hindsight is like writing no tests at all</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Spring Boot OAuth2 with Discord as provider]]></title>
            <link>https://medium.com/fourscouts/spring-boot-oauth2-with-discord-as-provider-fbe81ba2a721?source=rss----ad181d2289c3---4</link>
            <guid isPermaLink="false">https://medium.com/p/fbe81ba2a721</guid>
            <category><![CDATA[spring-security]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[discord]]></category>
            <category><![CDATA[oauth2]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Rolf Suurd]]></dc:creator>
            <pubDate>Fri, 15 Jun 2018 08:56:45 GMT</pubDate>
            <atom:updated>2019-11-14T10:53:17.056Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*x9JsspAvISd9Mq0D56IROQ.gif" /></figure><blockquote>Before you read on, please note that <a href="https://medium.com/p/284a1d758865">a new blog</a> has been published about this subject, using spring security 5.1 to address the issue discussed here.</blockquote><p>So, you want to add an OAuth2 login to your spring boot application, and you want <a href="https://discordapp.com/">Discord</a> to be the provider? No problem.</p><p>But why Discord? Discord is a free voice and text chat app for gamers which lets them create their own servers to chat with each other. Discord has <a href="https://discordapp.com/developers/docs/intro">an API </a>which allows other developers to create applications to integrate with these servers. Developers can leverage OAuth2 in order to use this API.</p><h4>First things first</h4><p>First, we will need to register an application with Discord to obtain a Client ID and Client Secret. You can do that here: <a href="https://discordapp.com/developers/applications/me">https://discordapp.com/developers/applications/me</a></p><p>Discord requires you to whitelist one or more URLs to redirect to upon succesful authentication. For now, http://localhost:8080/login/oauth2/code/discord<em> </em>should be fine. Add an optional description and icon to your liking. When created, note the Client ID and Client Secret, we will need those later.</p><h4>The code</h4><p>We will be making a small application to demonstrate. It’s a standard spring boot application, and the code is located on <a href="https://github.com/fourscouts/blog/tree/master/oauth2-discord">GitHub</a>.</p><p>Configuring Discord as a OAuth2 provider is fairly straightforward. You will need to specify a bunch of properties in your application configuration:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ac118037f4ef9fcd4eb8c725880fe2e4/href">https://medium.com/media/ac118037f4ef9fcd4eb8c725880fe2e4/href</a></iframe><p>Note the client-id and client-secret parameters. You will need to provide the values you have got from registering your application at discord. If you don’t want to put this information in your application.yml, you can pass them as environment variables: SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_DISCORD_CLIENT_IDand SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_DISCORD_CLIENT_SECRET.</p><p>We are only requesting the identity scope right now, in order to get the current user. If you want to interact with other resources, you will need to add the <a href="https://discordapp.com/developers/docs/topics/oauth2#shared-resources-oauth2-scopes">correct scopes</a>.</p><p>Next, we’ll specify that we’re using an oauth2 login in our security configuration:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6e412e8282ff4101a0493efc9344b8c6/href">https://medium.com/media/6e412e8282ff4101a0493efc9344b8c6/href</a></iframe><p>Usually, this is enough to configure a custom OAuth2 provider in spring boot.</p><p>However, If you would try to login now, you will find out that something is wrong! The problem is that Discord requires a User-Agent header that the standard implementation of spring security isn’t sending. We can fix this by providing custom implementations of OAuth2AccessTokenResponseClient and OAuth2UserService.</p><p>The OAuth2AccessTokenResponseClient will need to POST some form data to exchange an access code for a token:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dbccdab7989b71f5a0fb1161ae77577b/href">https://medium.com/media/dbccdab7989b71f5a0fb1161ae77577b/href</a></iframe><p>This code posts to the tokenUri and ultimately constructs an oauth2 access token with parameters from the response. Note the inclusion of theUser-Agent to the request.</p><p>Next, we’ll need to do something similar for the OAuth2UserService. This service retrieves the user from the Discord API and constructs a Principal spring security can work with.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8d3ef87ae9c19a11c55c38299a016a67/href">https://medium.com/media/8d3ef87ae9c19a11c55c38299a016a67/href</a></iframe><p>Finally, we need to start using these new components. We can do this in our security configuration:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/29e5cc63206267c15a4c8036af5f94a4/href">https://medium.com/media/29e5cc63206267c15a4c8036af5f94a4/href</a></iframe><p>Now, when we open our application we can log in using Discord. Go to <a href="http://localhost:8080/">http://localhost:8080</a> and try it out:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F272542447%3Fapp_id%3D122963&amp;dntp=1&amp;url=https%3A%2F%2Fvimeo.com%2F272542447&amp;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2Fdefault_1280&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1316" height="720" frameborder="0" scrolling="no"><a href="https://medium.com/media/2d18ad2b9dd8041f8a54f4dbb71fc608/href">https://medium.com/media/2d18ad2b9dd8041f8a54f4dbb71fc608/href</a></iframe><h4>Conclusion</h4><p>In this article we’ve demonstrated how to add a custom provider to the spring security oauth2 client, and added custom implementations in order to work with Discord’s requirement of having a User-Agent header for each request.</p><p>The problem that this article tackles is discussed here: <a href="https://github.com/spring-projects/spring-security/issues/4958">https://github.com/spring-projects/spring-security/issues/4958</a></p><p>The code for this project is on the <a href="https://fourscouts.nl/">FourScouts</a> <a href="https://github.com/fourscouts/blog/tree/v1.0/oauth2-discord">GitHub</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fbe81ba2a721" width="1" height="1" alt=""><hr><p><a href="https://medium.com/fourscouts/spring-boot-oauth2-with-discord-as-provider-fbe81ba2a721">Spring Boot OAuth2 with Discord as provider</a> was originally published in <a href="https://medium.com/fourscouts">FourScouts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>