<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Manserpatrice on Medium]]></title>
        <description><![CDATA[Stories by Manserpatrice on Medium]]></description>
        <link>https://medium.com/@manserpatrice?source=rss-1b0e7cac3855------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*am3sR7H3jpv9hEAhn4A1xQ.jpeg</url>
            <title>Stories by Manserpatrice on Medium</title>
            <link>https://medium.com/@manserpatrice?source=rss-1b0e7cac3855------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 22:59:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@manserpatrice/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[SonarQube Pull Request Analysis for huge monolithic applications]]></title>
            <link>https://manserpatrice.medium.com/sonarqube-pull-request-analysis-for-huge-monolithic-applications-62533403e35f?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/62533403e35f</guid>
            <category><![CDATA[bitbucket]]></category>
            <category><![CDATA[monolithic-architecture]]></category>
            <category><![CDATA[gradle]]></category>
            <category><![CDATA[sonarqube]]></category>
            <category><![CDATA[sonar]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Sat, 20 Jan 2024 13:57:27 GMT</pubDate>
            <atom:updated>2024-01-29T08:49:02.825Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9M3_cfep4Q8z8ZxMdRu_Uw.png" /></figure><p>Integrating SonarQube PR analysis into a large-scale monolithic application is a challenge familiar to many, yet rarely mastered. At our company, we faced this head-on, dealing with an enormous codebase that pushed the limits of SonarQube’s capabilities, a non-negotiable performance target of under 20 minutes per pull request, and the complexities of melding this with BitBucket.</p><p>In this post, I’ll share our journey of navigating these challenges. From overcoming oversized scan reports to fine-tuning performance and achieving seamless integration, I aim to provide insights and practical strategies that can aid any software development team facing similar hurdles. Let’s dive into how we turned a daunting task into a successful integration story.</p><h3>Background</h3><p>Our journey centers around an Enterprise Resource Planning (ERP) software, intricately tailored for insurance companies. This colossal Gradle project, a labyrinth of over 1000 submodules, has evolved organically over time. It’s a behemoth in terms of scale, comprising more than 5 million lines of Java code.</p><p>Until recently, our approach to maintaining code quality involved conducting daily SonarQube scans, but these were exclusively limited to the master branch. This process was far from efficient, typically consuming around 4 to 5 hours for a complete scan. While this method provided some insights into code quality, it was clear that a more dynamic and integrated approach was needed to keep pace with the rapid development and complexity of our application.</p><h3>Challange 1: Handling Large Scan Reports</h3><p>When integrating SonarQube PR analysis into our monolithic ERP software, we encountered a significant hurdle: managing exceptionally large scan reports. SonarQube, though adept at analyzing large projects, met its match with our application’s scale. The core issue stemmed from the size of the reports generated by the scanner. These reports were too voluminous to be accommodated by SonarQube’s PostgreSQL database, a problem not uncommon in massive projects, as discussed in <a href="https://community.sonarsource.com/t/sonarqube-postgresql-invalid-memory-alloc-request-size/16581">this SonarSource community thread</a>.</p><p>The primary technical limitation was that SonarQube attempted to store the entire scan report in a single database field, capped at 1GB. However, our project’s scan report ballooned to approximately 1.6GB, far exceeding this limit. This challenge presented us with two potential paths:</p><ol><li><strong>File Exclusion Strategy:</strong> One approach was to reduce the report size by excluding about one-third of the files from the analysis. While this would bring down the file size to within the permissible limit, it raised concerns about the comprehensiveness of our code quality checks.</li><li><strong>Project Segmentation Approach: </strong>The alternative was to restructure our single SonarQube project into multiple smaller projects. This would allow us to upload the scan reports individually, effectively bypassing the size constraint by dividing the total report into smaller, more manageable segments.</li></ol><p>Since the first option would not provide us with accurate results and would produce even less code insights as the old SonarQube instance that we are running, we went with the second solution.</p><p>Since we already have different submodules in our Gradle project, we just applied the Sonar Plugin to all submodules individually via Gradle’s subprojects configuration block.</p><pre>apply plugin: &#39;java&#39;<br><br>subprojects {<br>   apply plugin: &#39;org.sonarqube&#39;<br><br>   sonarqube {<br>      properties {<br>         property &quot;sonar.projectName&quot;, &quot;MYCOMPANY- ${project.name}&quot;<br>         property &quot;sonar.projectKey&quot;, &quot;com.mycompany.name:${project.name}&quot;<br>         property &quot;sonar.scm.disabled&quot;, &quot;false&quot;<br>         property &quot;sonar.exclusions&quot;, &quot;build/**/*,src/main/resources/**&quot;<br>      }<br>   }<br>}</pre><p>When running the sonar task for all projects, suddenly the upload succeeds and the projects are automatically created on the new SonarQube instance.</p><p>Seems like this story is done ! — Not really… The total analysis time for all projects is around 4.5 hours, which is not feasible for any kind of pull request feedback and nowhere near our goal of 20 minutes analysis time.</p><p>When having a look at the Build Scan of Gradle, we saw that the scans at the beginning of the build are much faster than the ones at the end of it. It turned out that our SonarQube server was to slow for all those parallelized requests that our Jenkins build makes (40 parallel workers). We increased the cpu and memory resources allocated to our deployment of SonarQube, which solved this issue.</p><h3>Challange 2: Ensuring performant scans with fast analysis time</h3><p>There are two main issues with the approach that we are currently following.</p><ol><li>Code that did not change is still analyzed again.</li><li>The Sonar scanner-engine gets downloaded every time for all the projects, even if no single line has changed.</li></ol><p>For the first problem, sonarqube provides a solution. It is called <a href="https://docs.sonarsource.com/sonarqube/latest/analyzing-source-code/pull-request-analysis/">Sonar Pull Request Analysis</a>. To enable this behaviour for our pull requests, we needed to add the following flags to our sonar command.</p><pre>./gradlew sonar -Dsonar.pullrequest.key=5 -Dsonar.pullrequest.branch=feature/my-new-feature -Dsonar.pullrequest.base=master</pre><p>When comparing to the base branch master, this branch should already have a full scan done. This helps Sonar cache the results and makes it easier to detect changes. Also, we let the task run in Jenkins as a Pull Request build (A build that merges your feature branch with the current target branch on a detached head and executes commands on that state), which further improved performance.</p><p>This change saved us a lot of scan time. When only changing one single line of a project, it took us 1.5 hours to analyze our whole application, which is quite an impressive jump from the initial 4.5 hours, but still nowhere near our goal of 20 minutes.</p><p>To fix the second problem, we had to get some data about the average size of pull request in our organization. We came to the conclusion, that mostly only code in 1–10 submodules is changed in one single pull request. That means that only this small subset of projects would actually need to be scanned to see the changes. Also the download of the rulesets would only be needed for those specific projects in the subset.</p><p>We came up with a Gradle plugin that only executes the sonar analysis on projects that have actual file changes in it. The basic functionality of it looks like this:</p><pre>import org.gradle.api.provider.ProviderFactory<br>import org.gradle.api.services.BuildService<br>import org.gradle.api.services.BuildServiceParameters<br>import javax.inject.Inject<br><br>abstract class ChangedFilesService @Inject constructor(private val providers: ProviderFactory) :<br>    BuildService&lt;BuildServiceParameters.None&gt; {<br><br>    val changedFilePaths: List&lt;String&gt; by lazy {<br>        val targetBranch = System.getProperties().getProperty(&quot;targetBranch&quot;, &quot;master&quot;)<br>        findChangedFiles(&quot;origin/$targetBranch&quot;)<br>    }<br><br>    private fun findChangedFiles(reference: String): List&lt;String&gt; {<br>        return providers.exec { commandLine(&quot;git&quot;, &quot;diff&quot;, reference, &quot;--name-only&quot;) }<br>            .standardOutput.asText.get().trim().split(&#39;\n&#39;)<br>    }<br><br>}</pre><p>This service leverages git to get the changed files as a list of Paths.</p><pre>val sonarBuildService =<br>    project.gradle.sharedServices.registerIfAbsent(&quot;sonarBuildService&quot;, ChangedFilesService::class.java) {}<br><br>val projectHasChangedFiles = sonarBuildService.get().changedFilePaths.any {<br>    val prefix = project.projectDir.toPath().relativeTo(project.rootDir.toPath()).toString()<br>    it.startsWith(prefix)<br>}<br><br>if (projectHasChangedFiles) {<br>    project.tasks.getByName(&quot;sonarSelectiveScan&quot;).dependsOn(&quot;sonar&quot;)<br>    project.tasks.getByName(&quot;sonar&quot;).dependsOn(&quot;jacocoTestReport&quot;)<br>}</pre><p>The `sonarSelectiveScan` task can be run for all projects, but it will only execute the sonar task for the projects that really have file changes in them.</p><p>Now if we change our sonar command to the following one:</p><pre>./gradlew sonarSelectiveScan -DtargetBranch=master -Dsonar.pullrequest.key=5 -Dsonar.pullrequest.branch=feature/my-new-feature -Dsonar.pullrequest.base=master</pre><p>It will only download the rules, analyse the code and publish the report for just the projects that changed. Depending on the size of our pull requests, we get analysis speeds of about 1.5 –10 minutes. Thats an impressive gain in performance!</p><h3>Integration into Bitbucket</h3><p>Pull Request analysis itself is cool, but does not bring that much value if it is not automatically added to your pull request, so that you can change your code before you merge.</p><p>We use the <a href="https://marketplace.atlassian.com/apps/1212735/include-code-quality-for-bitbucket?hosting=datacenter&amp;tab=overview">“Include Code Quality for BitBucket”</a> Plugin in BitBucket, so that we can get the Pull Request annotations directly into BitBucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V36pmaOWScWIz9tJnm_Dbg.png" /></figure><p>Normally you would just add the project mapping manually to the configuration of this plugin but since we have over 1000 submodules, it does not make sense to do this manually. Therefore, we have added some more logic to our plugin to utilize their <a href="https://mibexsoftware.github.io/sonar-bitbucket-server-rest-api/">REST API</a> and create the new project mappings automatically.</p><p>The simplified logic looks like this:</p><pre>val existingProjectMappings = getProjectMappings()<br>if (existingProjectMappings.contains(ProjectMapping(project))) {<br>  createProjectMapping(project)<br>}</pre><p>This will be executed every time our sonarSelectiveScan task is ran. To ensure efficiency, this logic can be moved into a BuildService, so that it only gets executed the first time and reuses the results from cache afterwards.</p><p>After this last change in the BuildService, our Sonar Pull Request analysis was working perfectly and really efficient. On average we have complete analysis times with compiling and testing of just about 10 minutes. On our jurney we learned a lot about how sonarqube is intended or better not intended to be used and how to overcome those issues to make it work for our gradle build. I hope that this article will help you if you face similar difficulties.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=62533403e35f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Integrate Language-Server with VS-Code extension]]></title>
            <link>https://manserpatrice.medium.com/integrate-language-server-with-vs-code-extension-ffe8f33a79cf?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/ffe8f33a79cf</guid>
            <category><![CDATA[language-server-protocol]]></category>
            <category><![CDATA[vscode-extension]]></category>
            <category><![CDATA[vscode]]></category>
            <category><![CDATA[intellisense]]></category>
            <category><![CDATA[lsp]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Fri, 11 Nov 2022 07:16:29 GMT</pubDate>
            <atom:updated>2022-11-14T05:20:12.315Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9-1YR7JTGl9-bBpjPc4U-w.png" /></figure><p>This week my team and I wanted to create a new Language Server for a company internal language. There have already been efforts to create an Eclipse Language Plugin which was based on XText. In order to be able to use the Language Server on other IDEs than Eclipse, we created a new Language Server based on Microsofts <a href="https://microsoft.github.io/language-server-protocol/">Language Server Protocol</a> (LSP).</p><p>My part was to integrate the newly created Language-Server with VS-Code. I chose this Editor over IntelliJ IDEA since it is known to have an easier and better documented integration for Language-Servers than IntelliJ has.</p><h3>Communication between Client and Language-Server</h3><p>First of all, when the user opens a document, the client sends a notification with the content of the document to the Language Server.</p><p>The server sends a notification witht the Diagnostics of the current document back to the client.</p><p>If the user edits a document, the client sends a new notification with the current url of the document and the differences compared to the old version. This way it does not have to upload the whole document after every change.</p><p>If the user executes “Go to definition” he sends a Request to the server with the current documentURI and the position of the cursor.</p><p>The server sends back a response with the location of the linked class, method or variable.</p><p>When the user finally closes the document, it sends a notification with the documentURI to the server so that it can remove the document from its current context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/0*GVy1hQq_KqepEO9O.png" /><figcaption>Figure from <a href="https://microsoft.github.io/">https://microsoft.github.io</a></figcaption></figure><p>The default port for Language-Servers is port 5007. In this example we will also start our Language-Server on this port.</p><p>You can read more about the communication between server and client on <a href="https://code.visualstudio.com/api/language-extensions/language-server-extension-guide">https://code.visualstudio.com/api/language-extensions/language-server-extension-guide</a></p><h3>Creating the extension</h3><p>To create a language extension, you need to create a project with the following project structure:</p><blockquote>/package.json<br>/&lt;extension&gt;.configuration.json<br>/src/extension.ts<br>/src/tsconfig.json</blockquote><p>Where &lt;extension&gt; should be the extension of the new language that you want to support.</p><p>In the following sections of this tutorial, I will continue using “<strong>gui</strong>” as the extension of the language files.</p><p>The content of the package.json file looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba870e51d7bdacf7b2bc439b3bbd4735/href">https://medium.com/media/ba870e51d7bdacf7b2bc439b3bbd4735/href</a></iframe><p>Keep an eye on the parts from line 22 til 35 since this is the extension related part. It is really important that you add the dot before the extension name on line 33.</p><p>The gui.configuration.json file will be used for local support of the language like auto-closing brackets and comments.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ec7b4a8843237e7a5e5659fbc707a8fe/href">https://medium.com/media/ec7b4a8843237e7a5e5659fbc707a8fe/href</a></iframe><p>All these configurations are needed to improve the workflow of the enduser and providing basic functionalities for autocompletion.</p><p>The basic /src/tsconfig.json file looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d9dd757f6f2e7363733b33a882e7b957/href">https://medium.com/media/d9dd757f6f2e7363733b33a882e7b957/href</a></iframe><p>Now the really interesting part is located in the extension.ts file.</p><p>A LanguageClient gets created with a connection to port 5007 on localhost.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c1e9952ebfd162a58dc3ea13ab987ca8/href">https://medium.com/media/c1e9952ebfd162a58dc3ea13ab987ca8/href</a></iframe><p>On line 27 the gui extension is needed, so that the server knows which files he should track.</p><p>The fileEvents that should notify the server are defined on line 29. With this configuration the client only notifies the server if a “<strong>.gui”</strong> file changed.</p><p>On line 37 the client is started.</p><p>The deactivate() function is needed so that the server can close the connection if the extension gets disabled.</p><h3>Running the extension</h3><p>To run the extension, you need to open this project in VS-Code, if you are not already working with it, and press “Debug/Run”. This will open a new VS-Code window with you custom extension already installed and enabled. Now you can create a new file with the defined extension. You should now be able to use autocomplete functionality and semantic checking of your LSP server.</p><p>Further information can be found on these sites:</p><ul><li><a href="https://code.visualstudio.com/api/language-extensions/language-server-extension-guide">https://code.visualstudio.com/api/language-extensions/language-server-extension-guide</a></li><li><a href="https://learn.microsoft.com/en-us/visualstudio/extensibility/adding-an-lsp-extension?view=vs-2022">https://learn.microsoft.com/en-us/visualstudio/extensibility/adding-an-lsp-extension?view=vs-2022</a></li><li><a href="https://idiomaticsoft.com/post/2022-04-29-create-vscode-client/">https://idiomaticsoft.com/post/2022-04-29-create-vscode-client/</a></li></ul><h3>Reflection</h3><h4>What went good</h4><p>I think that the actual implementation of the extension was not that hard and I got it working in a pretty short time.</p><h4>What needs improvement</h4><p>The hardest part in this small project was to understand what exactly the Language Server Protocol is and how it works. At the beginning I did not read that much in the official docs but rather tried to find answers to specific questions I had at that moment on StackOverflow. The Problem with that approach was that I never had a full understanding of the topic itself and therefore ran into issues that I would not have had if I just informed myself a little bit better about that topic beforehand.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ffe8f33a79cf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to approach customer interviews]]></title>
            <link>https://manserpatrice.medium.com/how-to-approach-customer-interviews-79eb623e0db3?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/79eb623e0db3</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[project-management]]></category>
            <category><![CDATA[interview-questions]]></category>
            <category><![CDATA[customer-experience]]></category>
            <category><![CDATA[interview]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Wed, 19 Oct 2022 15:48:15 GMT</pubDate>
            <atom:updated>2022-10-23T04:49:15.683Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2GAIGVtU_-5bo2ag" /><figcaption>Photo by <a href="https://unsplash.com/@mkvandergriff?utm_source=medium&amp;utm_medium=referral">Maranda Vandergriff</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This week I got a new project for which I should be responsible from beginning to the end. Because there is only a quite blurry description of what the project should be like, I have to do some interviews with our customers. In my special case, our customers are also developers, that work on business code.</p><p>I separate this article into three different subtopics:</p><ul><li>Preparation</li><li>Doing the interview</li><li>Evaluating the Answers</li></ul><h3>Preparation</h3><p>First of all, the most important thing for an interview is the preparation. You have to define first what you want to know from your customer. After that, you can create open-ended questions for it.</p><p>In my case, I wanted to know if they have problems or inconveniences with the current solution and if they would benefit from the new project.</p><p>This is a quite short question and if I would ask it just like it stands there, the answers would only consist out of “yes” and “no”. This is a bad strategy, because neither do I know what features they want to have in this new project nor do I know what exactly caused these inconveniences in the first place.</p><p>To tackle this problem, we need to formulate open-ended questions from it. Open-ended questions are questions that don’t have a specified selection of possible answers. They should begin with words like: “<strong>why”, “how”, “what”, “describe”, “tell me about…” </strong>or<strong> “what do you think about</strong>…”</p><p>In my case, the questions that I generated out of the base question were these:</p><ul><li><strong>How</strong> do you interact with the current solution in your daily work?</li><li><strong>What</strong> are the pain points in your daily work depending the current solution?</li><li><strong>Describe</strong> what functionalities the new solution needs in your eyes.</li><li><strong>Tell me about </strong>how this new solution could impact your work.</li></ul><h3>Doing the interview</h3><p>Now that you have prepared yourself with interview questions, we can start the interview.</p><p>Normally it is best to reserve a time slot that has quite a margin so that you dont ‘t have to stress if a discussion takes longer than planned.</p><p>When you start the interview, introduce yourself, aswell as the topic you want to get informations about. Maybe the interviewee recognises that he has no experience in the requested field or in my case does not even need the old solution, so why should he want to use a new solution.</p><p>After these initial steps, you can start asking your open-ended questions. You don’t have to ask all of the questions if the interviewee gave you the answer in a prior question. The order of the questions does not matter that much.</p><p>The flow of the conversation should be as natural as possible. If the interviewee is stuck with answering the question, you can help him out by reformulating it in an easier way or asking some follow up questions to guide him in the right direction.</p><h3>Evaluating the Answers</h3><p>Now that you got all the answers it is finally time to evaluate them.</p><p>These were the notes that I created during one interview:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/847/1*lT41Ilc9YTZRQgKdLnmRDQ.png" /></figure><p>First of all, I created Personas from my notes that represent each group of people that I interviewed.</p><p>I then looked at the personas and created functional and non functional requirements for my project. This way, I knew exactly what the different groups of customers want and I was able to weigh some of the requirements a little heavier than others.</p><h3>Reflection</h3><h4>What went good</h4><p>In my opinion, the interview itself went pretty good. Sometimes I needed to interrupt the interviewees if they faded away a little bit to much. I think I learned a lot from this experience and it was interesting to see how the wording of a question can influence the answer.</p><h4>What needs improvement</h4><p>At the beginning I wasn’t sure how I should approach this task. I had problems finding the right people to do the interview with. After I discussed it with my team, more and more names popped up which I could then use for the interview. Next time, I would directly ask my PO for these kinds of information, because he has a much wider network than I have.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79eb623e0db3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Elasticsearch Highlighting with Kotlin]]></title>
            <link>https://manserpatrice.medium.com/elasticsearch-highlighting-with-kotlin-e4b35ee1780e?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/e4b35ee1780e</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[highlighting]]></category>
            <category><![CDATA[serialization]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Thu, 06 Oct 2022 06:07:26 GMT</pubDate>
            <atom:updated>2022-10-09T04:06:50.572Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*8i-q8pOl-2W_XOxv9WM-dg.png" /></figure><p>This week, I had to add highlighting functionality to the search field of my project. Currently, whenever I click on a search result on the frontend, it opens the corresponding documentation. These documentations are really long and therefore it is taking a long time to find the section in the documentation that you are looking for. To solve this issue I needed to highlight the best matches in the documentation, so that you can see it on first sight.</p><h3>Highlighting with KT-Search</h3><p><a href="https://github.com/jillesvangurp/kt-search">KT-Search</a> is a great Kotlin client for ElasticSearch. Even if it is not officially supported, it provides most features of the official Java client. <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/highlighting.html#highlighting">Highlighting</a>, sadly, is not supported with kt-search. Therefore, we need to write the functionality ourselves.</p><p>The end result of this tutorial will be a ElasticSearch Hit that contains informations about the highlighted terms.</p><p>The solution to this problem is structured in three parts:</p><ol><li>Creating the Query</li><li>Receiving the Response</li><li>Deserializing the SearchResponse</li></ol><h4>Creating the Query</h4><p>First of all, we need to find a way to send a request to ElasticSearch that contains the query with highlighting. Since there is no functionality for highlighting itself, we have to use the rawBody feature of kt-search.</p><p>Since this query can get pretty long, I created a separate function for it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4894ac2455caed3d0c783d7c8dfb25f7/href">https://medium.com/media/4894ac2455caed3d0c783d7c8dfb25f7/href</a></iframe><p>Please keep an eye on the lower part with the highlight block:</p><ul><li>The pre- and post-tags wrap around the matching terms. If you don’t define them, ElasticSearch uses &lt;em&gt; and &lt;/em&gt; for it.</li><li>The fields block defines, from which field, the best matches should be returned.<br>ElasticDocument is a data class with a value called content . I use this, because I can then refactor via IntelliJ and don’t have to change the String manually, every time i change the name of this attribute.</li><li>fragment_size defines the maximum length of the fragments. Fragments are a combination of the matched term with some context around it.</li><li>number_of_fragments defines how many of these fragment Strings are getting returned in one request.</li><li>The fragmenter and type should be chosen based on this <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/highlighting.html#set-highlighter-type">information</a>.</li></ul><h4>Receiving the response</h4><p>Since we now have the query for highlighting, we could use the search() function of the library right? No.</p><p>The SearchResponse of the search function does not contain highlighting. This means that we have to do it ourselves.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/55801b60950d470a4234269957a6ead8/href">https://medium.com/media/55801b60950d470a4234269957a6ead8/href</a></iframe><p>This function only returns the JSON as a String. It uses the restClient of the SearchClient to create a new post request. The path is set together by the name of the index and the function that we want to execute on ElasticSearch. In our case, it is _search .</p><p>The rawBody uses the JSON String that we defined in the createQuery() function.</p><h4>Deserializing the SearchResponse</h4><p>Lastly, we need to deserialize the String with the search results into a list of hits. Normally this would be handled by kt-search but since the default SearchResponse of kt-search doesn’t contain the highlighting info, we need to write it ourselves.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f05eb662b02dd04e165518ae29a22c7f/href">https://medium.com/media/f05eb662b02dd04e165518ae29a22c7f/href</a></iframe><p>These are the Serializable classes that are needed for the deserialization.</p><p>Note that the Hit class filters out the highlighted terms, so that you don’t have the &lt;highlighted&gt; wrappers in your search results.</p><p>Now we can combine all these steps into our own search function, so that we can call it just like we would normally do.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/829546f8a2e11f8d825e3b0abb55b795/href">https://medium.com/media/829546f8a2e11f8d825e3b0abb55b795/href</a></iframe><p>Since we annotated all the classes from above with @Serializable , we can now call the decodeFromString method of the DEFAULT_JSON object provided by kt-search.</p><p>This gives us back the SearchResponse with its nested hits. We can access them by calling searchResult.hits .</p><p>Finally we have hits that contain highlighting information.</p><h3>Reflection</h3><h4>What went good</h4><p>In my opinion, the actual implementation of the self written logic to retrieve highlighting information went pretty good. I did not have that much problems since I could test the behaviour with unit tests.</p><h4>What needs improvement</h4><p>The biggest problem that I had was understanding that the SearchResponse of kt-search really does not contain any information about highlighting. I was confused, because I sent a query with a highlighting request, but I didn’t get any result from it. I finally understood it when I sent a request via curl and saw, that it has something todo with the SearchResponse of kt-search. Next time, I would read the JavaDoc of the response object first, so that I can see on first sight that the field I’m looking for is not contained in it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e4b35ee1780e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Managing dependencies between Kubernetes Services]]></title>
            <link>https://medium.com/geekculture/connecting-two-services-with-dependencies-in-kubernetes-8e6b05b4ae4d?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e6b05b4ae4d</guid>
            <category><![CDATA[livenessprobe]]></category>
            <category><![CDATA[spring]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[minikube]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Thu, 29 Sep 2022 06:26:40 GMT</pubDate>
            <atom:updated>2022-09-29T09:53:28.060Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/462/1*KupWIYrQFcTuNkrb_p3vQQ.png" /></figure><p>Last week I had the job to add ElasticSearch to my project. Since my Application relies now on a third party service, I have to make sure that ElasticSearch is running. On startup, my application uses ElasticSearch to build up an index. This means ElasticSearch has to be started before my Application. Otherwise my Application will fail at startup.</p><h3>What are Liveness-, Readiness- and Startup-Probes</h3><p>According to <a href="https://kubernetes.io/">kubernetes.io</a>:</p><blockquote>The <a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/">kubelet</a> uses <strong>liveness probes</strong> to know <strong>when to restart a container</strong>. For example, liveness probes could catch a deadlock, where an application is running, but unable to make progress. Restarting a container in such a state can help to make the application more available despite bugs.</blockquote><blockquote>The kubelet uses <strong>readiness probes</strong> to know when a container is <strong>ready to start accepting traffic</strong>. A Pod is considered ready when all of its containers are ready. One use of this signal is to control which Pods are used as backends for Services. When a Pod is not ready, it is removed from Service load balancers.</blockquote><blockquote>The kubelet uses <strong>startup probes</strong> to know when a container <strong>application has started</strong>. If such a probe is configured, it disables liveness and readiness checks until it succeeds, making sure those probes don’t interfere with the application startup. This can be used to adopt liveness checks on slow starting containers, avoiding them getting killed by the kubelet before they are up and running.</blockquote><p>For our use case, this means that we can define a liveness probe on our own application, that checks, whether or not a connection to ElasticSearch can be established. If the probe failed, the application should restart and if it succeeded, it should keep running.</p><h3>Creating a Liveness-Probe with HTTP</h3><p>There are different types of liveness probes. We can use one of the following:</p><ul><li>Liveness Command</li><li>Liveness HTTP request</li><li>Liveness gRPC request</li></ul><p>For our example, we will use a simple HTTP request.</p><p>It is important to know that every HttpStatusCode between 200 and 400 will result in a positive probe, while everything above 400 or below 200 will result in a failed probe.</p><h4>Rest endpoint</h4><p>In our ElasticAdapter, we have a method that throws an exception if the cluster is not reachable.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0892d74c79223f261c8945319d0a1ee9/href">https://medium.com/media/0892d74c79223f261c8945319d0a1ee9/href</a></iframe><p>As you can see, it simply makes a call to clusterHealth() and does nothing with the result if it gets one. If it throws a ConnectionException it will catch it and throw a ElasticSearchUnreachableException() .</p><p>To make this check available to the outside, we need to create a new REST endpoint.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8fdd815263c3bf939d2f5b9fcbdb20b9/href">https://medium.com/media/8fdd815263c3bf939d2f5b9fcbdb20b9/href</a></iframe><p>This Controller creates a GetMapping for the /liveness endpoint which calls the isReachable() function of the elasticAdapter. If the check succeeds, it won’t throw an exception and HttpStatus.OK will be returned. If it fails, we will catch the exception and return HttpStatus.SERVICE_UNAVAILABLE which has a Status code of 503.</p><p>Since Status code 503 is higher than 400, it will make the probe fail.</p><h4>Add Liveness probe to Kubernetes Deployment</h4><p>Now we can use our REST endpoint to create a liveness probe in our deployment.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f0ca8d59e382fad9d0caf83e92209f44/href">https://medium.com/media/f0ca8d59e382fad9d0caf83e92209f44/href</a></iframe><p>In our liveness probe, we have to define the port on which our application runs on and the path on which our liveness probe should be checked. In our case, we defined it as /liveness . The initialDelaySeconds delays the liveness probe for the first three seconds. This is needed to give our own application some time to startup and make the endpoint available. The periodSeconds is the time between each probe.</p><p>This probe helps us in two scenarios:</p><ol><li><strong>Our application has started, while ElasticSearch is still unavailable. </strong><br>- Restarts our application until ElasticSearch is reachable and lets it run.</li><li><strong>Both apps started successfuly but ElasticSearch crashes.</strong><br>- Restarts our application until ElasticSearch is reachable again. <br>- Ensures that we always have both applications running and don’t loose data.</li></ol><h3>Reflection</h3><h4>What went good</h4><p>In my opinion, the Kubernetes documentation about livenessProbes was really good and it was really easy to implement. Also, creating the Rest-Endpoint itself was no problem for me, since I already did that a few times.</p><h4>What needs improvement</h4><p>For me, the biggest problem was testing the liveness probe. Since we use OpenShift in production, I wanted to use <a href="https://github.com/code-ready/crc">OpenShift CRC</a>. This did not work properly on my machine and I needed to restart the whole cluster a few times. After that, I switched to <a href="https://minikube.sigs.k8s.io/docs/start/">Minikube</a>, which solved all my issues. It was blazing fast, compared to the slow and resource intensive CRC. Next time, I would directly use Minikube to test local deployments or ask my team members, how they would test something like this.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e6b05b4ae4d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/connecting-two-services-with-dependencies-in-kubernetes-8e6b05b4ae4d">Managing dependencies between Kubernetes Services</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Spring TestConfigurations with Kotlin]]></title>
            <link>https://medium.com/geekculture/spring-testconfigurations-with-kotlin-b3b8d3ae5236?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3b8d3ae5236</guid>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[spring]]></category>
            <category><![CDATA[junit]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[test-configuration]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Thu, 08 Sep 2022 06:55:53 GMT</pubDate>
            <atom:updated>2022-09-22T06:36:33.495Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*cvDeJACC-YyLf3-_ii2otQ.png" /></figure><p>This week, I had to add Elasticsearch to my project. Since the client for Elasticsearch needs a running server to establish a connection and make requests, my project now has a dependency to other infrastructure. This was not a really big problem, because I could just mock the client for the tests that are using the client.</p><p>A little later, I added logic to initialize the Elasticsearch index on startup of my application. This was when the problem became obvious. Since this startup function was calling functions of the client, I would have needed to mock the client for every test, which can work for a few tests, but gets really messy when you have a few hundreds. 😅. Because of that reason, I started using Spring TestConfigurations.</p><h3>What is a TestConfiguration?</h3><p>According to <a href="https://reflectoring.io/">Reflectoring.io</a>:</p><blockquote>The @TestConfiguration annotation is <strong>a useful aid for writing unit tests of components in a Spring Boot application</strong>. It allows us to define additional beans or override existing beans in the Spring application context to add specialized configurations for testing.</blockquote><p>Specifically for our use case, we can overwrite an existing bean for the tests that don’t actually use the real implementation. We can define a mock and its behaviour once, and can use this configuration on every test class that we want.</p><h3>Creating a TestConfiguration</h3><p>First of all, we need to create a new file to place our configuration in.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/be27792f713c4279df9f65b3b349182d/href">https://medium.com/media/be27792f713c4279df9f65b3b349182d/href</a></iframe><p>Here we annotate the configuration class with @TestConfiguration to exclude it from Component-Scanning and letting Spring know that we want to define a new configuration.</p><p>After that, we define a new elasticClient bean which is a standard Kotlin function that returns the mock that we defined in it. The only special thing is, that you have to annotate it with @Bean . Note that the name of this function needs to be the same as the name of the bean that you want to inject from another class.</p><h3>How to use the Configuration</h3><p>First open the test, that you want this configuration to apply. This test might look something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6c691096484be3e57f973413d5b15a56/href">https://medium.com/media/6c691096484be3e57f973413d5b15a56/href</a></iframe><p>As you can see, an ElasticSearchAdapter gets mocked, which in its original implementation would have a save method, which calls methods of the elasticClient that relies on external infrastructure.</p><p>The minimal implementation of the StandardDocumentationRepository looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bd111faf36ac656fe14fe77606890d14/href">https://medium.com/media/bd111faf36ac656fe14fe77606890d14/href</a></iframe><p>Since the onStartup() function runs even before the BeforeEach() function, we still use the original beans instead of our mocks. This naturally leads to failing tests.</p><p>The solution to this problem is really simple. We use our configuration, that we created above and apply it to every test. A minimal example looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ab699b008ec3324cb1938b28e917bb8b/href">https://medium.com/media/ab699b008ec3324cb1938b28e917bb8b/href</a></iframe><p>The only thing we added is the @Import annotation. This makes sure that our configuration gets loaded and the bean gets overwritten even before the onStartupEvent function.</p><p>One last thing to do is to add the following line to your src/test/resources/application.properties file.</p><p>Please take notice that it’s really important to put it in the test properties and not into the main properties, since we don’t want to override beans in the main context.</p><pre><em>spring.main.allow-bean-definition-overriding</em>=<em>true</em></pre><p>This makes sure, that we can overwrite the existing bean with our mocked bean.</p><p>This leads us to having green tests again, since the mock in our test can now overwrite the configuration with its own specific behaviour.</p><h3>Reflection</h3><h4>What went good</h4><p>There were many great resources available for Java, so I understood the concept quite easily. The actual implementation of this configuration was also pretty simple.</p><h4>What needs improvement</h4><p>What really took time, was to understand why the tests where failing in first place. First I thought that it might be because I had two “ApplicationReadyEvent” functions in my project, but this is allowed in Spring. After reading the logs of Spring, it became obvious, that it had something to do with the ElasticClient. Especially with its connection to the server (which wasn’t running… obviously). Next time I would read the logs first, and not only the first part of the exception.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3b8d3ae5236" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/spring-testconfigurations-with-kotlin-b3b8d3ae5236">Spring TestConfigurations with Kotlin</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Elasticsearch with Kotlin]]></title>
            <link>https://medium.com/geekculture/using-elasticsearch-with-kotlin-f802a738b389?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/f802a738b389</guid>
            <category><![CDATA[search]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[indexing]]></category>
            <category><![CDATA[spring]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Thu, 25 Aug 2022 05:46:44 GMT</pubDate>
            <atom:updated>2022-08-26T07:00:19.501Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/719/1*uJfwTGVDbrBCmq63l-Zf9A.png" /><figcaption>Indexing Elastic Documents with Kotlin</figcaption></figure><p>Last week, I had to add search functionality to my website. There already was some kind of search mechanism, but it could only list files based on their title and not their content. Since the content is the most important part of these files, I needed to find ways to dynamically search the content of files without having to deal with performance issues. That was exactly where I stumbled upon Elasticsearch.</p><h3>What is Elasticsearch?</h3><p>According to Elastics own <a href="https://www.elastic.co/de/what-is/elasticsearch">website</a>…</p><blockquote>Elasticsearch is <strong>a distributed, free and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured</strong>. Elasticsearch is built on Apache Lucene and was first released in 2010 by Elasticsearch N.V. (now known as Elastic)</blockquote><p>This means that you can search pretty much any kind of data stored as Chars, as long as you map it into the right base-format.</p><h3>How Elasticsearch works</h3><p>Elasticsearch takes in data from different locations, stores and indexes it according to user-specified mapping (which can also be derived automatically from data), and makes it searchable. Its distributed architecture makes it possible to search and analyze huge volumes of data in near real time.</p><p>The main benefit of Elasticsearch is its speed. Normally, when you would filter your content manually, it would take ages to find the according document. With Elasticsearch, the search takes dramatically less time, because of its clever storage and indexing in JSON format.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*hcerSho1pTgiQphp1dWFPQ.png" /></figure><p>Elasticsearch uses Shards, to distribute the data in the cluster. A shard is the unit in which the data of the cluster gets organized. Each index can constist of at least one to multiple shards. When a disk suddenly stops working, there are replicas on the other nodes of the cluster to restore the data.</p><h3>How to use Elasticsearch</h3><p>Elasticsearch provides many possibilities to interact with its API. On the one hand, there is the standard REST API, on the other hand, there are many different <a href="https://www.elastic.co/guide/en/elasticsearch/client/index.html">API clients</a> available for every major programming language. Obviously there is one for java, which we could also use in Kotlin, but this client needs a lot of third party dependencies to work. Most of these dependencies are not common in a standard Kotlin project and are normally replaced by their Kotlin alternatives. You could change the used dependencies, but this would make the project unnecessarily complex.</p><p>The good thing is, that there is a great community-supported Kotlin-client available on <a href="https://github.com/jillesvangurp/kt-search">Github</a>. This client relies on standard Kotlin libraries that don’t interfere with the rest of the project.</p><h4>Creating the client</h4><p>First of all, we need to declare the dependency in our build.gradle.kts file.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6e1f2635a0c7ede6c3f3b19ac8734f04/href">https://medium.com/media/6e1f2635a0c7ede6c3f3b19ac8734f04/href</a></iframe><p>After that, we can create our client like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/46398e604436c59007dc1153432d37f2/href">https://medium.com/media/46398e604436c59007dc1153432d37f2/href</a></iframe><h4>Indexing documents</h4><p>The next thing to do is to create a data class which will be used to upload data as JSON to the index of Elasticsearch.</p><p>But first, we need to know what an index is. According to the Elastic website:</p><blockquote>An index is like a ‘database’ in a relational database. It has a mapping which defines multiple types.<br>An index is a logical namespace which maps to one or more primary shards and can have zero or more replica shards.</blockquote><p>Ok. So there are two concepts in that definition. First, an index is some type of data organization mechanism, allowing the user to partition data a certain way. The second concept relates to replicas and shards, the mechanism Elasticsearch uses to distribute data around the cluster.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/80f4293de283686ac55fe6f9bb782707/href">https://medium.com/media/80f4293de283686ac55fe6f9bb782707/href</a></iframe><p>As you can see, the indexing function takes a list of ElasticDocuments(which could be any data class) as an argument and loops through them. I have chunked this list into smaller pieces, because there is a limit on how many documents you can index in one bulk request. The exact value of “ES_BULK_CHUNK_SIZE” can only defined by trying out and keeping an eye on the performance.</p><p>After that, I call the bulk method of the client and index my ElasticDocument. Here it is important to know that you can use the DEFAULT_JSON provided by kt-search and don’t need to create your own JSON mapper. Each Document needs an ID. Thats why we set the esDocument-id as the id of the resource. Another thing to note, is, that you always need to define the name of the index in which the document should be saved at.</p><h4>Searching in the index</h4><p>The most important part of Elasticsearch is the search itself. Elasticsearch provides many possible combinations of search filters, but I will only look at the one that I used in my project.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4493d1cb05a5643f42bb3f8497a0a94b/href">https://medium.com/media/4493d1cb05a5643f42bb3f8497a0a94b/href</a></iframe><p>This function returns us only the documents that contain parts of the title and the version that we are providing. In this case “Some Title” and “v1.2.3”.</p><p>You can configure the fuzzyness of this search query by providing the fuzziness attribute and configuring a different value.</p><p>If you want to exactly match values with a query, you can use the term matcher. This matcher straight up compares the given value with the index and returns its results. It doesn’t use any kind of fuzzy matching and is therefore ideal for deleting documents based on their identifying attributes.</p><p>You can add more complexity by adding other keywords like should or by adding more parameters. <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html">Here</a> is a list of what is possible with ElasticSearch query-DSL.</p><p>If you want to seee some more great examples, take a look at this <a href="https://coralogix.com/blog/42-elasticsearch-query-examples-hands-on-tutorial/">blogpost</a>.</p><p>The .ids function at the end of the call, returns only the Ids of the found search results. If you need the whole object, you need to call .hit.hit.source.toString , which will return the object as a JSON String.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f637745259297b23a16a3cb209b9c8d2/href">https://medium.com/media/f637745259297b23a16a3cb209b9c8d2/href</a></iframe><h3>Reflection</h3><h4>What went good</h4><p>I think the actual implementation of the search, after I figured out which plugin to use, worked pretty good and was self explanatory.</p><h4>What needs improvement</h4><p>I wasted way too much time on trying to use the Java client instead of looking for more suitable solutions for my problem. The next problem was how I could test it. First, I was only able to test it manually by running a local Docker container, which is not optimal by any means. After I asked my team for any better solutions, I started using <a href="https://www.testcontainers.org/">Testcontainers</a> for my Integration tests. With these tests, my code became much more reliable and I was finally able to write some advanced logic for my project.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f802a738b389" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/using-elasticsearch-with-kotlin-f802a738b389">Using Elasticsearch with Kotlin</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Clean Code Development | Integration Operation Separation Principle]]></title>
            <link>https://medium.com/geekculture/clean-code-development-integration-operation-separation-principle-ec85a0e47bcb?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec85a0e47bcb</guid>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[clean-code]]></category>
            <category><![CDATA[software-design]]></category>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[iosp]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Fri, 18 Mar 2022 06:15:35 GMT</pubDate>
            <atom:updated>2022-03-18T18:25:55.576Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*O6EVllsGHLIM5CSOGOYM7g.png" /></figure><p>This week, I had to add a new feature to a project that I started a whole year ago and didn’t touch for a pretty long time. When I first looked at my code that I considered pretty clean at the moment I wrote it, I was shocked, how inconsistent and unstructured it was. I had to invest a lot of time, just to get a clear understanding of what my code does.</p><p>Since I can’t always invest that amount of time, just to understand, what i wanted to express with my code, I took a look at some Clean Code Strategies. The one, that I found most important was the Integration Operation Segregation Principle (IOSP).</p><h3>What is Integration Operation Segregation</h3><p>IOSP (Integration Operation Segregation Principle) is a Clean Code strategy, that allows us to clearly separate between integration code and operational code. It allows us to systematically reduce the complexity of large methods into much simpler ones.</p><p><strong>Integration Methods</strong> only compose other units (including other Integration Methods) but do not contain any logic.</p><p><strong>Operational Methods</strong> just contain logic and never integrate any other functional Methods. They should be really small, composable and easy to read.</p><h3>How to apply this principle</h3><p>If you have big methods or just messy methods in general, you can apply this pattern in just a few simple steps.</p><ol><li>Find big methods that do operational and integration tasks at once</li><li>Split them up into smaller methods</li><li>Classify your methods as either <strong>Integration Methods</strong> or <strong>Operational Methods</strong></li></ol><p>If you aren’t sure, if a method is integrational or operational, the method needs to be split up even more. If you did that right, you should now have a really easy to read integration method, that implements the other methods.</p><h4>Example</h4><p>Here I have a small example class with just one big, junky method in it. On the first sight, you can’t really see what this function is all about. You can see the title of it, but what it actually does inside, is really difficult to tell.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/90c53790c2f97752df1f85e6d547813e/href">https://medium.com/media/90c53790c2f97752df1f85e6d547813e/href</a></iframe><p>In the first step, we can extract a new function from line 14. This function could be called generateKrokiImage() since it calls the generateImage() function of the krokiAdapter.</p><p>Next, we can extract a new function from line 11 to 16, and call it something like replaceAdocDiagramWithKroki() . This function replaces the old value that was found by the regex, with the new one that got generated by our generateKrokiImage() function.</p><p>The next function we can extract sits on lines 8to 10. It walks thru the whole file tree and checks each item, if it really is an asciidoc file and matches the given regex. We can call this function forEachAdocFile() .</p><p>Lastly, we have a really easy to understand function, that is well structured and maintainable. It could look something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c1e26fae3663f897afbafbd07fb7c82c/href">https://medium.com/media/c1e26fae3663f897afbafbd07fb7c82c/href</a></iframe><p>As you can clearly see, the convert function is the only integration method in this class, and is the one, that gets called from the outside. All the other private methods are operational.</p><h3>Reflection</h3><h4>What went good</h4><p>The actual implementation of the principle went really good and was pretty straight forward because of the refactoring methods of IntelliJ IDEA.</p><h4>What needs improvement</h4><p>First, I didn’t really get the purpose of this pattern, because in the end, the code was longer, than it was before (because of the whitespace between different methods). I got a little confused by that and had to ask my mentor. After a quick discussion, he cleared things up and it was clear to me, that with this way, you can see on the first sight, what the whole class or method does.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec85a0e47bcb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/clean-code-development-integration-operation-separation-principle-ec85a0e47bcb">Clean Code Development | Integration Operation Separation Principle</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Domain Driven Design | Deep dive into context mapping]]></title>
            <link>https://medium.com/geekculture/domain-driven-design-deep-dive-into-context-mapping-f9cd8acddd7e?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9cd8acddd7e</guid>
            <category><![CDATA[domain-driven-design]]></category>
            <category><![CDATA[software-design]]></category>
            <category><![CDATA[eric-evans]]></category>
            <category><![CDATA[context-mapping]]></category>
            <category><![CDATA[ddd]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Mon, 21 Feb 2022 14:17:30 GMT</pubDate>
            <atom:updated>2022-03-02T06:41:07.154Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/721/1*9m0eBRsmz2-_in8K9_D_gA.png" /></figure><p>A few months ago, I created a story about Domain Driven Design (DDD) with a pretty high level overview about what is and is not part of the DDD language. In this story I want to have a more specific eye on the Context Maps in DDD.</p><h3>Why use Context Mapping?</h3><p>A context map, as the name already suggests, helps you define the relationships between the different contexts of your application in a mostly visual way. This is very useful, if for example a word has one meaning in one context, and a totally different in another one. This can happen quite fast and will result in a confusing language barrier between the two contexts.</p><h3>What is Context Mapping?</h3><p>Context mapping is a tool that allows developers and domain experts to identify the relationship between bounded contexts and the relationship between the teams that are responsible for them.</p><p>There are different ways of how we can integrate between two or more bounded contexts:</p><ul><li><strong>Partnership</strong></li><li><strong>Shared kernel</strong></li><li><strong>Customer supplier</strong></li><li><strong>Conformist</strong></li><li><strong>Anticorruption Layer</strong></li><li><strong>Open Host Service</strong></li><li><strong>Published Language</strong></li><li><strong>Separate Ways</strong></li><li><strong>Big Ball of Mud</strong></li></ul><h4>Partnership</h4><p>When teams in two Contexts will succeed or fail together, a cooperative relationship needs to emerge. The teams institute a process for coordinated planning of development and joint management of integration. The teams must cooperate on the evolution of their interfaces to accommodate the development needs of both systems. Interdependent features should be scheduled so that they are completed for the same release.</p><h4>Shared Kernel</h4><p>Sharing part of the model and associated code forms a very intimate interdependency, which can leverage design work or undermine it. Designate with an explicit boundary some subset of the domain model that the teams agree to share. Keep the kernel small. This explicit shared stuff has special status and shouldn’t be changed without consultation with the other team. Define a continuous integration process that will keep the kernel model tight and align the Ubiquitous Language of the teams.</p><h4>Customer-Supplier Development</h4><p>When two teams are in an upstream-downstream relationship, where the upstream team may succeed interdependently of the fate of the downstream team, the needs of the downstream team come to be addressed in a variety of ways with a wide range of consequences. Downstream priorities factor into upstream planning. Negotiate and budget tasks for downstream requirements so that everyone understands the commitment and schedule.</p><h4>Conformist</h4><p>When two development teams have an upstream/downstream relationship in which the upstream team has no motivation to provide for the downstream team’s needs, the downstream team is helpless. Altruism may motivate upstream developers to make promises, but they are unlikely to be fulfilled. The downstream team eliminates the complexity of translation between bounded contexts by slavishly adhering to the model of the upstream team.</p><h4>Anticorruption Layer</h4><p>Translation layers can be simple, even elegant, when bridging well-designed Bounded Contexts with cooperative teams. But when control or communication is not adequate to pull off a shared kernel, partner, or customer-supplier relationship, translation becomes more complex. The translation layer takes on a more defensive tone. As a downstream client, create an isolating layer to provide your system with functionality of the upstream system in terms of your own domain model. This layer talks to the other system through its existing interface, requiring little or no modification to the other system. Internally, the layer translates in one or both directions as necessary between the two models.</p><h4>Open Host Service</h4><p>Define a protocol that gives access to your subsystem as a set of services. Open the protocol so that all who need to integrate with you can use it. Enhance and expand the protocol to handle new integration requirements, except when a single team has idiosyncratic needs. The, use a one-off translator to augment the protocol for that special case so that the shared protocol can stay simple and coherent.</p><h4>Published Language</h4><p>The translation between the models of two Bounded Contexts requires a common language. Use a well-documented shared language that can express the necessary domain information as a common medium of communication, translating as necessary into and out of that language. Published Language is often combined with Open Host Service.</p><h4>Separate Ways</h4><p>We must be ruthless when it comes to defining requirements. If two sets of functionality have no significant relationship, they can be completely cut loose from each other. Integration is always expensive, and sometimes the benefit is small. Declare a Bounded Context to have no connection to the others at all, enabling developers to find simple, specialized solutions within this small scope.</p><h4>Big Ball of Mud</h4><p>As we survey existing systems, we find that, in fact, there are parts of systems, often large ones, where models are mixed and boundaries are inconsistent. Draw a boundary around the entire mess and designate it a Big Ball of Mud. Do not try to apply sophisticated modeling within this Context. Be alert to the tendency for such systems to sprawl into other Contexts.</p><p>To sum up, understanding various context mapping techniques allows a more effective integration between Bounded Contexts. It is also important to think about whether integration is necessary and brings benefits to the business in the first place. Using multiple approaches at the same time is also acceptable and sometimes preferred. For example, having a RESTful API naturally provides <strong>OHS</strong>, but at the same time, the downstream may also be encouraged to implement its own <strong>ACL</strong>, especially when the upstream service is by third party. You and your team will be the best people to decide which approaches to use based on the respective situation.</p><p>If you want to get more information about Context Mapping, visit <a href="https://github.com/ddd-crew/context-mapping">https://github.com/ddd-crew/context-mapping</a></p><p>For more information about DDD in general, I can only recommend the book <a href="https://www.amazon.de/Implementing-Domain-Driven-Design-Vaughn-Vernon/dp/0321834577">Implementing Domain Driven Design</a> by <a href="https://www.amazon.de/Vaughn-Vernon/e/B0096T71SA/ref=dp_byline_cont_book_1">Vaughn Vernon</a></p><h3>Reflection</h3><h4>What went good</h4><p>Since this post was all about context mapping based on DDD by Eric Evans, I was able to get most of the information about this topic from the book, which was really great.</p><h4>What needs improvement</h4><p>Next time, I would take a quick look at some examples before I begin to read the theory. This way, I could get a clearer picture upfront about what this is all about and could connect the newly learned stuff with the example.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9cd8acddd7e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/domain-driven-design-deep-dive-into-context-mapping-f9cd8acddd7e">Domain Driven Design | Deep dive into context mapping</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kotlin | Create tree structure from a list of metadata]]></title>
            <link>https://medium.com/geekculture/kotlin-create-tree-structure-from-a-list-of-metadata-510c787adcf4?source=rss-1b0e7cac3855------2</link>
            <guid isPermaLink="false">https://medium.com/p/510c787adcf4</guid>
            <category><![CDATA[thymeleaf]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[tree-structured-data]]></category>
            <category><![CDATA[datastrucutre]]></category>
            <dc:creator><![CDATA[Manserpatrice]]></dc:creator>
            <pubDate>Fri, 14 Jan 2022 07:06:02 GMT</pubDate>
            <atom:updated>2022-01-15T07:44:51.294Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/426/1*Vby-8gXo4Iv29VLyidpaYw.png" /></figure><p>A few weeks ago, I had a project where I wanted to display a menu with different entries and hierarchies on a web-app. The picture above is the end result of this tree structure. To do this, first of all, I needed to create the menu structure itself. The only input that I got at this point was a list of metadata, that had these attributes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/337/1*Ff-Q9SukmZF9IPB6HlxhSA.png" /></figure><p>Most of you will already recognize, that we have a hierarchy in it, which will be quite important for the future steps. This hierarchy is just a string, divided with forward-slashes.</p><h3>Defining a class diagram</h3><p>First of all, we need to create a node class, which is responsible for the structure itself, since it links to instances of itself in the children map. This way we can go through the whole hierarchy and add as many hierarchy levels as there are needed.</p><p>If the current node is really just a node and not a leaf-node, hence it is not the last one in the hierarchy, then, it does not need to contain metadata. Only the leaf nodes do require the metadata.</p><p>Every node needs to have a title. The leaf nodes take the title from the metadata title and the hierarchy nodes take the title from the hierarchy step they are currently on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/1*RRWwnPgPKfXxLwNRWwd9Sg.png" /></figure><p>The root class inherits from the node class. It can be created without any constructor parameters and can create a complete hierarchy tree with just one function call. There is also a possibility to add only one child to an existing tree.</p><h3>Implementing the Node class</h3><p>The getItemOrAddChild function just creates a new leaf or hierarchy node based on the hierarchyList’s size.</p><p>The createLeafNode function just adds a new node to the current nodes children.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/249d79f542cbe71540fae70f59c51c41/href">https://medium.com/media/249d79f542cbe71540fae70f59c51c41/href</a></iframe><p>The createHierarchyNode function takes the current hierarchy and adds the underlying hierarchy as a new node to the current one. If the children of it already contain the currentHierarchy, it will just take it and go through the same process again, until there is no hierarchy anymore.</p><p>All these functions will return a Node, but the most important one of them is the getItemOrAddChild function, since it is the one that will be called from the root Class.</p><h3>Implementing the Root class</h3><p>As you can see, the Root class is implementing the Node class and therefore has access to all the attributes and functions that the Nodes have.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ace8e90a84167bca4b063448b99fff36/href">https://medium.com/media/ace8e90a84167bca4b063448b99fff36/href</a></iframe><p>The createHierarchyList function will just split up the hierarchy string into a mutable list. This list gets used on the addChildToTree function. It simply calls the getItemOrAddChild function that we defined in the Node class earlier and returns the root object.</p><p>The createWholeTree function just calls the addChildToTree function for every item in its list and sorts it alphabetically based on its title. In the end, it will also return the root object itself.</p><p>If we run this function with a list of metadata, it will create the whole tree for us and return the root object to us. In this root object, we can see all the leaf nodes on first level and also the hierarchy nodes with deeper hierarchies.</p><p>To display our generated data structure, check out my follow-up article about <a href="https://manserpatrice.medium.com/thymeleaf-create-dynamic-hierarchy-trees-with-kotlin-micronaut-4b4449574f85">displaying tree structures with Thymeleaf</a>.</p><h3>Reflection</h3><h4>What went good?</h4><p>I guess the thing that went best, was when I already had the Node class defined and only needed to access its functions with the root class. The root class was pretty easy to implement and since it was not referencing itself the whole time, it was also pretty easy to debug and test.</p><h4>What needs improvement?</h4><p>At the beginning I had some problems with the understanding of this whole principle of self referencing classes. I did not know this compound pattern and first needed to read into it to fully understand it. Next time a have such a problem, I will first search for a fitting build pattern and implement it directly this way. It would have saved me lots of time trying to come up with my own custom solutions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=510c787adcf4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/kotlin-create-tree-structure-from-a-list-of-metadata-510c787adcf4">Kotlin | Create tree structure from a list of metadata</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>