<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Isuru Perera on Medium]]></title>
        <description><![CDATA[Stories by Isuru Perera on Medium]]></description>
        <link>https://medium.com/@isurutharanga?source=rss-7abfec95c9d2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*HM79mf2-_ysSBabYCqhO5A.jpeg</url>
            <title>Stories by Isuru Perera on Medium</title>
            <link>https://medium.com/@isurutharanga?source=rss-7abfec95c9d2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 05:30:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@isurutharanga/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AI Code Reviews with Copilot in IntelliJ IDEA]]></title>
            <link>https://medium.com/@isurutharanga/ai-code-reviews-with-copilot-in-intellij-idea-b1adeec6824d?source=rss-7abfec95c9d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/b1adeec6824d</guid>
            <category><![CDATA[intellij]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[copilot-programming]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Isuru Perera]]></dc:creator>
            <pubDate>Sun, 23 Mar 2025 13:45:50 GMT</pubDate>
            <atom:updated>2025-03-23T13:56:06.715Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dezIT1PZ9w0bFetHqK6A9A.png" /></figure><h4><strong>TL;DR</strong></h4><p>The following article describes my workflow for performing AI code reviews on any code changes I’m about to merge with the main branch before the human code reviewers even see a line of my code.</p><p>I have summarized my workflow in a 4 step process, and given simple examples for anyone interested in following the same workflow.</p><h4><strong>Why code reviews?</strong></h4><p>Code reviews are an essential part of any software project to maintain good coding standards. One of the main focuses of a good code review would be to adhere to the <a href="https://docs.sonarsource.com/sonarqube-server/9.8/user-guide/clean-as-you-code/">“Clean as you code”</a> approach, where we make sure any new code we add to the project is compliant with the best practices and standards in Software Engineering.</p><h4><strong>How code reviews are typically done?</strong></h4><p>Usually, the code reviews are done by your peers on any new code you are about to merge into the main branch. The code review would be more focused on the new code, or the “code changes” you have done in your merge request.</p><p>In any code review, the reviewer would check if the new code changes are following the standards or best practices in the industry/team/project. Also, the code reviewer would focus on any functional or technical issues that may have been introduced in the new code changes.</p><p>If you have read up to this point, I assume you would know how a code review is done even before reading this article. So, let’s skip to the good part!</p><h4><strong>Break down of workflow</strong></h4><p>For any automation, we would first break down the manual process at hand into multiple steps that we can automate. So, let’s break down a typical code review into a few steps, and look at how we can automate those steps with the tools we already have.</p><blockquote>1. Extract the new code changes that are being merged</blockquote><blockquote>2. Review the new code changes in terms of best practices</blockquote><blockquote>3. Suggest any improvement points on the changes</blockquote><blockquote>4. Iterate through step 1–3 until the code changes are complient (or you’re satisfied)</blockquote><h4><strong>Copilot for Code Reviws</strong></h4><p>As we already know, <a href="https://docs.github.com/en/copilot/using-github-copilot/code-review/using-copilot-code-review">Github.com</a> and <a href="https://graphite.dev/guides/how-ai-code-review-works-in-vscode">VSCode</a> support code reviews in their platform via Copilot. However, the following article is about performing code reviews generically on any platform without locking ourselves into a specific vendor.</p><h4><strong>Step 1: How to extract new code</strong></h4><p>Git diff is a powerful tool to see the new code changes that are being submitted in a merge request. Typically in any platform, the <a href="https://github.com/features/code-review">code review interface</a> would show a git diff between the existing code and the new code that’s being added. We will now see how we can extract a git diff from the code that you’re going to merge into a text file.</p><p><strong><em>Ex 1: If you haven&#39;t already committed your code</em></strong></p><pre>git diff &gt; code-changes.txt</pre><p>The above command will persist the difference between the previous state of your current branch and any of your uncommitted changes into file <em>code-changes.txt</em>.</p><p><strong><em>Ex 2: If you have committed your code</em></strong></p><pre>git diff main &gt; code-changes.txt</pre><p>The above command will persist the difference between the main branch and your current branch into file <em>code-changes.txt</em>.</p><p>The above two examples are very basic usages of the git diff command. As software engineers, we all know how to get creative with git-diff and get the diff between the two states of your repository.</p><p>Here, we can be creative and extract only source code changes. If your project is a Java project, you can limit your diff to return only Java source files. This approach would eliminate any unrelated changes in your code changes which could possibly lead to LLM hallucinations.</p><p><strong><em>Ex 3: Focus only on source code changes</em></strong></p><pre>git diff main -- &#39;*.java&#39; &gt; code-changes.txt</pre><p>The extracted git diff in code-changes.txt would look like below.</p><pre>diff --git a/src/main/java/org/example/Main.java b/src/main/java/org/example/Main.java<br>index 63133db..95d39ac 100644<br>--- a/src/main/java/org/example/Main.java<br>+++ b/src/main/java/org/example/Main.java<br>@@ -1,5 +1,8 @@<br> package org.example;<br> <br>+import java.time.Duration;<br>+import java.time.LocalTime;<br>+<br> public class Main {<br>     public static void main(String[] args) {<br>         findTime(&quot;10:00&quot;, &quot;12:00&quot;);<br>@@ -7,6 +10,9 @@ public class Main {<br>     }<br> <br>     private static void findTime(String start, String end) {<br>-        System.out.println(&quot;Time between &quot; + start + &quot; and &quot; + end + &quot; is&quot;);<br>+        LocalTime startTime = LocalTime.parse(start);<br>+        LocalTime endTime = LocalTime.parse(end);<br>+        Duration duration = Duration.between(startTime, endTime);<br>+        System.out.println(&quot;Time between &quot; + start + &quot; and &quot; + end + &quot; is &quot; + duration.toHours() + &quot; hours and &quot; + duration.toMinutesPart() + &quot; minutes.&quot;);<br>     }<br> }<br>\ No newline at end of file</pre><p>Although we humans can’t easily read the diff, AI or LLMs are very good at understanding the code changes in a git diff.</p><h4>Step 2: Review the code changes</h4><p>Now, we have extracted a diff, and we are ready to review the code with Copilot. Let’s write a prompt to review our code. We will save this prompt in a file called <em>code-review-template.txt</em> (We can keep this prompt template in our project for any future code reviews that we may do)</p><pre>Perform a code review for the code changes given in code-changes.txt file.<br>The code changes in code-changes.txt file are given in the format of a git diff output.<br>Focus only on the code changes given in the code-changes.txt file and ignore any other code.<br><br>The code review should focused on following aspects:<br>programming best practices, code readability, and code maintainability.<br>intended functionality and the performance of the application.<br>code formatting and code styling.<br>code structure and code design.<br>proper variable/method naming conventions.<br>code comments and documentation.<br><br>The code review should include up to 5 pros and up to 5 improvement points of the code changes.<br>If there are no pros, mention that there are no pros.<br>If there are no improvement points, mention that there are no improvement points.<br><br>Use appropriate styling, colors, icons in the code review, so that it is easy to read and understand.<br>Write a small summary on the overall code changes whether it is good or bad and why.<br>At the end of the code review, provide a solution or an alternative code snippet to improve the code changes.<br>Along with the code snippets, mention the file name where the changes should be done.</pre><p><em>The above prompt describes how we expect the code review to be done, and how the review comments should be presented to us.</em></p><p>For the code review, let’s create a small Java program that prints the time difference between 2 timestamps, and we will make some obvious mistakes for testing the code review workflow.</p><pre>package org.example;<br><br>import java.time.Duration;<br>import java.time.LocalTime;<br><br>public class Main {<br>    public static void main(String[] args) {<br>        findTime(&quot;10:00&quot;, &quot;12:00&quot;);<br>        findTime(&quot;12:00&quot;, &quot;14:00&quot;);<br>    }<br><br>    private static void findTime(String start, String end) {<br>        LocalTime x = LocalTime.parse(start);<br>LocalTime y = LocalTime.parse(end);<br>// Calculate the time between<br>        Duration duration = Duration.between(x, y);<br>        System.out.println(&quot;Time between &quot; + start + &quot; and &quot; + end + &quot; is &quot; + duration.toHours() + &quot; hours and &quot; + duration.toMinutesPart() + &quot; minutes.&quot;);<br>    }<br>}</pre><p>Now, we will extract the git diff for our code changes by following Step 1</p><pre>diff --git a/src/main/java/org/example/Main.java b/src/main/java/org/example/Main.java<br>index 63133db..e1d14e0 100644<br>--- a/src/main/java/org/example/Main.java<br>+++ b/src/main/java/org/example/Main.java<br>@@ -1,5 +1,8 @@<br> package org.example;<br> <br>+import java.time.Duration;<br>+import java.time.LocalTime;<br>+<br> public class Main {<br>     public static void main(String[] args) {<br>         findTime(&quot;10:00&quot;, &quot;12:00&quot;);<br>@@ -7,6 +10,10 @@ public class Main {<br>     }<br> <br>     private static void findTime(String start, String end) {<br>-        System.out.println(&quot;Time between &quot; + start + &quot; and &quot; + end + &quot; is&quot;);<br>+        LocalTime x = LocalTime.parse(start);<br>+LocalTime y = LocalTime.parse(end);<br>+// Calculate the time between<br>+        Duration duration = Duration.between(x, y);<br>+        System.out.println(&quot;Time between &quot; + start + &quot; and &quot; + end + &quot; is &quot; + duration.toHours() + &quot; hours and &quot; + duration.toMinutesPart() + &quot; minutes.&quot;);<br>     }<br> }<br>\ No newline at end of file</pre><h4>Step 3: Get suggestions for improvement points</h4><p>Let’s add the the prompt (<em>code-review-template.txt</em>) and the diff (<em>code-changes.txt</em>) we created into the context, and ask Copilot to do a code review.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YP_W4XHyj6T5LS_-N9KZBQ.png" /><figcaption>We will use chat with the code review prompt and the git diff we created</figcaption></figure><p><strong>The Output</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SCn6TftiCLLTOXUD1MMvVg.png" /><figcaption>Copilot has identified some potential issues in our code (also the issues we delebarately planted)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xKd3GF8t9F9QszdNAmZ68g.png" /><figcaption>The final code changes are is given with the suggestions already added</figcaption></figure><h4>Step 4: Iterate through Steps 1–3</h4><p>Now, we will apply the suggested code improvements in our project, and we will follow Steps 1–3 again to see if there are any additional issues we’ve missed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*NaTJDtbPxpKIRUNpnISpvA.png" /><figcaption>Code review after few iterations</figcaption></figure><p>After a few iterations, we can see code is now more optimized and clear.</p><p><strong>Final Code</strong></p><pre>package org.example;<br><br>import java.time.Duration;<br>import java.time.LocalTime;<br>import java.time.format.DateTimeParseException;<br><br>public class Main {<br>    public static void main(String[] args) {<br>        String[][] timePairs = {{&quot;10:00&quot;, &quot;12:00&quot;}, {&quot;12:00&quot;, &quot;14:00&quot;}};<br>        for (String[] pair : timePairs) {<br>            printDurationBetweenTimes(pair[0], pair[1]);<br>        }<br>    }<br><br>    /**<br>     * Prints the duration between two times.<br>     *<br>     * @param startTimeStr the start time in HH:mm format<br>     * @param endTimeStr the end time in HH:mm format<br>     */<br>    private static void printDurationBetweenTimes(String startTimeStr, String endTimeStr) {<br>        if (isNullOrEmpty(startTimeStr) || isNullOrEmpty(endTimeStr)) {<br>            System.err.println(&quot;Start time and end time must not be null or empty.&quot;);<br>            return;<br>        }<br>        try {<br>            LocalTime startTime = LocalTime.parse(startTimeStr);<br>            LocalTime endTime = LocalTime.parse(endTimeStr);<br>            // Calculate the time between<br>            Duration duration = Duration.between(startTime, endTime);<br>            System.out.println(&quot;Time between &quot; + startTimeStr + &quot; and &quot; + endTimeStr + &quot; is &quot; + duration.toHours() + &quot; hours and &quot; + duration.toMinutesPart() + &quot; minutes.&quot;);<br>        } catch (DateTimeParseException e) {<br>            System.err.println(&quot;Invalid time format. Please use HH:mm format. Error: &quot; + e.getMessage());<br>        }<br>    }<br><br>    private static boolean isNullOrEmpty(String str) {<br>        return str == null || str.isEmpty();<br>    }<br>}</pre><h4>Conclusion</h4><p>In this article, we looked at a simple and generic workflow to perform code reviews in the new code we’re about to submit using GitHub Copilot.</p><p>We used git diff to isolate our code changes to focus the code review only on our code changes, ignoring any other code in our repository.</p><p>We created a descriptive prompt and kept it in a file for re-using in our workflow in the future.</p><h4>Additional Remarks</h4><p>This article is a very simplified version of the code review workflow I use regularly before I submit my code as a merge request. You can experiment with different prompting techniques and LLM models to make the code review more efficient. I would personally prefer Claude 3.7 in my code reviews. However, I did all the examples here using the default model GPT-4o since it’s available in the Copilot free version.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1adeec6824d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Multi-File Edits in IntelliJ IDEA with GitHub Copilot: A Productivity Power Move]]></title>
            <link>https://medium.com/@isurutharanga/mastering-multi-file-edits-in-intellij-idea-with-github-copilot-a-productivity-power-move-fc98fd9dd239?source=rss-7abfec95c9d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc98fd9dd239</guid>
            <category><![CDATA[github-copilot]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[productivity-hacks]]></category>
            <category><![CDATA[productivity]]></category>
            <dc:creator><![CDATA[Isuru Perera]]></dc:creator>
            <pubDate>Sun, 26 Jan 2025 08:05:50 GMT</pubDate>
            <atom:updated>2025-03-23T13:56:23.265Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*j9pHUim4edVu9w2eZkgTtg.png" /></figure><p><strong>Edit(23/03/2025):</strong><br>From 20/03/2025 Copilot Edits are finally available in Intellij Copilot Plugin. This article is still relevant in terms of ways we can use LLMs to perform any task not given out of the box through prompt engineering.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g3exvDVhIMGe3ge_YgT7Ug.png" /></figure><h4>TL;DR</h4><p>The following article is about a GitHub Copilot hack I’ve discovered to edit multiple source files in a single go through prompt engineering. If you’ve already used Copilot in your development, you may already know, VSCode is leagues ahead compared to IntelliJ IDEA in terms of integration with Copilot.</p><p>VSCode offers <a href="https://code.visualstudio.com/docs/copilot/overview">Copilot Chat as well as Copilot Edits</a>. However, the IntelliJ IDEA <a href="https://plugins.jetbrains.com/plugin/17718-github-copilot">Copilot plugin</a> doesn’t offer any feature comparable to Copilot Edits which I found very useful in VSCode. However, for those who can’t make the switch to VSCode yet as a daily driver, you can use the following prompt engineering hack to generate a git patch with all the code changes (in multiple files) to get similar functionality as Copilot Edits in VSCode.</p><p>For the power users, here’s the prompt…</p><pre>&lt;Your prompt to do whatever your code changes&gt;<br><strong>Generate git patch for all the code changes, where I can copy the patch to clipboard.<br>Do not output any code in the chat, and include all code changes in the patch itself.</strong></pre><p>Yes, that’s it. As simple as that, we can now generate a patch to edit (or create) multiple files in a single go without having to break a sweat.<br>To avoid reaching output token limits, the above prompt will generate only the patch.</p><h4>The Long Version</h4><p>If you still can’t understand, bear with me, I’ve prepared a full tutorial with examples to get you up to speed.</p><p>Copilot Edits is a feature in VSCode where you can ask Copilot to do some code changes, where it’ll change multiple files, or multiple places in a single file in the current workspace.</p><p>However, in IntelliJ IDEA there’s no such feature in the Copilot plugin (at the time of writing 26/01/2025). So you’re forced to use the chat, and manually create the new files Copilot is suggesting, or manually compare and apply the new changes to the existing files. However, being a daily user of IDEA it was a tough pain-point for me. Initially, I started using VSCode and IDEA side by side and used VSCode purely for Copilot. <em>(I know this is how many of my colleagues use it as well)</em></p><p>A little trip to <a href="https://plugins.jetbrains.com/plugin/17718-github-copilot">IntelliJ IDEA Copilot plugin homepage</a> would prove many other developers felt the same way as I do.</p><p>If you’re from the Java ecosystem, you may already know, IntelliJ IDEA is one of the most powerful IDEs, and VSCode is still lacking some of its powerful features for Java development. So, switching to VSCode or any other IDE is still not an option for me (yet). <strong>Please note that this article is not about which IDE is more powerful or better for development with AI, but about unlocking new things via creative ways of using prompt engineering.</strong></p><p>The following tutorial explains how you can use IntelliJ IDEA, and GitHub Copilot plugin to generate a large set of changes in your project in a single go, possibly improving productivity.</p><p>The entire tutorial is based on the following sample Spring Boot app. The code is available in my GitHub repo.</p><p><a href="https://github.com/isuruperera/spring-boot-example">https://github.com/isuruperera/spring-boot-example</a></p><figure><img alt="For simplicity, let’s imagine we have a REST controller with CRUD operations to perform on an entity called ‘TestEntity’" src="https://cdn-images-1.medium.com/max/674/1*bp27w_Wlr-2huXfc9e-eDg.png" /><figcaption>The project structure</figcaption></figure><p>Now, imagine you need to create a new feature in the above project, where you have to add CRUD operations to a different entity called ‘TestEntityTwo’ which has a different set of fields, and expose it via a REST API. We will assume TestEntityTwo will have id, name, age, phone, and address fields.</p><p>If you need to generate all the code to implement this feature, you could prompt Copilot with something like this. <em>(Note that I’ve added my existing classes into the context, where Copilot will use them as ‘examples’ to generate the new code.)</em></p><blockquote>Generate code to add a new entity called TestEntityTwo, and expose it via a new REST controller. The TestEntityTwo will have only id, name, age, phone, and address fields. Use the given classes to generate the new code changes.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l3FEVIAGKEMmzKrdSeRW_g.png" /><figcaption>Copilot generated response with new code</figcaption></figure><p>Here, we can see Copilot has generated the full code to add TestEntityTwo, along with code to generate many other components, such as DTO, Repository, Controller, etc. However, if you need to add all these changes to your project, you have to manually create the files and copy the generated code. (painful imo)</p><p><strong>Enter Prompt Engineering…</strong></p><p>In IntelliJ IDEA you can use <a href="https://git-scm.com/docs/git-apply">git patches</a> with changes for multiple code blocks, and also creation and deletion of multiple source files. Let’s try to use prompt engineering to generate a patch with all these changes, instead of copying files manually. We will now use an engineered prompt to generate the code in a slightly different way.</p><blockquote>Generate code to add a new entity called TestEntityTwo, and expose it via a new REST controller. The TestEntityTwo will have only id, name, age, phone, and address fields. Use the given classes to generate the new code changes.</blockquote><blockquote><strong>Generate git patch for all the code changes, where I can copy the patch to clipboard.<br>Do not output any code in the chat, and include all code changes in the patch itself.</strong></blockquote><p>The above prompt is an example of <a href="https://www.promptingguide.ai/techniques/prompt_chaining">Prompt Chaning</a> where we can instruct the LLM to use the output of a prompt into a different prompt like in a chain, and give us the final outcome.</p><p><em>Here, the first part will instruct the LLM to generate the code, and the second part will instruct to convert the code changes into a git patch, and third part will instruct to output only the patch excluding , without compromising the output token limit.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-mQArmr3L0LvqyxFE5ZHGw.png" /><figcaption>Copilot generated response with git patch</figcaption></figure><p>Now, the newly generated code looks different. Because it’s generated in the format of git patch. We can easily copy this patch, and apply it to our project, where it’ll do all the updates in a single go.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ponkxEgJ9pfLNMtu22N4lw.png" /><figcaption>You can copy the patch to clipboard</figcaption></figure><p>Go to Git (or VCS) &gt; Patch &gt; Apply patch from Clipboard…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xDPt4Fpsl3F_kYvNSR2xUg.png" /><figcaption>You can apply the patch from clipboard</figcaption></figure><p>Now, IntelliJ IDEA will prompt you to add all the changes generated by Copilot to your current change set (Or a new change set if you need to review the AI generated code)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0jZGlTcf_jAIS6z3oiTIjg.png" /><figcaption>Review and apply the patch</figcaption></figure><p>That’s it. Now you know how to effectively use GitHub Copilot to generate and apply a large amount of changes in one go, using IntelliJ IDEA. Hope it’ll be useful. Happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc98fd9dd239" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to accelerate your RESTful microservices with Quarkus]]></title>
            <link>https://medium.com/@isurutharanga/how-to-accelerate-your-restful-microservices-with-quarkus-feec11e68a2?source=rss-7abfec95c9d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/feec11e68a2</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[quarkus]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Isuru Perera]]></dc:creator>
            <pubDate>Sun, 28 May 2023 09:14:05 GMT</pubDate>
            <atom:updated>2023-05-28T09:14:05.971Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*_aMmDtPvoWs9ouaAW3l9Pw.png" /></figure><p>Do you love Java? Do you want to create fast and fun web applications that make your users happy? Do you want to quickly develop your web apps? Do you want your web apps to be bloat-free from the abundance of unused features? If you answered yes to any of these questions, then you need Quarkus in your life. Quarkus is a framework that aims to make Java development fast, fun, and productive. It combines the best-of-breed libraries and standards with innovative features and optimizations to create a <strong>supersonic subatomic</strong> Java platform.</p><p>We live in a world full of microservices, some of the challenging parts of microservices development are, prototyping applications extremely fast, supporting quick boot times for making them more elastic, and using as less code as possible.</p><p>In this article, I will show you how to get started with Quarkus and create a simple RESTful web service that returns a random dog fact. You will learn how to use Quarkus extensions, profiles, live reload, native compilation, and more. By the end of this article, you will be able to Quarkus like a pro.</p><h3>Prerequisites</h3><p>To follow along with this tutorial, you will need the following:</p><ul><li>Java 17 or higher</li><li>Maven 3.9.2 or higher</li><li>An IDE of your choice</li></ul><p>If you want to explore the code before everything below, head over to this GitHub <a href="https://github.com/isuruperera/quarkus-rest-api.git">repo</a>.</p><h3>Creating the project</h3><p>The easiest way to create a Quarkus project is to use the Maven plugin. Open a terminal and run the following command:</p><pre>mvn io.quarkus:quarkus-maven-plugin:3.1.0.Final:create \<br>    -DprojectGroupId=com.example \<br>    -DprojectArtifactId=quarkus-dogs \<br>    -DclassName=&quot;com.example.DogFactResource&quot; \<br>    -Dpath=&quot;/dogfacts&quot; \<br>    -Dextensions=&quot;resteasy-jackson&quot;</pre><p>This command will generate a project with the following structure:</p><pre>quarkus-dogs<br>├── src<br>│   ├── main<br>│   │   ├── docker<br>│   │   ├── java<br>│   │   │   └── com<br>│   │   │       └── example<br>│   │   │           └── DogFactResource.java<br>│   │   └── resources<br>│   │       ├── application.properties<br>│   │       └── META-INF<br>│   │           └── resources<br>│   │               └── index.html<br>│   └── test<br>│       ├── java<br>│       │   └── com<br>│       │       └── example<br>│       │           └── DogFactResourceTest.java<br>│       └── resources<br>│           └── application.properties<br>├── .dockerignore<br>├── .gitignore<br>├── mvnw<br>├── mvnw.cmd<br>├── pom.xml<br>└── README.md</pre><p>The most important files are:</p><ul><li><strong>pom.xml</strong>: The Maven build file that defines the project dependencies and plugins.</li><li><strong>application.properties</strong>: The configuration file that holds the application settings and properties.</li><li><strong>DogFactResource.java</strong>: The Java class that exposes the REST endpoint for getting dog facts.</li><li><strong>DogFactResourceTest.java</strong>: The Java class that tests the REST endpoint using Quarkus Test.</li></ul><h3>Writing the code</h3><p>Let’s take a look at the DogFactResource.java file. It should look something like this:</p><pre>package com.example;<br><br>import jakarta.ws.rs.GET;<br>import jakarta.ws.rs.Path;<br>import jakarta.ws.rs.Produces;<br>import jakarta.ws.rs.core.MediaType;<br>@Path(&quot;/dogfacts&quot;)<br>public class DogFactResource {<br>@GET<br>    @Produces(MediaType.TEXT_PLAIN)<br>    public String hello() {<br>        return &quot;Hello RESTEasy&quot;;<br>    }<br>}</pre><p>This is a very simple class that defines a GET method on the /dogfacts path that returns a plain text response. Nothing too exciting here, right? Let’s spice it up by adding some dog facts.</p><p>To get some random dog facts, we will use an external API called Dog API (https://dogapi.dog). This API allows us to fetch fun facts about dogs from various categories and formats. We will use the JSON format for simplicity.</p><p>To call the API from our Quarkus application, we will use the RestClient extension. This extension provides an easy way to create REST clients using MicroProfile Rest Client specification.</p><p>To add the RestClient extension to our project, we can run the following command:</p><pre>./mvnw quarkus:add-extension -Dextensions=&quot;rest-client&quot;</pre><p>This will update our pom.xml file with the necessary dependencies.</p><p>Next, we need to create an interface that defines the methods and parameters of the Dog API. We can call it DogService.java and place it in the same package as our DogFactResource.java file. It should look something like this:</p><pre>package com.example;<br><br>import com.example.beans.DogFact;<br>import jakarta.ws.rs.GET;<br>import jakarta.ws.rs.Path;<br>import jakarta.ws.rs.QueryParam;<br>import org.eclipse.microprofile.rest.client.inject.RegisterRestClient;<br><br>@Path(&quot;/api/v2&quot;)<br>@RegisterRestClient(configKey = &quot;dog-api&quot;)<br>public interface DogService {<br>    @GET<br>    @Path(&quot;/facts&quot;)<br>    DogFact getDogFact(@QueryParam(&quot;limit&quot;) int number);<br>}</pre><p>This interface defines a GET method on the /api/v2/facts the path that takes a query parameter: limit. It also returns an DogFact object that represents the JSON response from the API. We also annotate the interface @RegisterRestClient to register it as a REST client and specify a configuration key for it.</p><p>We also need to create a DogFact class that maps to the JSON response from the API. We can use Jackson annotations to customize the mapping. The class should look something like this:</p><pre>package com.example.beans;<br><br>import io.vertx.codegen.annotations.DataObject;<br><br>import java.util.List;<br><br>@DataObject<br>public class DogFact {<br>    private List&lt;DogData&gt; data;<br><br>    // Getters and setters omitted for brevity<br>    // A custom method to return a random fact from the array<br>    public String getRandomFact() {<br>        if (getData() != null &amp;&amp; getData().size() &gt; 0) {<br>            // Get a random index from 0 to facts.length - 1<br>            int index = (int) (Math.random() * getData().size());<br>            // Return the fact at that index<br>            return data.get(index).getAttributes().getBody();<br>        } else {<br>            // Return a default fact if something goes wrong<br>            return &quot;Dogs are awesome.&quot;;<br>        }<br>    }<br><br>    public List&lt;DogData&gt; getData() {<br>        return data;<br>    }<br><br>    public void setData(List&lt;DogData&gt; data) {<br>        this.data = data;<br>    }<br>}</pre><p>This class defines fields that correspond to the JSON properties from the API response. We also define a custom method getRandomFact() that returns a random fact from the array of facts, or a default fact if something goes wrong.</p><p>Now that we have defined our REST client and our model class, we can inject it into our DogFactResource class using @Inject annotation and use it to fetch dog facts from the API. We can also add some error handling in case something goes wrong. The updated DogFactResource class should look something like this:</p><pre>package com.example;<br><br>import com.example.beans.DogFact;<br>import jakarta.inject.Inject;<br>import jakarta.ws.rs.GET;<br>import jakarta.ws.rs.Path;<br>import jakarta.ws.rs.Produces;<br>import jakarta.ws.rs.WebApplicationException;<br>import jakarta.ws.rs.core.MediaType;<br>import jakarta.ws.rs.core.Response;<br>import org.eclipse.microprofile.rest.client.inject.RestClient;<br><br>@Path(&quot;/dogfacts&quot;)<br>public class DogFactResource {<br><br>    @Inject<br>    @RestClient<br>    DogService dogService;<br>    @GET<br>    @Produces(MediaType.TEXT_PLAIN)<br>    public String getDogFact() throws WebApplicationException {<br>        try {<br>            // Call the dog API with number 1 and Any category<br>            DogFact dogFact = dogService.getDogFact(10);<br>            // Return the random fact as a plain text response<br>            return dogFact.getRandomFact();<br>        } catch (Exception e) {<br>            // Throw an internal server error if something goes wrong<br>            throw new WebApplicationException(e.getMessage(), Response.Status.INTERNAL_SERVER_ERROR);<br>        }<br>    }<br>}</pre><p>We inject our DogService interface using the @RestClient annotation and call its getDogFact() method with the desired parameters. We then return the random fact as a plain text response using our custom method getRandomFact(). We also wrap our code in a try-catch block and throw an internal server error if something goes wrong.</p><p>One last thing we need to do is to configure our REST client with the base URL of the Dog API. We can do that by adding the following line to our application.properties file:</p><pre>dog-api/mp-rest/url=https://dogapi.dog</pre><p>This tells Quarkus to use this URL as the base URL for our REST client that has the configuration key dog-api.</p><h3>Running and testing the application</h3><p>Now that we have written our code, we can run and test our application using Quarkus Dev Mode. This mode allows us to run our application in development mode with live reload and debugging capabilities.</p><p>To start Dev Mode, we can run the following command:</p><pre>./mvnw quarkus:dev</pre><p>This will start our application on port 8080 and watch for any changes in our source files. If we make any changes, Quarkus will automatically recompile and reload our application.</p><p>We can test our application by opening a browser and navigating to <a href="http://localhost:8080/dogfacts.">http://localhost:8080/dogfacts.</a> We should see a random dog fact displayed on our screen. If we refresh the page, we should see another fact.</p><p>We can also test our application using curl or any other HTTP client tool. For example:</p><pre>curl http://localhost:8080/dogfacts</pre><p>We should see a random dog fact printed in the terminal.</p><p>We can also run our unit tests using the following command:</p><pre>./mvnw test</pre><p>This will run our DogFactResourceTest class that uses Quarkus Test to verify that our REST endpoint returns a 200 OK response with a non-empty body.</p><h3>Building and running the native executable</h3><p>One of the coolest features of Quarkus is its ability to compile Java applications into native executables using GraalVM. This allows us to reduce the memory footprint and startup time of our applications, as well as improve their performance and security.</p><p>To build a native executable of our application, we need to have GraalVM and its native-image tool installed on our system. You can find the installation instructions here: <a href="https://quarkus.io/guides/building-native-image#prerequisites-for-oracle-graalvm-ceee">https://quarkus.io/guides/building-native-image#prerequisites-for-oracle-graalvm-ceee</a></p><p>Once we have GraalVM and native-image installed, we can run the following command:</p><pre>./mvnw package -Pnative</pre><p>This will create a native executable file named quarkus-rest-api-1.0.0-SNAPSHOT-runner in the target directory.</p><p>To run the native executable, we can use the following command:</p><pre>./target/quarkus-rest-api-1.0.0-SNAPSHOT-runner</pre><p>This will start our application on port 8080, just like before. However, this time, we should notice that the startup time is much faster and the memory usage is much lower.</p><p>We can test our application using a browser or curl, just like before. We should see the same results as before.</p><h3>Conclusion</h3><p>In this short tutorial, we went through the implementation of a simple RESTful API using Quarkus framework and saw how powerful Quarkus extensions can be when it comes to fast-paced development. We integrated a simple API with our application and presented its data to the user. Finally, we went through how to test our Quarkus application and went through the GraalVM native build process, which makes our application start up extremely quickly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=feec11e68a2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Migrating your middleware layer to the cloud for enhanced scalability and reduced cost — (Part 2)]]></title>
            <link>https://medium.com/@isurutharanga/migrating-your-middleware-layer-to-the-cloud-for-enhanced-scalability-and-reduced-cost-part-2-9995c062bbb9?source=rss-7abfec95c9d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/9995c062bbb9</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[apache-camel]]></category>
            <category><![CDATA[quarkus]]></category>
            <category><![CDATA[enterprise-integration]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Isuru Perera]]></dc:creator>
            <pubDate>Thu, 27 Apr 2023 07:10:53 GMT</pubDate>
            <atom:updated>2023-04-27T07:28:13.400Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ahB5uN8I7Gi34AfkNYLfA.png" /></figure><p>In <a href="https://medium.com/@isurutharanga/migrating-your-middleware-layer-to-the-cloud-for-enhanced-scalability-and-reduced-cost-part-1-f0cd47fd4b05">Part 1</a> of this post, I walked you through a simplified version of what is “Enterprise Application Integration” and talked about using Apache Camel and Quarkus to deploy your integrations in the cloud as Lambda Functions. In this post, let’s talk about how to implement a super simple integration using Camel Quarkus.</p><p>Here, I’m assuming you’re familiar with Java 11. If you need to go through the code before reading, head over to the <a href="https://github.com/isuruperera/super-simple-camel-quarkus-app">GitHub repo</a> which contains the complete code.</p><h4>Dependencies</h4><p>First, we need to import all the dependencies needed for Camel Quarkus. In the example, I’m using Camel Quarkus BOM which makes dependency version management a little easier.</p><pre>&lt;dependencyManagement&gt;<br>    &lt;dependencies&gt;<br>        &lt;dependency&gt;<br>            &lt;groupId&gt;${quarkus.platform.group-id}&lt;/groupId&gt;<br>            &lt;artifactId&gt;${quarkus.platform.artifact-id}&lt;/artifactId&gt;<br>            &lt;version&gt;${quarkus.platform.version}&lt;/version&gt;<br>            &lt;type&gt;pom&lt;/type&gt;<br>            &lt;scope&gt;import&lt;/scope&gt;<br>        &lt;/dependency&gt;<br>        &lt;dependency&gt;<br>            &lt;groupId&gt;${camel-quarkus.platform.group-id}&lt;/groupId&gt;<br>            &lt;artifactId&gt;${camel-quarkus.platform.artifact-id}&lt;/artifactId&gt;<br>            &lt;version&gt;${camel-quarkus.platform.version}&lt;/version&gt;<br>            &lt;type&gt;pom&lt;/type&gt;<br>            &lt;scope&gt;import&lt;/scope&gt;<br>        &lt;/dependency&gt;<br>    &lt;/dependencies&gt;<br>&lt;/dependencyManagement&gt;</pre><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;io.quarkus&lt;/groupId&gt;<br>    &lt;artifactId&gt;quarkus-amazon-lambda&lt;/artifactId&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;io.quarkus&lt;/groupId&gt;<br>    &lt;artifactId&gt;quarkus-arc&lt;/artifactId&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;com.amazonaws&lt;/groupId&gt;<br>    &lt;artifactId&gt;aws-lambda-java-events&lt;/artifactId&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;org.apache.camel.quarkus&lt;/groupId&gt;<br>    &lt;artifactId&gt;camel-quarkus-core&lt;/artifactId&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;org.apache.camel&lt;/groupId&gt;<br>    &lt;artifactId&gt;camel-componentdsl&lt;/artifactId&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;org.apache.camel.quarkus&lt;/groupId&gt;<br>    &lt;artifactId&gt;camel-quarkus-direct&lt;/artifactId&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;org.apache.camel.quarkus&lt;/groupId&gt;<br>    &lt;artifactId&gt;camel-quarkus-http&lt;/artifactId&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;org.apache.camel.quarkus&lt;/groupId&gt;<br>    &lt;artifactId&gt;camel-quarkus-log&lt;/artifactId&gt;<br>&lt;/dependency&gt;</pre><h4>Handler Function</h4><p>In a lambda function, the Handler function is the starting point of execution for the application. A simple handler function with Camel Quarkus would look like the one below.</p><pre>@Named(&quot;exampleLambdaHandler&quot;)<br>public class Handler implements RequestHandler&lt;Map&lt;String, Object&gt;, Map&lt;String, Object&gt;&gt; {<br><br>    @Inject<br>    ProducerTemplate producerTemplate;<br><br>    @Override<br>    public Map&lt;String, Object&gt; handleRequest(Map&lt;String, Object&gt; stringObjectMap, Context context) {<br><br>        String response = DefaultFluentProducerTemplate.on(producerTemplate.getCamelContext())<br>                .to(&quot;direct:incomingRoute&quot;)<br>                .request(String.class);<br><br>        return Map.of(<br>                &quot;statusCode&quot;, 200,<br>                &quot;statusDescription&quot;, &quot;200 OK&quot;,<br>                &quot;isBase64Encoded&quot;, false,<br>                &quot;headers&quot;, Map.of(&quot;Content-Type&quot;, &quot;application/json; charset=utf-8&quot;),<br>                &quot;body&quot;, Map.of(&quot;message&quot;, response)<br>        );<br>    }<br>}</pre><p>Here, We’re implementing the <a href="https://docs.aws.amazon.com/lambda/latest/dg/java-handler.html">RequestHandler&lt;I,O&gt;</a> interface from AWS Java SDK, which acts as the main starting point for our business logic. Inside the <em>handleRequest </em>method, we’re using a Camel producer template, which sends the event received by the Lambda function over to the <em>direct:incomingRoute </em>Camel Route. Finally, we use the response from the Camel Route to create the response for the Lambda function.</p><p><em>Notice that all the business logic in the above Lambda is handled by a Camel Route, which decouples the AWS Lambda handler implementation and Camel-based implementation. If you’re migrating your existing Camel integrations to the cloud, this would be a pretty neat starting point.</em></p><p><em>Notice that the Quarkus framework supports Dependency Injection out of the box. In the above code, we use Quarkus DI to inject the producer template into the Lambda handler. It’s pretty cool that we can inject our own app dependencies in Quarkus applications, but </em><a href="https://quarkus.io/guides/cdi-reference"><em>more details on Quarkus DI</em></a><em> are out of the scope of this post.</em></p><h4>Camel Integrations</h4><p>If you’re familiar with Apache Camel, you might already know, in Camel every integration is abstracted into camel routes. Here, in our example application, we’re using a camel route to send an HTTP get request to <a href="https://www.google.com/">https://www.google.com</a> and return the first 10 characters of the response, and the count of remaining characters.</p><pre>public class ExampleCamelRoute extends RouteBuilder {<br>    @Override<br>    public void configure() throws Exception {<br>        from(&quot;direct:incomingRoute&quot;)<br>                .log(&quot;Incoming request ${body}&quot;)<br>                .to(&quot;https://www.google.com&quot;)<br>                .convertBodyTo(String.class)<br>                .process(exchange -&gt; {<br>                    String response = exchange.getIn().getBody(String.class);<br>                    exchange.getIn().setBody(response.substring(0, 10) + &quot;...(&quot; + (response.length() - 10) + &quot; characters more)&quot;);<br>                }).end();<br>    }<br>}</pre><p>The above example is a very basic Camel Route, which doesn’t do anything useful. But, we’re using it here as a very simple example to prove that we can run a camel integration on a Lambda function.</p><h4>Building the Application</h4><p>Since we went through the code for our super simple Camel Quarkus application, now let’s see how we can run this application as a Lambda function.</p><pre>&lt;plugin&gt;<br>    &lt;groupId&gt;${quarkus.platform.group-id}&lt;/groupId&gt;<br>    &lt;artifactId&gt;quarkus-maven-plugin&lt;/artifactId&gt;<br>    &lt;version&gt;${quarkus.platform.version}&lt;/version&gt;<br>    &lt;extensions&gt;true&lt;/extensions&gt;<br>    &lt;executions&gt;<br>        &lt;execution&gt;<br>            &lt;goals&gt;<br>                &lt;goal&gt;build&lt;/goal&gt;<br>                &lt;goal&gt;generate-code&lt;/goal&gt;<br>                &lt;goal&gt;generate-code-tests&lt;/goal&gt;<br>            &lt;/goals&gt;<br>        &lt;/execution&gt;<br>    &lt;/executions&gt;<br>&lt;/plugin&gt;</pre><p>In our pom.xml we’re using the <em>quarkus-maven-plugin </em>to build our application as a lambda function. When you build the application using <em>mvn package </em>command, the lambda function archive will be automatically created in the <em>target </em>directory with file name <em>function.zip </em>upon successful build of the application.</p><pre>&lt;profile&gt;<br>    &lt;id&gt;native&lt;/id&gt;<br>    &lt;activation&gt;<br>        &lt;property&gt;<br>            &lt;name&gt;native&lt;/name&gt;<br>        &lt;/property&gt;<br>    &lt;/activation&gt;<br>    &lt;properties&gt;<br>        &lt;quarkus.package.type&gt;native&lt;/quarkus.package.type&gt;<br>        &lt;quarkus.native.container-build&gt;true&lt;/quarkus.native.container-build&gt;<br>    &lt;/properties&gt;<br>&lt;/profile&gt;</pre><p>In order to gain the performance benefits of GraalVM, we can build our application as a Native image using the <em>quarkus-maven-plugin. </em>For activating the native build, we need to pass property <em>quarkus.package.type </em>as <em>native. </em>Use of only <em>quarkus.package.type </em>property will assume your system is capable of building GraalVM native images. However, if your system is not configured for building native images, you can enable Quarkus native container build using the <em>quarkus.native.container-build maven </em>property, which will automatically download the official Quarkus native build image and perform the native image build for you inside a Docker container.</p><p>For ease of use, we can add both <em>quarkus.package.type </em>and <em>quarkus.native.container-build </em>maven properties<em> </em>into a maven build profile and activate the profile only when we need to use <em>mvn package -Pnative</em> command.</p><p><em>Note that the Native build takes a significantly more amount of memory and CPU, and it might take a significantly long time to finish the build compared to the default JVM build.</em></p><h4>Deploying the Lambda Function</h4><p>Now we have built our Camel integration as a Lambda function, and are ready to deploy the function archive in the AWS management console. Let’s head over to the AWS Management console and create a new Lambda function.</p><p>Use the following basic information for the function.</p><ol><li><strong>Function Name:</strong> MySuperSimpleCamelQuarkusApp (Or any name you like!)</li><li><strong>Runtime:</strong> Java 11 (Corretto)</li><li><strong>Architecture:</strong> x86_64</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VtG8XMl7Qc3wIRzWxz1FLw.png" /><figcaption>Create Lambda Function</figcaption></figure><p>After the Lambda Function is created, Upload the <strong>function.zip </strong>file as the code source.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GmVt2TH3f1WyWqNgMBAoFg.png" /><figcaption>Upload Code Source</figcaption></figure><p>Edit your runtime settings to add the correct Runtime and Handler function name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZbtjkcCUuPZqaeHR0P1tmA.png" /><figcaption>Edit Runtime Settings</figcaption></figure><h4>Lambda Configuration for JVM Mode</h4><p>If you built the example project in default (JVM) mode, you should update the runtime settings with the following parameters.</p><ol><li><strong>Runtime: </strong>Java 11 (Corretto)</li><li><strong>Handler:</strong> io.quarkus.amazon.lambda.runtime.QuarkusStreamHandler::handleRequest</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*PLmAHS6Ikq3fs2wedQ_QRg.png" /><figcaption>Runtime settings for JVM mode</figcaption></figure><p><em>Note that we’ve used </em><a href="https://github.com/quarkusio/quarkus/blob/main/extensions/amazon-lambda/runtime/src/main/java/io/quarkus/amazon/lambda/runtime/QuarkusStreamHandler.java"><em>QuarkusStreamHandler</em></a><em> as the Handler for our Lambda function, instead of the Handler implemented inside our project’s Handler class. We need to use the Handler function from QuarkusStreamHandler because Quarkus will set up everything automatically including Camel context and Dependency Injections under the hood and finally hand over the request to our Handler class automatically. This way, we can focus only on our business logic inside our project’s Handler class, and Quarkus will do everything else under the hood for keeping our code clean and relevant.</em></p><h4>Lambda Configuration for Native Mode</h4><p>If you built the project in Native mode by passing <em>-P native </em>flag in maven build, you should update the runtime settings with the following parameters in order to set up the Lambda function to evaluate the native image.</p><ol><li><strong>Runtime:</strong> Custom runtime on Amazon Linux 2</li><li><strong>Handler:</strong> not.used.in.provided.runtime</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*9aUWibrudKFS3TATPRRF4w.png" /></figure><h4>Camel Quarkus JVM Mode vs Native Mode</h4><p>Even with a quick glance at the Lambda execution statistics reports for JVM vs Native mode, you may notice that the Native mode uses significantly less memory, and the function init and billed durations are significantly low without any changes to our code. With native images, we can use significantly low resources to reduce the infrastructure cost for our integrations. <em>(In the below example stats, native image is 5 times faster and uses 46% less memory compared to JVM mode, which results in a significant cost reduction to perform the same business logic)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vQ8821npBIhOjWijcddRjw.png" /><figcaption>Function execution stats for JVM mode Lambda</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vihyzxgGWgGis4rP1F7mSw.png" /><figcaption>Function execution stats for Native mode Lambda</figcaption></figure><p><em>Note that the above comparisons are for quick reference. For a proper comparison between Camel Quarkus JVM vs Native mode execution stats, functions should be invoked multiple times and average figures should be calculated. If you’re interested in a more formal comparison, head over to </em><a href="https://camel.apache.org/blog/2021/08/DeployingCamelRouteInAWSLambda/"><em>this blog post</em></a><em>.</em></p><h4>Conclusion</h4><p>In this two-part post, we went through some interesting topics in Enterprise Application Integration where we discussed traditional Enterprise Integrations and how to leverage cloud technologies to implement modern, highly scalable, low-cost Enterprise Application Integrations with low management overhead.</p><p>We went through the implementation of a very simple proof of concept modern Enterprise Application Integration using AWS Lambda and Camel Quarkus.</p><p>We discussed how to optimize the application without any changes to the application code using GraalVM native images and significantly improve the performance and reduce cost.</p><p>I hope you’ve gained some valuable insights into modernizing Enterprise Integrations using low-cost cloud technologies. If you have any questions or suggestions, please mention them in the comments below.</p><h4>References</h4><ol><li><a href="https://camel.apache.org/camel-quarkus/3.0.x/index.html">https://camel.apache.org/camel-quarkus/3.0.x/index.html</a></li><li><a href="https://camel.apache.org/docs/">https://camel.apache.org/docs/</a></li><li><a href="https://quarkus.io/guides/">https://quarkus.io/guides/</a></li><li><a href="https://en.wikipedia.org/wiki/Enterprise_application_integration">https://en.wikipedia.org/wiki/Enterprise_application_integration</a></li><li><a href="https://www.softwarereviews.com/categories/enterprise-service-bus">https://www.softwarereviews.com/categories/enterprise-service-bus</a></li><li><a href="https://www.graalvm.org/latest/docs/">https://www.graalvm.org/latest/docs/</a></li><li><a href="https://quarkus.io/guides/cdi-reference">https://quarkus.io/guides/cdi-reference</a></li><li><a href="https://camel.apache.org/blog/2021/08/DeployingCamelRouteInAWSLambda/">https://camel.apache.org/blog/2021/08/DeployingCamelRouteInAWSLambda/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9995c062bbb9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Migrating your middleware layer to the cloud for enhanced scalability and reduced cost — (Part 1)]]></title>
            <link>https://medium.com/@isurutharanga/migrating-your-middleware-layer-to-the-cloud-for-enhanced-scalability-and-reduced-cost-part-1-f0cd47fd4b05?source=rss-7abfec95c9d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/f0cd47fd4b05</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[enterprise-integration]]></category>
            <category><![CDATA[apache-camel]]></category>
            <category><![CDATA[quarkus]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Isuru Perera]]></dc:creator>
            <pubDate>Thu, 27 Apr 2023 07:04:13 GMT</pubDate>
            <atom:updated>2023-04-27T07:26:39.733Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ahB5uN8I7Gi34AfkNYLfA.png" /></figure><p>What comes to your mind when you read “Enterprise Application Integration”? If you imagined something like below, this post is for you. If you imagined something else or didn’t imagine anything at all, or just interested in how the heck you’re gonna cut down your costs and improve the performance of your enterprise integrations, this post is definitely for you…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/586/1*_lHDt54RNrM767SO2Yh_9w.png" /><figcaption>A typical Enterprise Application Integration</figcaption></figure><h4>Enterprise Application Integration</h4><p>Before moving forward with this post, let’s see what is “Enterprise Application Integration”</p><blockquote>Enterprise application integration is an integration framework composed of a collection of technologies and services which form a <a href="https://en.wikipedia.org/wiki/Middleware_(distributed_applications)">middleware</a> or “middleware framework” to enable integration of systems and applications across an enterprise.</blockquote><p>According to <a href="https://en.wikipedia.org/wiki/Enterprise_application_integration">this</a> Wikipedia article, Enterprise Application Integration is simply a way of allowing communication between two different applications possibly with different communication protocols using a middleware (a translator) between them.</p><p>Using middleware between applications or services to do the translation of communication protocols is a great way to decouple applications while focusing only on the business logic in the application and offloading any communication-related logic to the middleware. Any future changes to the communication protocols or any future changes to third-party API interfaces can be handled inside the middleware layer, without doing any major changes to your application code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/444/1*qaiOEPppuof1vc5eHEJm9A.png" /><figcaption>Using middleware to perform communication protocol transformations</figcaption></figure><p>The use of middleware to allow communication between two applications is a time-tested design pattern we used for decades. There are many out-of-the-box solutions out there with a plethora of features to choose from. If interested, go check out this list of <a href="https://www.softwarereviews.com/categories/enterprise-service-bus">Best Enterprise Service Bus (ESB) Software on SoftwareReviews.</a></p><h4>A lightweight approach</h4><p>Using an ESB to centrally manage all API integrations in your enterprise is great, but there are some drawbacks to that approach. Here, I’m only going to address a few major drawbacks in the scope of this post.</p><ol><li>Scalability (How many ESB instances should be running to keep up with your workload? 1 instance? 5 instances? or thousands of Instances? and how fast can your ESB scale from a few instances to thousands of instances to handle the demand?)</li><li>Cost (How much are you paying for keeping your middleware running? Are you paying for the underlying infrastructure platform which keeps your middleware running even when it’s not used, or lightly used?)</li><li>Infrastructure Management Overhead (How are you going to provision underlying infrastructure to run the middleware? Are you going to install and configure servers to run your ESB? What about future OS patches and security updates after setting up the servers to run your ESB?)</li></ol><p>Scalability and Cost are two important characteristics of modern software architectures because businesses thrive by serving as many customers as possible while using as less resources as possible to keep the cost down. More infrastructure management overhead means more costs for the organization.</p><p>So, for the middleware in between our applications, what if there was a way to simply switch off the middleware when not in use, and quickly spin up thousands of instances in a matter of milliseconds? What if there was a way to implement the integration logic and forget about managing the servers throughout the lifetime of the middleware while saving the infrastructure management cost which can be used for important business activities?</p><p>If you’re a Cloud enthusiast like me, you might already have thought about AWS Lambda while reading the above statement. If not, AWS Lambda is a serverless, event-driven compute service which can scale instantly and you only pay for what you use. You can write any code in the form of a function with any supported programming languages including Java, Python, C#, JavaScript(NodeJS), Go, or Ruby, and run that function on demand while paying only for the execution time of the function and the amount of memory used. You can spin up a large number of Lambda functions concurrently within a short amount of time.</p><p>Due to the server-less nature of AWS Lambda, you do not need to manage any sort of servers to run your code. Any underlying infrastructure management is automatically done for you by AWS.</p><blockquote><a href="https://aws.amazon.com/lambda/">AWS Lambda</a> is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. You can trigger Lambda from over 200 AWS services and software as a service (SaaS) applications, and only pay for what you use.</blockquote><p>What if there was an easy way to write your integrations in the form of a Lambda function and redeem all the Performance and Cost benefits it offers while forgetting entirely about infrastructure management?</p><h4>Enter Camel Quarkus</h4><p><a href="https://camel.apache.org/">Apache Camel</a> is Java based open source lightweight enterprise integration (middleware) framework that supports a large number of Enterprise Integration Patterns(EIP) and it <a href="https://camel.apache.org/components/next/">has over 300 stable plug-and-play components</a> to perform any kind of Enterprise Application Integrations.</p><p><a href="https://quarkus.io/">Quarkus is a Supersonic/Subatomic Java Framework</a> for developing cloud-native, highly scalable Java applications with faster boot times. Quarkus supports Java native images with <a href="https://www.graalvm.org/">GraalVM</a> with ultra-low memory footprint and millisecond scale boot times.</p><p><a href="https://camel.apache.org/camel-quarkus/3.0.x/index.html">Camel Quarkus</a> Extensions is an integration layer between Apache Camel and Quarkus to expand the capabilities of both frameworks while benefitting from Camel’s awesome integration capabilities and cloud-native, fast startup and native capabilities of Quarkus.</p><p>So, wouldn’t it be cool if you can write your integration layer or middleware using Apache Camel, and run the integration as a Lambda function using Camel Quarkus to scale out extremely fast while paying only for the usage without any infrastructure management?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*bqDvmKw_Zyaf0cB7fH_8PA.png" /><figcaption>An example Enterprise Application Integration architecture using AWS Lambda</figcaption></figure><p>Still interested and want to learn more? Head over to <a href="https://medium.com/@isurutharanga/migrating-your-middleware-layer-to-the-cloud-for-enhanced-scalability-and-reduced-cost-part-2-9995c062bbb9">Part 2</a> of this post to find out how to implement an example Enterprise Integration using Camel Quarkus and deploy it in AWS Lambda.</p><h4>References</h4><ol><li><a href="https://camel.apache.org/camel-quarkus/3.0.x/index.html">https://camel.apache.org/camel-quarkus/3.0.x/index.html</a></li><li><a href="https://camel.apache.org/docs/">https://camel.apache.org/docs/</a></li><li><a href="https://quarkus.io/guides/">https://quarkus.io/guides/</a></li><li><a href="https://en.wikipedia.org/wiki/Enterprise_application_integration">https://en.wikipedia.org/wiki/Enterprise_application_integration</a></li><li><a href="https://www.softwarereviews.com/categories/enterprise-service-bus">https://www.softwarereviews.com/categories/enterprise-service-bus</a></li><li><a href="https://www.graalvm.org/latest/docs/">https://www.graalvm.org/latest/docs/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f0cd47fd4b05" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Yet another Blockchain story about Supply Chain Traceability]]></title>
            <link>https://medium.com/@isurutharanga/yet-another-blockchain-story-about-supply-chain-traceability-60f7cc72f9b0?source=rss-7abfec95c9d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/60f7cc72f9b0</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[supply-chain]]></category>
            <dc:creator><![CDATA[Isuru Perera]]></dc:creator>
            <pubDate>Tue, 05 Jun 2018 04:08:33 GMT</pubDate>
            <atom:updated>2018-06-05T04:08:33.299Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/372/1*FJlPHccCHD6DTLp-ZUNsEA.png" /></figure><blockquote>Image courtesy: https://goo.gl/images/LLX659</blockquote><h3>Yet another Blockchain story about Supply Chain Traceability</h3><p>The word “Blockchain” has recently become a hype word around the tech community to the point where it is applied for virtually everything, including the places where it shouldn’t actually be used. May be it is because of the massive hype built around the success of crypto currencies including Bitcoin. But, this story does not include crypto currencies, yet you’ll indulge yourself in all the goodness Blockchain has to provide.</p><h4>Blockchain, How?</h4><p>First things first, what is blockchain? And why it is that important? Well, blockchain is an offspring of long existed hashing mechanisms, cryptographic algorithms and distributed systems. Simply put, it is a data structure which can be used to store immutable sets of records, or transactions consistently in a distributed network of computers. Since a picture is worth of thousand words and I’m too lazy to type in thousand more words here, let’s look at a typical diagram and try to understand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QB60hURnqiCVcAIAQ6wAIA.png" /></figure><blockquote>Image courtesy: https://goo.gl/images/LVzf91</blockquote><p>First, let’s understand the concept of hashing. Say there’s a function, and if you input a specific data item into that function, it will always return a string value, which is a representative of the data item (which is of course called the hash of the data item) you entered to the function. So, this way you can represent a set of data items using a unique hash. Well, it is statistically proven some hashing algorithms always return a unique hash value for a given input.</p><p>As now we know what is hashing, let’s look at the diagram. Let’s say the box(block) labeled &#39;1&#39; is the record we generate at first in our data structure. So, when we hash the data in the 1st block, it will return the unique hash, ‘1Z8F’. Since this block is the first in our data structure, we call this by the fancy term ‘genesis block’. Now, it’s time to put some useful data into our data structure. We can bundle some data such as cash transactions, or asset ownership details together and call the whole thing block ‘2’. But, we are going to do a nifty trick here, because we get the hash of the block ‘1&#39; and add to block &#39;2&#39; before we generate the hash of block &#39;2’. Because of this, the hash of block &#39;2&#39; will have some data about block &#39;1’. In this approach, we can generate the hash of block &#39;3&#39; including the hash of block &#39;2’. As you can notice here, each of the blocks will have a ‘dependency’ to the previous block. In simple terms, all the blocks of data in our data structure are ‘chained’ with the previous block using hashes. So, if somebody changes something in block &#39;2’, it will change the hash of block &#39;2’, but, since we used the previous hash of block &#39;2&#39; for generating the hash of block &#39;3’, this change would result in generating an invalid hash for block &#39;3&#39; after changing the block &#39;2’. In most simple terms, updating previous data will result in generating invalid hashes in the blocks which come later, making the whole data structure immutable.</p><p>So, this is the power of the blockchain data structure. But, how are we gonna make this useful? Distributed Systems enter! What we do now is, use a distributed system to share the same version of this data structure in every node of the distributed system. Well, this is how the fancy term ‘distributed ledger’ comes into play.</p><h4>Supply Chains? Why should we track them?</h4><p>As we know what is a blockchain now, let’s talk about Supply Chains. Well, a supply chain is the network of entities, work flows and processes involved during the life cycle of a consumer product. For an example, say we have a cheese production supply chain. For producing cheese, we need to buy milk, preservatives, yeast and salt from various suppliers. These raw materials should be transported to the production factory. At the end of the production, cheese should be packaged and distributed to finally be consumed by the end customers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KmdYf8FELaZwc-wzeJ-jag.png" /></figure><blockquote>Cheese production supply chain</blockquote><p>As we can clearly see, the entities even in this overly simplified supply chain are in geographically different areas. The milk could be produced in Ambewela, the salt could be produced in Hambanthota. The production factory could be in Colombo and the actual packaging could happen in Switzerland. The packaged cheese could be sold in America and it finally could be consumed on Mars. (Yes <a href="https://medium.com/u/76caee166b9c">Elon Musk</a>, we believe in you). So, as you can see, the entire ecosystem is embarrassingly distributed throughout the world, (or solar system). The issue now is, the data related to these supply chains are fragmented and distributed throughout the supply chain without any connection between them. This phenomenon is called ‘data silos’ and the end customers will not be able to realize the connections between these data and trace back to the origin of the product. As an example, if one of our Martian friends gets food poisoning from consuming the cheese he took with him, it will be impossible to determine if the issue was with milk, yeast or poor packaging. So, in current supply chains, traceability is becoming a major requirement.</p><h4>Blockchain for Supply Chains?</h4><p>Until now, we only looked into Blockchain and Supply Chains. But, where does these two intersect? Well, we saw supply chains are embarrassingly distributed from our overly simplified cheese production supply chain. And we discussed blockchain is useful as a distributed system than a single data structure. So it is somewhat intuitive to combine the two. But, is this the only reason? Actually No. In our previous discussion, we discussed about blockchain as an immutable data structure and how it is impossible to manipulate the data in it. Well, this can be very useful in our supply chain tracking solution. Revisiting the previous cheese example where our Martian friend gets food poisoning, let’s assume the tracking system was distributed but not immutable. So, when our beloved Musk finds a party to blame for the low quality cheese, a rogue sysadmin in the packaging company could change the data in the system so it would show they have used a A grade packaging material although what was used in actual packaging was C grade packaging. As you can see, it is important to maintain an immutable data structure for the supply chain tracking solution while it is being in a distributed system. Thus, <strong>Blockchain!</strong></p><h4>Well, a Conclusion?</h4><p>When humans write documents, it is a tradition to write a short conclusion and a summary at the end. So, keeping up with the traditions, here it goes.</p><p>The trending new technology, Blockchain is a distributed immutable data structure and it is proven successful in practice. Supply chains are distributed, yet fragmented into data silos. So, blockchain is a fitting alternative to build a distributed supply chain tracking solution.</p><p>If you have survived throughout this post so far, I should mention that this post was intended for an audience which is not familiar with blockchain or supply chain traceability. Thus, some of the things are overly simplified for understanding concepts while keeping it educational. This will be the first of the series of posts I’m planning to write since this is the research topic of our undergraduate final year research project. So, thank you for bearing with me, and your comments on this will be greatly appreciated.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=60f7cc72f9b0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Improving your business productivity using Data Analytics (ELK Stack)]]></title>
            <link>https://medium.com/@isurutharanga/so-wheres-the-animal-a7ece51add51?source=rss-7abfec95c9d2------2</link>
            <guid isPermaLink="false">https://medium.com/p/a7ece51add51</guid>
            <category><![CDATA[kibana]]></category>
            <category><![CDATA[logstash]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <dc:creator><![CDATA[Isuru Perera]]></dc:creator>
            <pubDate>Mon, 30 Oct 2017 17:35:28 GMT</pubDate>
            <atom:updated>2017-10-30T17:41:53.252Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DW5LlewLmUGwoJqW8G1J-Q.png" /></figure><p>So, where’s the animal? For the smallest fraction of you came here looking for an elk, you are in the wrong article or in the right one by accident if you’re into data science. For the rest of you, thanks for surviving through my desperate attempt to make some totally unrelated people laugh. From the jokes apart, let’s dive into what the first part of the title says.</p><h3>So, What’s data analytics</h3><p>According to the first result of google search,</p><blockquote>Data analytics refers to qualitative and quantitative techniques and processes used to enhance productivity and business gain.</blockquote><p>Boring right? But what’s not boring about the whole thing is that last two words, “business gain”. Why? Because one thing I learned from my internship in a very reputed company is, no matter how techy your tech is, it’s useless if you can’t apply it to improve the productivity and of course the business gain of your company. With that in mind, let’s see how data analytics can be applied to your everyday tech business.</p><p>If you ever developed or at least maintained business software, you must already know how painful it is to read through all those logs to verify if that one customer who keeps telling his cash account balance is wrong, is telling the truth or just forgotten how he spent his cash. For the rest of you who has no clue about what I’m trying to say, don’t worry. Read through the following log and try to find the word “rabbit”. I’ll give you 1 minute to try.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EedEzoeBMXb5DBq637H4Ew.jpeg" /></figure><p>Since now that minute has passed, or you’ve just skipped reading the log because you have much more important things to do other than reading a log, it’s safe to reveal that there’s no word “rabbit” in the log. If you’ve really read through the example log, you must have felt how painful it is to read through it just to find out there’s no “rabbit”. In most of the business cases, there’s no rabbit even after two days of reading through server logs containing more than two billion log lines. Again, you can just say, “Hey, if you’ve just provided me the text file, I’d just ctrl+F and find the error” but in real life, you can’t just ctrl+F into the unknown.</p><p>Then how are you gonna increase your business productivity or at least give a relief to the poor guy who’s supposed to read logs and find out if there’s any issue with your system? Obviously through data analytics, since this article is about it. But how to do it easily enough to integrate log data analytics right into your business application?</p><p>Enter ELK Stack.</p><h3>ELK? What do you mean?</h3><p>In short, ELK stands for Elasticsearch, Logstash and Kibana. It’s an emerging open source software stack that you can analyze, store and visualize virtually any type of data, without changing anything in your current application code.</p><p>And, Yes, Elasticsearch, I know right? The full text searchable database you, me and everyone else who’s into NoSql heard enough about. If this is the first time you ever heard about Elasticsearch, don’t worry it’s just like any other database which you can query with Lucene query syntax. Not enough? Let’s dive into details.</p><h3>Meet Elasticsearch</h3><p>The heart or the backend of the ELK stack, which is a powerful distributed database capable of searching and producing analytics of your data right away. Boring enough? Say no more! Right from the Elasticsearch website I quote,</p><blockquote>Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data so you can discover the expected and uncover the unexpected.</blockquote><p>As you can see, the description from their site is self explanatory. And all of their developer documents are easy to understand as well. Move on shall we?</p><h3>Logstash what?</h3><p>Since now we have a slight idea about Elasticsearch, let’s talk about Logstash. But before that, let me guess, you must be wondering well, Elasticsearch can do analytics and all, I talked about analyzing log data but how the hell am I supposed to insert my log data into Elasticsearch? Hold on, we’re getting there.</p><p>Now it’s about time to meet Logstash. It’s a data analyzer which is capable of matching your log data with regular expressions so the data can be broken into more meaningful fields which can be stored in Elasticsearch. Let’s break down the analysis process so you can understand more.</p><h4>Input</h4><p>As you might have already guessed, input is your log file or log data that needs to be processed.</p><h4>Filter</h4><p>Well, this is where the magic happens, you can setup filters to match with regular expressions so they can be broken down into more meaningful fields in Elasticsearch. Let’s understand this further using an example.</p><p>Suppose you have the following log line.</p><blockquote>2017-07-02 20:52:39 ERROR MagicRabbitImpl:200 - MagicRabbitException was thrown while trying to poke the Rabbit</blockquote><p>Well, this is a very common handled error message if you have ever used log4j for logging. Basically it includes a timestamp, loglevel, java class, line number and a log message.</p><p>Now look at the following regular expression.</p><blockquote>%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel}%{DATA:class}:%{DATA:line} - %{GREEDYDATA:message}</blockquote><p>It doesn’t look like much, or like gibberish if you’re not into regular expressions, but let me show you how powerful it is.</p><blockquote>{</blockquote><blockquote>&quot;timestamp&quot;: [ [ &quot;2017-07-02 20:52:39&quot; ] ],</blockquote><blockquote>&quot;loglevel&quot;: [ [ &quot;ERROR&quot; ] ],</blockquote><blockquote>&quot;class&quot;: [ [ &quot; MagicRabbitImpl&quot; ] ],</blockquote><blockquote>&quot;line&quot;: [ [ &quot;200&quot; ] ],</blockquote><blockquote>&quot;message&quot;: [ [ &quot;MagicRabbitException was thrown while trying to poke the Rabbit&quot; ] ]</blockquote><blockquote>}</blockquote><p>If you read the data above, you can easily see now that the unstructured, unpleasant log line has converted into a json string you and I love, just by using the regular expression shown above in the article. You can see, how the variables map into the json string. I assume it’s self explanatory.</p><h4>Output</h4><p>So, we have converted our log line into json, now what? We can export the data into our beloved Elasticsearch database right from Logstash.</p><p>In summary, here’s what Logstash does.</p><ul><li>input(logs) =&gt;</li><li>filter(match logs with regular expressions and convert to json) =&gt;</li><li>output(export to Elasticsearch)</li></ul><p>So, now we have analyzed log data stored in a Elasticsearch database. Did your business productivity increase even by a fraction so far? No clue? Read on, you’re gonna like what’s about to come.</p><blockquote>I explained only one use case of filter which is matching log line with a regular expression. This is done through grok filter and the regular expressions are called grok patterns. There exist other filtering methods which are out of the scope of this article.</blockquote><blockquote>Input for the Logstash can be anything, Here, I explained the simplest usecase of an input which is a text file. Input can be configured to receive log events from beats plugin, kafka statistics, jvm statistics etc, which is out of the scope of this article.</blockquote><h3>You’ll love Kibana</h3><p>Upto this sentence, you came across the boring part of the article. Now it’s time for the fun part. We have analyzed and stord our log data so far, using Logstash and Elasticsearch. But how can you visualize the data? Kibana is the answer.</p><p>Kibana can be used to visualize the data we’ve stored in Elasticsearch using beautiful bar charts, pi-charts, graphs, time series and data tables. Or if that’s not hardcore enough, you can use timelion to build graphs using its unique expression language or you can build your own kibana plugins as well.</p><p>If you have no clue what I was trying to say about kibana, just take a look at this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*3ASzb8kmSXASlHDuKrql8A.png" /></figure><p>Yes, you too can build killer dashboards to visualize your own log data and boost the productivity of your business. You can represent millions or billions of lines of log data in a single dashboard like the above.</p><p>So, to wrap up what we’ve just did using ELK Stack, we converted our unstructured log file into structured data using Logstash and stored it in Elasticsearch. And we visualized the data using Kibana.</p><p>Here is a simplified architecture diagram of ELK log analyzer for you to understand it better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RTVnGPqsquPbap5oVLhZiw.png" /></figure><p>Finally, about that annoying customer we came across in the beginning of this article, now you can tell him either <em>sorry for the inconvenience, we will fix the issue now!</em> or to <em>go home!</em> just in a click of a button.</p><p>To summarize what you’ve just read,</p><ul><li>Data analytics is an emerging field with the high amount of data produced in the production environments of modern application software.</li><li>ELK stack can be used to easily analyze, store and visualize log data related to your business applications.</li></ul><p>On a side note, this is my first ever article on Medium.com so, pardon me if I’ve gone wrong somewhere. And that’s why your comments are much appreciated. I kept the article very short since I my intention was to give an intro to ELK for someone who would find it interesting. ELK stack is easy to understand, but there’s much to learn if you really wanna use it on a production environment. If you’re interested, I’m more than happy to write another article explaining specific details about each component of the ELK Stack.</p><blockquote>References:</blockquote><ul><li><a href="https://www.elastic.co/guide/en/kibana/3.0/saving-and-loading-dashboards.html">https://www.elastic.co/guide/en/kibana/3.0/saving-and-loading-dashboards.html</a></li><li><a href="https://www.elastic.co/">https://www.elastic.co/</a></li><li><a href="https://www.techopedia.com/definition/26418/data-analytics">https://www.techopedia.com/definition/26418/data-analytics</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a7ece51add51" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>