<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rohith Kunnath on Medium]]></title>
        <description><![CDATA[Stories by Rohith Kunnath on Medium]]></description>
        <link>https://medium.com/@rohithkunnath?source=rss-412e98e71c5a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*cO3AGcL2VeQSoazob3WwSQ.jpeg</url>
            <title>Stories by Rohith Kunnath on Medium</title>
            <link>https://medium.com/@rohithkunnath?source=rss-412e98e71c5a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 09:39:56 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rohithkunnath/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How We Continuously Improved UI Automation with Selenium in a Regulated Medical Software…]]></title>
            <link>https://rohithkunnath.medium.com/how-we-continuously-improved-ui-automation-with-selenium-in-a-regulated-medical-software-a81d4f593b8a?source=rss-412e98e71c5a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a81d4f593b8a</guid>
            <category><![CDATA[efficiency]]></category>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[ui-automation-testing]]></category>
            <dc:creator><![CDATA[Rohith Kunnath]]></dc:creator>
            <pubDate>Thu, 26 Mar 2026 08:29:33 GMT</pubDate>
            <atom:updated>2026-03-26T08:29:33.507Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>How We Continuously Improved UI Automation with Selenium in a Regulated Medical Software Environment</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ITkcuCgO8cwg-2MZsMD8Ww.png" /></figure><p>When teams talk about UI automation, the conversation usually starts with test frameworks and tools. In our case, the story was much bigger than Selenium alone. It was about cost, release pressure, regulatory expectations, test-data complexity, infrastructure constraints, reporting overhead, and eventually building an ecosystem around automation instead of just writing UI tests.</p><p>Over time, our UI automation journey became a continuous improvement program rather than a one-time engineering effort. What started as a small set of simple Selenium tests running on expensive Windows machines evolved into a scalable, data-aware, cost-efficient, and compliance-oriented automation platform that now supports a meaningful part of our regression process.</p><h3>The Release Reality We Had to Support</h3><p>Our product release cycle is demanding by design. We typically deliver one release every two months, and every release artifact must pass regression in multiple non-production environments before it can move forward. The execution order is strict: first the stage cluster, then the certification environment. Each regression cycle takes roughly one week per environment.</p><p>This means testing is not a one-time activity at the end of development. It is a recurring operational process that repeats for every release and for every environment. On top of that, we work in a regulated medical domain, so testing is not only about finding defects. It must also satisfy documentation, traceability, review, and sign-off expectations.</p><p>At the time, more than ten full-time manual QA engineers were involved in executing this lifecycle. The effort was significant, the timelines were fixed, and every inefficiency in testing had a direct cost in people, infrastructure, and release confidence.</p><h3>Where We Started</h3><p>Our initial automation setup was driven by convenience and local developer familiarity more than long-term scalability.</p><p>The technology stack was Java, Kotlin, and Maven. Since company laptops were Windows-based, the automation framework was primarily developed and validated on Windows. That made development fast in the beginning, but it also shaped infrastructure decisions later in ways that became expensive.</p><p>We ran our tests on AWS EC2 Windows instances. These machines were not cheap. We had up to five instances configured, and each one cost around 1000 euros per month. The cost was even higher because our product required rendering of GL-based elements, so GPU-capable machines were needed.</p><p>The framework was also effectively single-threaded. Since the tests were not headless and depended on a graphical session, we could only run one test at a time per machine in a reliable way. That meant our first automation scope had to be carefully chosen. We started with easy and stable scenarios, mainly around new patient workflows, because those were easier to automate and easier to maintain while the framework was still maturing.</p><p>This phase gave us our first value from automation, but it also made the limitations obvious. The framework worked, but it was expensive, narrow in coverage, and not yet capable of handling the real complexity of our regression suite.</p><h3>Refactoring Phase 1: Making Complex Test Cases Possible with a Test Data Service</h3><p>As soon as we wanted to automate medium and heavy test cases, a new bottleneck appeared: test data.</p><p>Simple test cases can often be created from scratch. Complex cases are different. They depend on patient histories, pre-existing states, linked files, external assets, and environment-specific structures. In our domain, realistic data is often the difference between a useful regression test and a test that only validates the happy path.</p><p>To solve this, we introduced a dedicated test-data-service built as a Spring Boot application in Kotlin. Its role was much more than generating dummy records. It could anonymize production-like data and provision it for automated tests in a reusable and environment-aware way.</p><p>The service was capable of:</p><ul><li>anonymizing sensitive production-derived data,</li><li>copying and preparing S3 assets,</li><li>copying network files,</li><li>adapting the prepared data to the structures used in different environments,</li><li>and running the process asynchronously so that test preparation did not block the entire pipeline.</li></ul><p>This was a major shift in how we designed test automation. Instead of forcing every test to create all of its own data from scratch, new tests could request prepared, realistic data sets from the service, modify only what was necessary for the scenario, and execute against a much more production-representative state.</p><p>We also designed a cleanup model around this. After test execution, cleanup is initiated, but the data is intentionally retained for two days before deletion. That retention period is extremely useful for debugging, failure analysis, and investigation. In regulated environments, being able to inspect exactly what a failed test used can save a lot of time.</p><p>Another important design decision was to make the service compatible with all of our test environments. Release artifacts in different environments do not always behave against identical structures, so the service had to understand those differences and adapt accordingly. Without that, automation would have remained fragile and environment-specific.</p><p>This phase was critical because it moved us from “automation for easy scenarios” to “automation that can support realistic regression coverage.”</p><h3>Refactoring Phase 2: Migrating from Windows to Linux and Reducing Infrastructure Cost</h3><p>Our next major challenge was infrastructure cost.</p><p>The Windows EC2 model was expensive, hard to scale efficiently, and tightly coupled to the original way the tests were written. We needed to reduce cost without losing execution capability, especially because automation was beginning to prove its value and we wanted more of it.</p><p>The move to Linux was the obvious answer, but it was not a simple lift-and-shift. The framework had implicit assumptions about Windows paths, file handling, and supporting dependencies. We had to refactor the codebase so it could run cleanly on both Linux and Windows where necessary.</p><p>At the same time, we changed the execution model. Instead of relying on separate AWS EC2 Windows instances, we started using larger machines within our existing Kubernetes cluster. This allowed us to consolidate infrastructure, simplify management, and eliminate the need for individually maintained Windows hosts.</p><p>Another key change was moving to headless test execution. Headless operation opened the door to better automation density, easier scheduling, and more container-friendly execution. For a UI framework, this was a foundational improvement.</p><p>We also addressed the GPU dependency. Because parts of the product relied on GL rendering, GPU-backed machines had originally seemed unavoidable. By moving toward OpenGL/Vulkan support in a way that worked in our execution context, we reduced or removed the hard dependency on dedicated GPU machines. That decision had a very direct financial impact.</p><p>Even after these improvements, concurrency was still limited. We now had bigger Kubernetes machines, but practical execution was still close to one test at a time per pod because the framework and supporting components were not yet optimized for higher density. Still, this phase was a huge step forward because it converted automation from a high-cost experiment into a platform worth investing in.</p><h3>Automating Result Publication into TestLink</h3><p>While execution was improving, another inefficiency remained: reporting the outcome.</p><p>Uploading results into TestLink was time-consuming and repetitive. Even when the test itself was automated, part of the process still relied on manual handling to get the final result into the test management system. That created friction and reduced the real productivity gain of automation.</p><p>To solve this, we built an intermediate service around RabbitMQ.</p><p>The idea was straightforward but powerful. After Jenkins completed test execution, it published result data to RabbitMQ. Our intermediate service consumed those messages, transformed the raw execution output into the format expected by TestLink, and then published results to the correct project and test artifacts.</p><p>We also improved the test suite itself to carry TestLink identifiers directly, which made the mapping deterministic and removed ambiguity. On top of that, we built custom reporting modules so the published data contained more than just pass/fail status.</p><p>This turned out to be one of the highest-leverage investments in the whole journey. The biggest win was not only that automated tests could report themselves. It was that the same intermediate service could also support manual test result publication workflows. In other words, we were no longer optimizing only for automation engineers. We were improving the entire quality process.</p><p>That made reporting consistent, reduced manual effort, and created a central place from which metrics could later be extracted.</p><h3>Refactoring Phase 3: Virtual Threads, Interceptors, and Better Evidence Collection</h3><p>Once execution and result publication became more stable, the next logical step was improving documentation and diagnostic value.</p><p>In a regulated environment, a test result alone is not enough. You also need supporting evidence. When a test fails, teams need to understand why. When a test passes, teams often need enough context for review, traceability, and certification discussions.</p><p>To address that, we began enriching every UI test with interceptors and recorders which is about another 8 threads for each test. These captured supporting signals such as:</p><ul><li>network requests,</li><li>console logs,</li><li>screenshots and execution evidence,</li><li>and additional metadata useful for custom reporting.</li></ul><p>This dramatically improved the usefulness of our reports, but it also increased the runtime footprint of each test. A single UI test could now involve around eight supporting threads because of the added interceptors and background collectors.</p><p>At that scale, traditional thread management started becoming a limiting factor. The answer for us was Java 21 and virtual threads.</p><p>At the time, Selenium 4.x did not provide the level of support we needed for this model. So we made a bold decision: we pulled the necessary Selenium internals into our own codebase and replaced the relevant thread pool usage with our virtual-thread-based implementation.</p><p>This was not a small change, and it came with maintenance responsibility. Because browser DevTools behavior evolves, we sometimes need to adjust our customized Selenium code when upstream changes happen. However, the trade-off has been absolutely worth it.</p><p>With virtual threads, we were able to run a JVM pod with a maximum heap around 1 GB for smaller cases, and later scale to around 3 GB heap while supporting up to 10 parallel tests, each with multiple interceptors (in average of 8 threads) attached. That density would have been much harder and more expensive to achieve with a conventional threading model.</p><p>This phase gave us more than raw concurrency. It made the framework operationally viable at scale while simultaneously improving the quality of reporting and documentation.</p><h3>Connecting Automation to Deployment Pipelines and Certification</h3><p>The next maturity step was organizational, not just technical.</p><p>As automation became more reliable, we connected it directly to deployment pipelines. This meant automation was no longer an isolated engineering asset. It became part of release readiness.</p><p>At the same time, the certification process became stricter. We introduced a benchmark where the regression cycle is evaluated over a seven-day period, with a minimum average pass percentage of 80%.</p><p>At first glance, some might ask why the threshold is not 100%. The reason is simple: UI automation at scale can still be affected by instability from external systems, environment behavior, timing issues, last-minute requirement changes, and occasionally genuine test flakiness. The 80% threshold reflects a practical quality gate that balances ambition with operational reality.</p><p>When the required level is not met, it triggers stakeholder discussion rather than silent acceptance. That discussion focuses on identifying the failure source:</p><ul><li>Was it an environment issue?</li><li>Was it a dependency outage?</li><li>Was there a last-minute requirement change?</li><li>Was the automation outdated?</li><li>Was there a manual process error in setup or interpretation?</li></ul><p>This changed the culture around automation. Instead of treating failures as isolated technical issues, we started treating them as signals in a larger release system.</p><h3>Expanding to Multi-Product End-to-End Scenarios</h3><p>Once the core platform became robust enough, we stopped limiting automation to lighter flows and started targeting much heavier end-to-end scenarios involving multiple products.</p><p>These are the kinds of tests that provide the most business confidence because they validate real workflows across system boundaries. They are also the hardest to automate because they depend on richer data, more integrations, and more coordination.</p><p>To make test selection more systematic, we introduced TestLink processes using filters and keywords to identify which manual test cases should be automated next and in what priority. This gave us a better pipeline for automation candidates instead of relying only on ad hoc selection.</p><p>That process improvement mattered a lot. Automation success is not only about execution technology; it is also about choosing the right tests to automate. By identifying high-value candidates earlier, we were able to expand faster and spend effort where it delivered the most impact.</p><p>We also made screenshots part of the reporting flow, with evidence uploaded through the intermediary service into TestLink. This improved visibility for both engineering and QA stakeholders and made review easier.</p><h3>Release-Level Reporting on Automation Coverage</h3><p>Because the intermediate service sat between execution and test management, it gradually became a central source of truth for automation metrics.</p><p>In earlier stages, calculating automation savings, coverage improvements, and execution trends required manual effort. We would estimate time saved or coverage percentages with separate calculations. That was useful, but not scalable.</p><p>Now the system can generate reporting automatically. When a new product version is deployed, the intermediary service receives a message, prepares the relevant automation coverage and execution report, and sends it to stakeholders the next day.</p><p>This closed an important loop. Automation was no longer just executing tests; it was also producing management-ready visibility. That made it easier to justify investments, monitor progress, and prioritize the next improvements.</p><h3>Where We Are Today</h3><p>Today, more than 30% of our regression tests are automated. In a medical and regulated environment with complex UI flows, heavy data dependencies, and multi-environment validation, that number represents much more than just coverage percentage. It represents a sustained engineering effort to remove bottlenecks across the whole testing lifecycle.</p><p>The gains are tangible:</p><ul><li>significant reduction in manual validation effort,</li><li>roughly 10 hours saved on average per regression cycle across our test environments,</li><li>automated result publication,</li><li>automated documentation support,</li><li>the ability to execute in any supported environment without long setup delays,</li><li>and a dramatic reduction in infrastructure cost, from roughly 25,000 euros per year to less than 500 euros per month.</li></ul><p>More importantly, our automation capability is no longer limited to “simple test cases.” With the help of the test-data-service, intermediary reporting platform, and scalable execution model, we can automate far more realistic and valuable scenarios.</p><p>The certification process has also become stricter and more measurable, which is a positive outcome. Automation is now part of how we govern release confidence, not just how we save execution time.</p><h3>What This Journey Taught Us</h3><p>The biggest lesson from this journey is that UI automation does not scale by test scripts alone.</p><p>It scales when test data is solved.<br>It scales when infrastructure cost is controlled.<br>It scales when evidence collection is built in.<br>It scales when reporting is automated.<br>And it scales when the organization treats automation as part of release engineering, quality governance, and compliance readiness.</p><p>Selenium was an important part of our story, but it was never the full story. The real transformation came from continuously improving the ecosystem around Selenium until automation could operate as a reliable and efficient testing platform.</p><p>And that is probably the most important takeaway: sustainable UI automation is not a framework decision. It is an architecture and process decision.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a81d4f593b8a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Migrating a Windows OpenGL Application from AWS to On‑Prem: A Productivity vs Cost Story]]></title>
            <link>https://rohithkunnath.medium.com/migrating-a-windows-opengl-application-from-aws-to-on-prem-a-productivity-vs-cost-story-e2d108a01359?source=rss-412e98e71c5a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2d108a01359</guid>
            <category><![CDATA[appium]]></category>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[selenium]]></category>
            <category><![CDATA[automation]]></category>
            <dc:creator><![CDATA[Rohith Kunnath]]></dc:creator>
            <pubDate>Thu, 19 Feb 2026 22:00:22 GMT</pubDate>
            <atom:updated>2026-02-19T22:00:22.842Z</atom:updated>
            <content:encoded><![CDATA[<p>Migrating infrastructure is rarely about technology alone. More often, it is a balancing act between productivity, cost, and long‑term sustainability.</p><p>This article documents our journey of migrating a Windows‑based application that uses OpenGL for 3D rendering from AWS to an on‑prem environment. The journey unfolded in three distinct versions — each iteration teaching us something critical about tooling, scalability, and the real cost of automation.</p><p>The Problem Space</p><p>Automating a Windows application that relies heavily on GPU‑accelerated OpenGL rendering is fundamentally different from automating web applications.</p><p>Key constraints shaped every decision we made:</p><ul><li>UI automation requires an active Windows session</li><li>Keyboard and mouse interactions are unavoidable</li><li>Only one test can run per active session per machine</li><li>GPU‑enabled machines are expensive — especially in the cloud</li><li>With those constraints in mind, let’s walk through the three versions of our solution.</li></ul><p><strong>Version 1: Ranorex (SaaS Prototype)</strong></p><p>We started with a quick prototype using Ranorex, a SaaS tool designed specifically for automating Windows applications.</p><p>What Worked Well</p><p>Ranorex helped us move fast initially:</p><ul><li>Minimal coding required</li><li>Easy test creation using video recording</li><li>Built‑in element scanner to identify UI components quickly</li><li>For early experimentation, this was undeniably productive.</li></ul><p>Why It Didn’t Work for Us</p><p>However, cracks appeared almost immediately:</p><ul><li>We have more engineers who prefer coding than those who don’t</li><li>Licensing costs were high and fragmented: One license for Ranorex Studio (IDE), rotated among engineers. Separate licenses for runtime agents (for Jenkins or CI)</li><li>Even minor test changes felt tedious. Generating Java code manually — or now with AI assistance — felt far easier and more flexible</li></ul><p>The decision was quick. Within a week, we moved away from Ranorex.</p><p><strong>Version 2: Selenium + WinAppDriver on AWS</strong></p><p>Most of our engineers come from a JVM and Selenium background, so the next choice felt natural.</p><p>The Approach</p><p>We built a prototype using:</p><ul><li>Selenium</li><li>TestNG</li><li>Windows Application Driver (WinAppDriver)</li></ul><p>At the time, there were very few examples online showing Selenium being used for Windows automation. Many resources dismissed Selenium entirely, claiming it was only meant for web applications.</p><p>Persistence paid off.</p><p>We managed to get a working prototype by directly integrating Selenium with WinAppDriver, an open‑source Microsoft project that listens for automation commands and performs actions on Windows systems.</p><p>We evaluated Appium as well but decided to integrate directly with WinAppDriver at this stage.</p><p>The Harsh Realities</p><ul><li>This setup came with serious challenges:</li><li>WinAppDriver is effectively unmaintained</li><li>Minimal documentation and a very small community</li><li>Generic and cryptic errors that led engineers nowhere</li><li>Major compatibility issues with Selenium versions post 4.2</li></ul><p>The Upsides</p><p>Despite all that, there were strong positives:</p><ul><li>Selenium commands felt familiar to everyone</li><li>Most interactions worked seamlessly with WinAppDriver</li><li>Microsoft’s Application Insights tool made element inspection much easier</li><li>The stable version never crashed once during our entire prototype phase 😅</li><li>Cost model was simple: pay for the server and nothing else</li></ul><p>At this point, we had a technically sound solution — but the real problem was still ahead.</p><p><strong>The Scaling Reality Check</strong></p><p>Before jumping to the final version, here are some critical facts we realized from Version 2:</p><ul><li>Windows UI automation always requires an active UI session</li><li>Only one test per machine at a time</li><li>GPU is mandatory for OpenGL‑based 3D rendering 💵</li><li>We scaled from 10 regression tests to 600 in just 3 months</li></ul><p>The Cost vs Productivity Breakdown</p><ul><li>One GPU‑enabled AWS machine cost us ~$1000/month</li><li>That single machine took ~16 hours to run the full suite</li><li>CI pipelines were constantly blocked</li><li>Horizontal scaling was financially unrealistic</li></ul><p>At this point, productivity was clearly suffering — and budgets were under real pressure.</p><p><strong>Version 3 (Final): On‑Prem + Mesa3D + Grid‑Based Scaling</strong></p><p>This is where things got interesting.</p><p>The Core Idea</p><p>Our research focused on using on‑prem, decommissioned hardware combined with Mesa3D, an open‑source implementation of graphics APIs that can translate GPU instructions without requiring a physical GPU.</p><p>The Risks</p><p>We were fully aware of the trade‑offs:</p><ul><li>Slower rendering and computation compared to real GPUs</li><li>Dark or high‑contrast colors rendered flatter or lighter</li><li>Required color‑tolerance logic in tests to support both environments</li></ul><p>The Breakthrough</p><p>Despite the risks, the results exceeded expectations:</p><ul><li>We acquired an Mx740c blade with Windows Volume Licensing</li><li>The blade now hosts 25 virtual machines</li><li>Integrated using the latest Selenium Grid</li><li>Migrated to Appium Server with WinAppDriver running in the background</li><li>Integrated Windows Nova Driver: Open source, actively maintained, much faster keyboard and pointer interactions</li></ul><p>The Outcome</p><ul><li>600 UI tests running across 25 nodes in under 1 hour</li><li>Grid running 24/7 for months with 99.999% availability.</li><li>Pipeline wait times reduced to less than one hour</li></ul><p><strong>Final Thoughts: Productivity vs Cost</strong></p><p>On‑prem infrastructure comes with a cost of self‑maintenance — there’s no denying that.</p><p>But when viewed through the lens of productivity per dollar, the decision was clear.</p><p>We now operate under a model we truly believe in:</p><p>“You build it, you run it.”</p><p>This final architecture removed our biggest bottlenecks — cost, waiting time, and scalability — while giving engineers full ownership of the system.</p><p>And that’s why we believe this is our final version.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2d108a01359" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sometimes it is better to reinvent the wheel]]></title>
            <link>https://rohithkunnath.medium.com/sometimes-it-is-better-to-reinvent-the-wheel-6f59c4f400b1?source=rss-412e98e71c5a------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f59c4f400b1</guid>
            <category><![CDATA[monolith]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[clean-code]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Rohith Kunnath]]></dc:creator>
            <pubDate>Wed, 28 Dec 2022 13:06:45 GMT</pubDate>
            <atom:updated>2022-12-28T13:06:45.881Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*4svp7vMfNuMRQKjbmuljJA.jpeg" /></figure><p>It is time to share another work experience with the world :).</p><p>This year 2022 has been a thrilling ride with Linus Digital Finance solving problems and making life easier for our Investors. The craziest part was the challenge to enhance the open-source project “Fineract” for our business model.</p><p><strong>Why Fineract?</strong></p><p>One of the main components of our business model is lending loans and managing the Repayments. One of the things Fineract also does is manage Loans and Repayments. However, this was not only our selection criteria but also preferably an application built using any of the JVM languages. These requirements made us choose Fineract over the other alternatives.</p><p><strong>Positives</strong></p><ol><li>Liquibase support — Integration of Liquibase helps to have a clean database migration.</li><li>Supports the latest Java versions.</li><li>Kubernetes Deployment and service charts or YAML files are available.</li><li>Clear Documentations.</li></ol><p>Now let me try to explain after almost a year, why I think “Sometimes it is better to reinvent an MVP model”.</p><p><strong>Fineract Build times</strong></p><p>Everyone knows why management and developers love microservices. Some of the reasons are</p><ol><li>The power to scale teams much faster</li><li>Releasing things faster by making a small number of resources.</li><li>Clearly defined boundaries.</li></ol><p>Fineract as a monolith with more than five modules uses Gradle for the build process. The whole build takes 15–20 minutes to produce a jar file.</p><p><strong>Integration Tests</strong></p><p>Integration tests are run along the build using “Cargo Plugin”. ie, Runs the jar application as a process using the cargo plugin and then runs the series of tests against this jar. Imagine running more than 75 SQL migrations every time in your CI/CD pipeline when a build is triggered.</p><p><strong>No Clear Boundaries</strong></p><p>The modules are reused in every other module and make the whole codebase tightly coupled. This always makes me sleepless nights when thinking about splitting this monolith into microservices in the future.</p><p><strong>Never-ending while loops</strong></p><p>Do not be alarmed when your HTTP request does not get finished after a couple of seconds. There are while loops like this `WHILE(TRUE)` at least in 2–3 java classes without max tries. This potentially causes thread starvation sometimes. Although Integration tests can show a 100% success rate, certain property values misconfigurations can lead up to this :(.</p><p><strong>God Objects everywhere</strong></p><p>God Objects or classes can be seen everywhere. Single Responsibility Principle discarded in high regard.</p><pre>class A extends B{<br>}<br>class B extends C{<br>}<br>class C extends God{<br>}<br>class D extends God{<br>}<br>class Z extends God{<br>}</pre><p><strong>Debugging hell</strong></p><p>God objects explained above are one of the main reasons why we will experience Debugging hell. Forgetting where we started or forgetting what we came through is normal in Debug mode. There are quite a few reasons for that.</p><ol><li>Class methods have more than 100 lines of code.</li><li>The single-class method does multiple things even though the name says something different.</li><li>Propagating back and through to GOD classes doing so many things.</li><li>Similar method names but entirely different business logic.</li></ol><p><strong>What did we do?</strong></p><p>Realizing and prioritizing technical improvements helped us to achieve better and faster results in releasing new features using Fineract.</p><p>Some of the improvements we made are,</p><ol><li>Improve Gradle tasks by compiling only modules which has code changes (helps productivity during development phase).</li><li>Creating one sql data definition migration of fineract database instead of 75 sql migration files.</li><li>Capability to choose which modules to be loaded during the application startup.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f59c4f400b1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Making Share My Trip Feature much more robust.]]></title>
            <link>https://rohithkunnath.medium.com/making-share-my-trip-feature-much-more-robust-55c217bf8edd?source=rss-412e98e71c5a------2</link>
            <guid isPermaLink="false">https://medium.com/p/55c217bf8edd</guid>
            <category><![CDATA[event-based-architecture]]></category>
            <category><![CDATA[rabbitmq]]></category>
            <category><![CDATA[redis]]></category>
            <category><![CDATA[microservice-architecture]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Rohith Kunnath]]></dc:creator>
            <pubDate>Tue, 01 Feb 2022 18:02:03 GMT</pubDate>
            <atom:updated>2022-02-01T18:02:03.226Z</atom:updated>
            <content:encoded><![CDATA[<p>Working with one of the top urban mobility providers in Germany was definitely a High Point in my career. A lot of learning and unlearning happens on every career switch. One of them for me was refactoring or rewriting the Share My Trip feature.</p><p><strong>What is the Share My Trip feature?</strong></p><p>Whoever takes a ride in the App, can share the entire Trip route, plus information about the vehicle they are riding on via a Web URL. This is normally used to inform someone expecting your arrival about your current location. However, this can be also regarded as a Security feature in our fast-moving world.</p><h3><strong>Old Architecture</strong></h3><p>The old architecture was a microservice serving users real-time data collected from multiple data sources. The ratio of cons to pros was much higher in this design. Some of the cons caught my eyes and are listed below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s_7uII9PBe4s2vcsoD08aw.jpeg" /><figcaption>Legacy Arch.</figcaption></figure><p><strong>Increasing the chances of “Distributed Denial of Service Attack(DDoS)”.</strong></p><p>Since the URL is public which queries the big legacy monolith database with a lot of table joins, external driver service API calls, and who knows how many more things are going to be added there in the future. I would argue it is easy to trigger a DDoS Attack.</p><p><strong>Inefficient Resource Usage.</strong></p><p>Location Service stores the data in Distributed Redis Clusters and Redis eats nothing but the memory itself. As a company with thousands of transports on the road, we are talking about location updates at the rate of100k/min. Some features like ShareMyTrip making use of this resource every time is an Inefficient Resource Usage.</p><p><strong>Questioning “Availablity” and “Reliability” at the same time.</strong></p><p>Making our service available to our end users 99.9% percent of the time and being reliable to our end-users depends on the health of our mothership(the legacy database), a lot of microservices, and the monolith service which connects to the database.</p><h3>New Architecture</h3><p>The new architecture works purely based on Events. A hybrid model of Pub/Sub model and Producer/Consumer model is used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GrFwTaxXiLnZ1svvy3Gqsg.jpeg" /><figcaption>New Arch</figcaption></figure><p><strong><em>Stack</em></strong><em>:</em></p><p>Database - Mongo</p><p>Why? Mongo database helps us to easily mitigate schema changes from a lot of independent services. Similarly helps us to maintain a TTL for each booking.</p><p>Service - Spring boot service.</p><p>Producer/Consumer - Rabbimq.</p><p>Pub/Sub - Redis.</p><p>Understanding this would be easy if I starts explaining a User Journey.</p><ol><li>A user initiates the search for a Cab to travel from Position A to Position B.</li><li>Booking Service receives the request and creates publishes an event to the Rabbitmq via <a href="https://www.rabbitmq.com/tutorials/amqp-concepts.html#exchange-fanout">Fanout exchange</a>.</li><li>Our new service holds consumers which listen to the above events and save local copies to our Database.</li><li>Once a driver is available and is allocated to the booking, the Driver Service publishes another event to Rabbitmq via <a href="https://www.rabbitmq.com/tutorials/amqp-concepts.html#exchange-fanout">Fanout exchange</a>.</li><li>Our service updates the Driver Information for the respective booking with a TTL of estimated driving time plus a buffer of 15 minutes.</li><li>At the same time, our service will create a new <a href="https://redis.io/topics/pubsub">subscription</a> on Location Service for the respective Driver Id. This will help us to <strong>not</strong> receive all the location updates from every driver but only from drivers who are riding now.</li><li>For every further booking update, we receive an event from Booking Service and the same is updated in our database.</li><li>From the moment we receive the Booking event with the status “PASSENGER_CARRY”, the API responds with 200 status and respective body content.</li><li>From the moment we receive the Booking event with the status “PASSENGER_DROPPED”, the Redis Subscription for the Driver Id is deleted and the TTL for the booking will be set to 5mins. This is to make sure the data only exists for 5 more mins for anyone to track.</li></ol><p><strong>Learnings:</strong></p><ol><li>Make sure you have a max limit in the Queues to avoid bursting up of RabbitMq in case your consumers are Errored out.</li><li><a href="https://github.com/rabbitmq/rabbitmq-sharding">Sharding</a> is a priority if you need to scale RabbitMq. This helps us to handle events much faster and perform better during our Daily Peak times (8 am — 10 am).</li><li>Using REST API within Event-based architecture is not a great idea. If it is unavoidable, then latency-related complexities, proper re-queuing techniques, and related Error handling need to be evaluated.</li><li>Create proper indexes in the document database.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=55c217bf8edd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Database Query Statistics]]></title>
            <link>https://rohithkunnath.medium.com/database-query-statistics-b8524fdea2df?source=rss-412e98e71c5a------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8524fdea2df</guid>
            <category><![CDATA[logging]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[hibernate]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Rohith Kunnath]]></dc:creator>
            <pubDate>Tue, 26 Jan 2021 09:14:20 GMT</pubDate>
            <atom:updated>2021-01-26T09:14:20.383Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lpk70bmdRwsqu5TZ4hAtCA.png" /></figure><p>I am pretty sure most of us developers will face some database performance problem every day.<br>Ninety percent of the time this could be due to the missing indexes.</p><p>Otherwise, the query does not use the newly created index and instead scans the whole table.</p><p>Most of the time we will not notice this since the application works perfectly fine for the first few weeks/months and performance gets worse slowly every day as the table grows.</p><p>I myself would prefer something automated which will prompt or notify in-case queries take a long time than expected. This is when I thought of making use of the hibernate-statistics setting and turning on this log. But this is doing a lot of things in the background and making use of a lot of non-week references which also takes a fair amount of JVM memory.</p><p>I thought of implementing a small library that will help me to customize to enable what we need and not store any other information.</p><p>My motivation was</p><ol><li>Easy customizations using system properties.</li><li>Injectable even on non-hibernate or non-spring projects by little tweaks.</li><li>Statistics reporting will help the data to be sent to Prometheus or similar databases and create alerts to notify whether there are queries that need to be improved.</li><li>Developer need logs for SQL queries with the time taken to execute.</li><li>Developers can create alerts based on the Splunk (logging system) or Prometheus if there are queries taking more time or fetching rows more than expected.</li></ol><p><a href="https://github.com/jango89/hibernate-minimal-logger">Library link here</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8524fdea2df" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Are libraries/frameworks an overkill?]]></title>
            <link>https://rohithkunnath.medium.com/are-libraries-frameworks-an-overkill-17e97dfaef34?source=rss-412e98e71c5a------2</link>
            <guid isPermaLink="false">https://medium.com/p/17e97dfaef34</guid>
            <category><![CDATA[libraries]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[framework]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[learning]]></category>
            <dc:creator><![CDATA[Rohith Kunnath]]></dc:creator>
            <pubDate>Wed, 23 Sep 2020 11:52:06 GMT</pubDate>
            <atom:updated>2020-09-23T11:52:06.374Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Are libraries/frameworks an overkill?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*J0B2K2iKzRiQcfYwWJ8wkQ.png" /></figure><p>I am a Java developer who has been working on different products in different industries for the last 9 to 10 years. One thing I found very common is the frameworks or libraries on which we as developers rely during the development phase.<br>Most of the time the frameworks itself create antipatterns as part of their minor or major releases. The frameworks do this as part of a hack to nullify the pain caused by them in the earlier releases. Developers will have different opinions around this topic which I am well aware of. However, to be honest I found a huge percent of likeminded people who agree to a lot of pain points but discard them on a daily job though due to different unfamous reasons :).</p><p><strong>Maintainability hardships.</strong><br>To be frank, most of the developers know what are be the typical hardships on the maintainability topic or at least faced a couple of them during their daily coding times. Libraries are developed and open-sourced mostly as part of helping the society in a way to avoid reinventing the wheel. But the problem is, some wheels are reinvented only thinking about the near future which is good, but not great in terms of people opting to use the libraries. And these are the services which need to be maintained for the next 20 years. <br>We use some libraries to decrease the effort of typing some code and increase productivity by letting them generate the static code for us. A sign of relief, right? <br>Tomorrow someone else in your team updates or includes a completely unrelated library and this breaks the above library or even has a side effect. Now, this is not fun anymore. This allows me to dive into my next sub-topic.</p><p><strong>Migration hardships.</strong><br>There have been days where I start my day with a super happy face in the morning where my task is to upgrade the dependent library and thinking about releasing it in another 10 mins. However, we end up doing pair programming figuring out the other dependencies which break my tests now. At-times after spending a day, we revert the changes and think of not updating the library in this sprint since it was not productive and ends up never taking up this task. This adds up the technical debt and also makes some of your colleague’s lives hard in-case we switch the company or team without proper comments or findings.</p><p><strong>Dependency hell</strong><br>Absolutely hell! Most of our projects by itself will have tons of dependencies, even though what we needed was just a couple of classes from those libraries. Then each of these dependencies will add up another hundred of them. I agree DRY principle is one of the important concepts which everyone needs to have in their mind before coding. However duplicating some classes in each of our services instead of including the whole dependency is not a bad idea, isn’t it? We see microservices already adopted duplicating data in their schemas and moving away from the old idea of having data at a single place(one big database). Packaging tools or build automation tools are nowadays used in most of the projects and personally, I have worked with maven a lot. That is one of the reasons I love this plugin <a href="https://plugins.jetbrains.com/plugin/7179-maven-helper">https://plugins.jetbrains.com/plugin/7179-maven-helper</a>, and I highly recommend this. At least it eases the pain to some extent on getting closer to heaven from hell (a dream though :)).</p><p><strong>Cleaning up happens rarely</strong><br>Most of us join the team by thinking we make the code so beautiful and make them perform better. Also, use all these fancy words such as reliability, performance, availability, etc. a lot. But very few of us deletes or clean up libraries which may not be used anymore. Some of us do not even care or have a simple read about them. However, I cannot complain since we rarely get time to code every day as 90 percent of the day will be in meetings and then we end up working overtime to release features on time.</p><p><strong>Easily available</strong><br>Most of us rarely think of copying the logic from the library and not using it as a dependency inside the service itself. Say even if someone does that, Pull Requests will become the real enemies here. There will be comments and declines from most of your teammates saying the library is already available and why are you reinventing the wheel. Most of the time we end up in long discussions which are not productive and finally ending up with adding the library to the service itself. At this point, what I used to do is create a single-purpose library including only the necessary code and use it in this service and not using the entire library with 50+ files.</p><p><strong>Does clean code matters?</strong><br>Say we follow all the clean code and clean architecture principles, can we say our service follows these principles unless you know how the dependent libraries are developed? At least, I do not agree with this. However, we could argue, that whatever comes under our responsibility or in plain words, whatever we develop or maintain only matters. <br>Some libraries generate getters and setters for the classes using annotations, and we then spread it everywhere or overuse it. Most of us may not be known, some of these generated methods have the visibility mode set to public and forgetting anyone can mutate the states of these objects.</p><p><strong>Who needs Immutability?</strong><br>It is been a long time I am working in Java and people complain about Mutability. Really? Most of these people talking also support a lot of libraries that modify byte code that gets generated during compile time :). I guess sometimes we need to broaden our mindset and try to accept the facts. I prefer immutability and that is one of the reasons for hating these libraries.</p><p><strong>Hard to change.</strong><br>It is quite hard to change or improve things at times. I remember an example, we use spring framework and also a dependency which helps us to produce/consume events in Rabbitmq. So there were scenarios where we need to switch off our consumers but still produce messages. Nearly impossible to control it, since the consumer implementation has a parent class and it has a private variable where the threads are getting initialized. It took us one week to deprecate the whole library and write simple AMQP classes to produce/consume messages.</p><p><strong>Easy configurations via annotations.</strong><br>One of the reasons why we integrate some libraries is that it would be solving your current problem. However, one thing most of us do not research is how is it being solved and what are the hundred other problems we could end up shortly. Some of the fancy libraries come up with just one line to include in the package management XML file and then its all about creating key-value in the property files (developer-friendly), easy right? Believe me, changing something to include your application-specific feature would be hard in those libraries. Why? Either the library is built in such a way it would be so tightly coupled to the properties and specific logic, or it would be so generic, you need to write code to include your specific logic by implementing or extending the library classes. At times, in the end, you will not be needing the library itself because you wrote almost all the classes including most of the business logic.</p><p><strong>Our service is 90% framework code and 10% business logic.</strong><br>I was even thinking of leaving a blank line for this topic. It is understandable since we hear it from everyone during conferences and talks :).</p><p><strong>Annotations and reflections.</strong><br>At this point, let me Thank God for super-powerful cloud machines. We use profilers to measure the problems in our code and we try to improve them. But can we do something about the 100ms spent inside the framework filters and logics? No. Most of the frameworks use reflections and annotation processing to support generic implementations. <br>This is why I always think twice before creating services for doing something generic. Do we need it for the future? The YAGNI principle is so true and admirable.</p><p><strong>Facades, factories, and what else.</strong><br>Similarly, one thing I noticed in some libraries is the development complexity. Simple business logic sits inside another 10 classes and it is like finding the treasure. Somehow I am a big fan of simple and understandable codebase rather than having all the fancy interfaces and abstract classes thinking about future implementations and making something very generic. Most of the time, as developers or users of these libraries, are interested in the file where the business logic is present and ignores all those files in the stack hierarchy.</p><p><strong>Some do reinvent the wheel.</strong><br>I have some friends working in companies like Revolut and they used to tell me about the development framework used within their company. The framework is very slim and helps the developers to produce more effective code (understanding the concepts before coding) and do not use all the freebie objects strangling around in those fatty libraries. They do strict code checks and have detailed discussions before anyone commits new code into their existing frameworks or libraries to prevent it from being another fat jar. Of course, this kind of development strategy will make you see a lot of duplicate code in the services lying around.</p><p><strong>Libraries and frameworks via GRPC/microservice?</strong><br>When we talk about the library, I want to see something small with a maximum of 5 files in it. And doing just one thing and only built for serving one single purpose. But most of them start by doing one thing but then end up supporting a hundred other things with hundred classes and around 10k lines of code. Often I think about having libraries and frameworks helping us via GRPC or maybe extreme cases acting like a microservice. One fine example would be Zalando’s Nakadi which has a very similar implementation for event streaming.</p><p><strong>High-Level Documentations.</strong><br>Extreme Programming as a methodology for example tells us to have minimum static documentations and have only detailed documentation that will be generated dynamically on every release. Up to an extent, this avoids having outdated pieces of information on what the service does. I agree with this and likes to see libraries with brief documentation on what the library does, what are the side effects if it has any, FAQ’s, etc…</p><p><strong>Real Test cases.</strong><br>I believe after talking about documentations, “real test cases” should be the next ideal sub-topic. Having some test classes or files to depict the working scenarios will help us understand what should be expected as the outcome and what should be provided as input. In this way, “copying and pasting” (developer’s favorite) the sample code from the test cases would also make things super easy and reduce the time spent to integrate.</p><p><strong>Stop auto-configuring for us.</strong><br>Some libraries even go to the extent of making us developers sit idle and do nothing. I hate to use these libraries. Once we integrate them into our service, they start creating and initializing hundreds of objects and among them, only two of them would be relevant to us. Ninety percent of the time, developers don’t care about starting the application in TRACE or DEBUG level to visualize what kind of objects are being created or lying around and do we need them, etc… We only start to analyze them once we end up in OOM error :)</p><p><strong>Think twice before including something.</strong><br>Finally, I would like to emphasize a couple of things. Only include what is necessary and do not accept the big package which is delivered to you for free. Integrating would be easy and we end the day thinking of us being productive on finishing the task.<br>But the reality is you could be making your colleague’s life harder soon. This write-up is not to say all the libraries are bad and is just a waste of time or criticizing some one’s excellent job. I was spitting out my feelings and some hard realities I faced in my past nine development years. Ending in a positive note, I do adore some libraries which are supposed to do one thing smart and efficient :).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17e97dfaef34" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Generate jooq classes using docker containers]]></title>
            <link>https://rohithkunnath.medium.com/generate-jooq-classes-using-docker-containers-365389d8a159?source=rss-412e98e71c5a------2</link>
            <guid isPermaLink="false">https://medium.com/p/365389d8a159</guid>
            <category><![CDATA[liquibase]]></category>
            <category><![CDATA[testcontainer]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Rohith Kunnath]]></dc:creator>
            <pubDate>Fri, 16 Aug 2019 14:03:54 GMT</pubDate>
            <atom:updated>2019-08-16T14:03:54.550Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*uigOaAQO1HsBo1SEEzZOcA.jpeg" /></figure><p><a href="https://dev.to/rohithmenon89/generate-jooq-classes-using-docker-containers-4g62">Generate jooq classes using docker containers</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=365389d8a159" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Elasticsearch(ES) and the hardships]]></title>
            <link>https://rohithkunnath.medium.com/elasticsearch-es-and-the-hardships-c36c8b9c4472?source=rss-412e98e71c5a------2</link>
            <guid isPermaLink="false">https://medium.com/p/c36c8b9c4472</guid>
            <category><![CDATA[nosql]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[development]]></category>
            <dc:creator><![CDATA[Rohith Kunnath]]></dc:creator>
            <pubDate>Fri, 20 Apr 2018 14:41:11 GMT</pubDate>
            <atom:updated>2018-05-30T07:41:58.952Z</atom:updated>
            <content:encoded><![CDATA[<p>I have been working as a Backend Developer for a CRM industry where it is all about searching :). Yep, you are correct, its a system with a lot of data table columns. So the backend framework chosen to support this highly customizable search was Java with spring integration. Yes, your thoughts are right, we chose Elasticsearch(ES) as our datastore. As always there was up votes and down votes, but it was the right decision in the end. It was four years before and the latest stable version was 1.7. Oops completely forgot, let me talk a bit about ES.</p><p>ES is an open source search engine based on Lucene. Certainly, this is not a primary data store, however efficient for systems with searches everywhere. ES is easily scalable. Elasticsearch stores metadata information about index (consider this as a table in SQL) and other data information in files <a href="https://www.elastic.co/blog/found-dive-into-elasticsearch-storage">How is it stored in ES</a>. Nodes are nothing but your servers which can together be called as a cluster. Nodes help to keep replicas of your data. ES keeps data in form of JSON and it particularly follows so-called laws of JSON. If you want to understand more about basics of ES terms <a href="https://www.elastic.co/guide/en/elasticsearch/reference/6.2/_basic_concepts.html">visit here.</a></p><p>Let us get back to what I am actually supposed to talk. Maintaining ES is really hard and costly. What I meant was, since most of the startups do not have so-called DevOps mechanism of auto-scaling and cool kinds of stuff, as the data increases, we need to boot up machines or nodes to keep ES stable. Or else will see “Heap Space Error” since this is completely built using Java, yeah JAVA developers would love this error :).</p><h3>Real Developer Hardships</h3><ol><li>Welcome to the real world of indexing. In elasticsearch you will always hear about indexing. Indexing is nothing but writing all your data to ES. Indexing is a create/update operation in the system as well. There are strategies for indexing and should be carefully configured in ES. That means read/write heavy systems should make appropriate configurations in the ES. Indexing eats your system RAM like an Indian eats spicy food (I just love it). Full indexing should be carefully carried out during peak hours since its a heavy operation.</li><li>No parent-child relationship in 1.7 version. Yes, you can maintain relations in ES between indexes in the later versions. How did we resolve it? We had to store child JSON data inside parent as well. This was kind of duplicating because we had the separate index for child data but still needed to keep in parent index. Then the question could be why again in parent index? Because since the system searches happen in your parent page by filtering for child data. The alternative does separate queries in each index and then link them together. But then the pagination needs to be customized and your <a href="https://en.wikipedia.org/wiki/Minimum_viable_product">MVP</a> model strategy will not work (cannot release soon because of issues all over).</li><li>Awwww then comes to the field analyzers and index analyzers. Whoever says yeah ES is easy, screw them. No I was just joking. But we had a hard time with configurations of metadata. So in ES data is stored either analyzing them or not. For example, if you want a wild card supporting search field, you need to store them in lowercase strings. If you want the search to work with spaces, then you want some other analyzers and the list goes on. The worst part is you cannot change it in an existing index directly, you can only add it to the new index. So we need to reindex data.</li><li>Say your client came up with something new. They want the search to happen for special characters, now you need to create special analyzers and add it to your metadata. Yes, you can just close the index and add it. But if you want to specify the newly created analyzer in any of the fields, then you cannot do it. So again back to first, create the new index and reindex data.</li><li>Now why can’t I store every field with analyzer first itself. You should not do this because the searching and indexing will take more time in fields with analyzers. So always be careful on assigning analyzers to fields.</li><li>Aggregations are the best part of ES. Group by is the best synonym I can find. People love to do aggregations in ES. Yes, but it can even break the system. Since its, a very heavy operation on ES eats up a lot of RAM. Should be carefully built and used. Most of the recommendation systems prefer to use this nowadays.</li><li><a href="https://www.npmjs.com/package/elasticdump">Elasticdump</a> is the nicest library for smooth releases and reindexing. I remember using this at-least 3 times a month.</li><li>Prefer query builders rather than filters. Filters are only applied after fetching data from nodes. Yes, you are right, its much time-consuming.</li></ol><h3>Conclusion</h3><p>Upgrade to the latest versions as soon as possible. The sooner you upgrade, the better and faster your release process will be. Even though all these issues occurred, I loved working with ES and still loves working on the same.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c36c8b9c4472" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mentor — Teacher, supervisor, friend …]]></title>
            <link>https://rohithkunnath.medium.com/mentor-teacher-supervisor-friend-d04bc1b17e26?source=rss-412e98e71c5a------2</link>
            <guid isPermaLink="false">https://medium.com/p/d04bc1b17e26</guid>
            <category><![CDATA[teachers]]></category>
            <category><![CDATA[college]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[mentor]]></category>
            <category><![CDATA[postgraduate]]></category>
            <dc:creator><![CDATA[Rohith Kunnath]]></dc:creator>
            <pubDate>Fri, 25 Nov 2016 00:00:00 GMT</pubDate>
            <atom:updated>2017-02-18T14:23:32.913Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/200/0*ATcHcaf6lQYMue_c.png" /></figure><h3><a href="https://www.linkedin.com/in/rohith-k-p-15748022">Rohith K P</a></h3><p>Follow Following Unfollow Rohith K P</p><p>Sign in to follow this author</p><h3>Software Engineer | Java/Django Backend developer</h3><p>Finally I am done with my Masters degree from Heriot Watt University Dubai (HW). It was one of the wise decision I took on enhancing my technical knowledge. As my bachelors was on Computer Science I was eager to learn more about the core concepts of Networking, Algorithms and data structures.</p><p>Learning is a continuous process which involves hard work, eagerness to learn new things and patience. But I always believed there should be someone who really shows you the right direction on achieving the targets. During my time at HW I was fortunate to meet a lot of technically skilled, caring and loving mentors who really wanted their students to understand what they are learning. Among them I met a great Mentor <strong>Talal Shaikh </strong>a tech enthusiast. I really wanted to thank him for guiding me in the right direction.</p><p>In this busy world, finding a mentor who spends enormous time just to understand their students and helps them to pursue knowledge by showing them the right path is really hard. He did this not to me alone but to many others as well. He will be right there in his office during holidays and post working time on weekdays always with a smile in his face.</p><p>What made me write this article is, soon after the graduation i was only able to spend few minutes with him as there are many other students waiting just to thank him. It was just like a crowded train station, students coming over hugging, shaking hands, asking for references. That point of time I understood the real meaning of the word ‘Mentor’.</p><p>I was very fortunate to undertake my research thesis under him. The thesis was completely new to me, as it was all about Recommender Systems, Natural Language Processing, Anagrams etc. But throughout the whole period he supervised me on choosing related research papers and choosing some great reference books.</p><p>Thank you for everything <strong>Talal Shaikh.</strong> You have added many synonyms to the word Mentor in my dictionary. I really hope you will be shining light for the future graduates as well. :)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/0*nGdO3VVenf9-qvI1.png" /></figure><p>Software Engineer | Java/Django Backend developer</p><h3>Looking for more of the latest headlines on LinkedIn?</h3><p><a href="https://www.linkedin.com/">Discover more stories</a></p><p><em>Originally published at </em><a href="https://www.linkedin.com/pulse/mentor-teacher-supervisor-friend-rohith-k-p"><em>https://www.linkedin.com</em></a><em> on November 25, 2016.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d04bc1b17e26" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>